I am working on a new K8s cluster with Terraform, and having problems installing certificate issuer.
Here is my current setup.
sealed-secrets.tf:
# helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets && helm repo update
resource "helm_release" "sealed_secrests" {
count = 1
name = "sealed-secrets"
namespace = "argocd"
repository = "https://bitnami-labs.github.io/sealed-secrets"
chart = "sealed-secrets"
version = "2.17.1"
timeout = 600
create_namespace = true
cleanup_on_fail = true
dependency_update = true
verify = true
}
cert-manager.tf:
# helm repo add jetstack https://charts.jetstack.io && helm repo update
data "http" "cert_manager_crds_yaml" {
url = "https://github.com/cert-manager/cert-manager/releases/download/v1.16.3/cert-manager.crds.yaml"
}
resource "kubectl_manifest" "cert_manager_crds" {
yaml_body = data.http.cert_manager_crds_yaml.response_body
}
resource "helm_release" "cert_manager" {
count = 1
name = "cert-manager"
namespace = "cert-manager"
repository = "https://charts.jetstack.io"
chart = "cert-manager"
version = "v1.16.3"
timeout = 600
create_namespace = true
cleanup_on_fail = true
dependency_update = true
verify = true
values = [ file("values/cert-manager.yaml") ]
depends_on = [kubectl_manifest.cert_manager_crds]
}
Cloudflare connection
data "local_file" "cloudflare_api_token_secret_file" {
filename = "secrets/cloudflare-api-token-secret.yaml"
}
resource "kubectl_manifest" "cloudflare_api_token_secret" {
yaml_body = data.local_file.cloudflare_api_token_secret_file.content
depends_on = [ helm_release.cert_manager ]
}
data "local_file" "certificate_issuer_manifest_file" {
filename = "manifests/cert-issuer.yaml"
}
resource "kubectl_manifest" "certificate_issuer" {
yaml_body = data.local_file.certificate_issuer_manifest_file.content
depends_on = [ kubectl_manifest.cloudflare_api_token_secret, helm_release.cert_manager ]
}
cloudflare-api-token-secret.yaml:
apiVersion: v1
kind: Secret
metadata:
name: cloudflare-api-token-secret
namespace: ambassador
type: Opaque
stringData:
api-token: ********
cert-issuer.yaml:
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: cloudflare-ambassador-wcard
namespace: ambassador
spec:
# ACME issuer configuration:
# `email` - the email address to be associated with the ACME account (make sure it's a valid one).
# `server` - the URL used to access the ACME server’s directory endpoint.
# `privateKeySecretRef` - Kubernetes Secret to store the automatically generated ACME account private key.
acme:
email: ****@*****.****
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: cloudflare-ambassador-wcard-private
# List of challenge solvers that will be used to solve ACME challenges for the matching domains.
solvers:
- dns01:
cloudflare:
email: ****@*****.****
apiTokenSecretRef:
name: cloudflare-api-token-secret
key: api-token
selector:
dnsNames:
- '*.priz.guru'
- 'priz.guru'
Everything is installed as expected until it gets to the Issuer.
Here is the error that I am getting:
2025-01-20T23:34:51.580-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: 2025/01/20 23:34:51 [ERROR] creating manifest failed: ambassador/cloudflare-ambassador-wcard failed to create kubernetes rest client for update of resource: resource [cert-manager.io/v1/Issuer] isn't valid for cluster, check the APIVersion and Kind fields are valid
2025-01-20T23:34:55.286-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: 2025/01/20 23:34:55 [DEBUG] ambassador/cloudflare-ambassador-wcard Unstructed YAML: map[apiVersion:cert-manager.io/v1 kind:Issuer metadata:map[name:cloudflare-ambassador-wcard namespace:ambassador] spec:map[acme:map[email:alex@priz.guru privateKeySecretRef:map[name:cloudflare-ambassador-wcard-private] server:https://acme-v02.api.letsencrypt.org/directory solvers:[map[dns01:map[cloudflare:map[apiTokenSecretRef:map[key:api-token name:cloudflare-api-token-secret] email:alex@priz.guru]] selector:map[dnsNames:[*.priz.guru priz.guru]]]]]]]
2025-01-20T23:34:55.286-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: 2025/01/20 23:34:55 [DEBUG] ambassador/cloudflare-ambassador-wcard apply kubernetes resource:
2025-01-20T23:34:55.286-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: apiVersion: cert-manager.io/v1
2025-01-20T23:34:55.286-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: kind: Issuer
2025-01-20T23:34:55.286-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: metadata:
2025-01-20T23:34:55.286-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: name: cloudflare-ambassador-wcard
2025-01-20T23:34:55.286-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: namespace: ambassador
2025-01-20T23:34:55.286-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: spec:
2025-01-20T23:34:55.286-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: # ACME issuer configuration:
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: # `email` - the email address to be associated with the ACME account (make sure it's a valid one).
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: # `server` - the URL used to access the ACME server’s directory endpoint.
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: # `privateKeySecretRef` - Kubernetes Secret to store the automatically generated ACME account private key.
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: acme:
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: email: alex@priz.guru
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: server: https://acme-v02.api.letsencrypt.org/directory
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: privateKeySecretRef:
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: name: cloudflare-ambassador-wcard-private
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: # List of challenge solvers that will be used to solve ACME challenges for the matching domains.
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: solvers:
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: - dns01:
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: cloudflare:
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: email: alex@priz.guru
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: apiTokenSecretRef:
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: name: cloudflare-api-token-secret
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: key: api-token
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: selector:
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: dnsNames:
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: - '*.priz.guru'
2025-01-20T23:34:55.287-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: - 'priz.guru'
2025-01-20T23:34:55.359-0800 [DEBUG] provider.terraform-provider-kubectl_v1.19.0: 2025/01/20 23:34:55 [ERROR] creating manifest failed: ambassador/cloudflare-ambassador-wcard failed to create kubernetes rest client for update of resource: resource [cert-manager.io/v1/Issuer] isn't valid for cluster, check the APIVersion and Kind fields are valid
2025-01-20T23:34:55.359-0800 [ERROR] provider.terraform-provider-kubectl_v1.19.0: Response contains error diagnostic: @module=sdk.proto tf_proto_version=5.7 tf_req_id=2faebf9b-8624-79dd-3b0d-cd4b01ce8f47 @caller=github.com/hashicorp/terraform-plugin-go@v0.25.0/tfprotov5/internal/diag/diagnostics.go:58 diagnostic_detail="" diagnostic_severity=ERROR diagnostic_summary="ambassador/cloudflare-ambassador-wcard failed to create kubernetes rest client for update of resource: resource [cert-manager.io/v1/Issuer] isn't valid for cluster, check the APIVersion and Kind fields are valid" tf_provider_addr="" tf_resource_type=kubectl_manifest tf_rpc=ApplyResourceChange timestamp=2025-01-20T23:34:55.359-0800
2025-01-20T23:34:55.583-0800 [DEBUG] State storage *statemgr.Filesystem declined to persist a state snapshot
2025-01-20T23:34:55.583-0800 [ERROR] vertex "kubectl_manifest.certificate_issuer" error: ambassador/cloudflare-ambassador-wcard failed to create kubernetes rest client for update of resource: resource [cert-manager.io/v1/Issuer] isn't valid for cluster, check the APIVersion and Kind fields are valid
╷
│ Error: ambassador/cloudflare-ambassador-wcard failed to create kubernetes rest client for update of resource: resource [cert-manager.io/v1/Issuer] isn't valid for cluster, check the APIVersion and Kind fields are valid
│
│ with kubectl_manifest.certificate_issuer,
│ on cert-manager.tf line 45, in resource "kubectl_manifest" "certificate_issuer":
│ 45: resource "kubectl_manifest" "certificate_issuer" {
│
╵
2025-01-20T23:34:55.823-0800 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2025-01-20T23:34:55.825-0800 [INFO] provider: plugin process exited: plugin=.terraform/providers/registry.terraform.io/gavinbunney/kubectl/1.19.0/darwin_amd64/terraform-provider-kubectl_v1.19.0 id=15363
I have rechecked everything:
- I have the right cert-manager API version supported.
- If I disable the Issuer step, everything else is getting applied properly.
I am not sure where else to dig...
