Migrate CustomResourceDefinition from apiextensions.k8s.io/v1beta1

CustomResourceDefinition is how operators and charts add their own resource types. Kubernetes 1.22 removed apiextensions.k8s.io/v1beta1, and v1 restructures the object: the schema, subresources and printer columns move under each entry in spec.versions, and a structural schema is required.

The error

After the upgrade to 1.22, kubectl apply fails like this:

error: resource mapping not found for name: "<name>" namespace: "<ns>" from "<file>": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
ensure CRDs are installed first

What replaces it

Replacement: apiextensions.k8s.io/v1
Available since: 1.16
Deprecated in: 1.16
Removed in: 1.22

Check a manifest against 1.22, or read the official documentation.

Migration notes

spec.scope is no longer defaulted to Namespaced and must be explicitly specified
spec.version is removed in v1; use spec.versions instead
spec.validation is removed in v1; use spec.versions[*].schema instead
spec.subresources is removed in v1; use spec.versions[*].subresources instead
spec.additionalPrinterColumns is removed in v1; use spec.versions[*].additionalPrinterColumns instead
spec.conversion.webhookClientConfig is moved to spec.conversion.webhook.clientConfig in v1
spec.conversion.conversionReviewVersions is moved to spec.conversion.webhook.conversionReviewVersions in v1
spec.versions[*].schema.openAPIV3Schema is now required when creating v1 CustomResourceDefinition objects, and must be a structural schema
spec.preserveUnknownFields: true is disallowed when creating v1 CustomResourceDefinition objects; it must be specified within schema definitions as x-kubernetes-preserve-unknown-fields: true
In additionalPrinterColumns items, the JSONPath field was renamed to jsonPath in v1

Before and after

Before:

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: backups.example.com
spec:
  group: example.com
  names:
    kind: Backup
    plural: backups
  scope: Namespaced
  version: v1
  validation:
    openAPIV3Schema:
      type: object
      properties:
        spec:
          type: object
          properties:
            schedule:
              type: string

After:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: backups.example.com
spec:
  group: example.com
  names:
    kind: Backup
    plural: backups
  scope: Namespaced
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                schedule:
                  type: string

Helm upgrade fails after a cluster upgrade

If you installed this with Helm, helm upgrade can fail after the cluster is upgraded, even if the chart is already fixed. Which error appears depends on your Helm version and the error text:

Error: UPGRADE FAILED: unable to build kubernetes objects from current release manifest: resource mapping not found for name: "<name>" namespace: "<ns>" from "": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"

Error: UPGRADE FAILED: current release manifest contains removed kubernetes api(s) for this kubernetes version and it is therefore unable to build the kubernetes objects for performing the diff. error from kubernetes: unable to recognize "": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"

Helm compares your chart with the manifest stored with the release. That stored manifest still has the old API, and the cluster can no longer read it.

To check, get the stored manifest, and paste the output into the checker:

helm get manifest <release> -n <namespace>

To fix the stored record, use the helm-mapkubeapis plugin, latest tag v0.6.1. Try --dry-run first. Its README lists support for Helm v3, so check that it suits your Helm version.

helm plugin install https://github.com/helm/helm-mapkubeapis --version v0.6.1
helm mapkubeapis <release> --namespace <namespace> --dry-run

Helm's advice is to move releases to supported APIs before you upgrade the cluster, and not to roll back to a release from before that change.