自定义资源
概述
自定义资源（Custom Resource，CR）是Kubernetes API的扩展机制，允许用户定义自己的资源类型。通过自定义资源定义（Custom Resource Definition，CRD），用户可以扩展Kubernetes API，创建新的资源类型，并像使用内置资源一样使用kubectl操作这些资源。CRD是Kubernetes扩展生态的基石，为Operator、控制器和自定义工作负载提供了强大的扩展能力。
核心概念
1. Custom Resource Definition (CRD)
CRD是定义自定义资源类型的Kubernetes资源：
- 定义资源的名称、作用域、版本
- 定义资源的结构（Schema）
- 定义资源的验证规则
- 定义资源的打印列和子资源
2. Custom Resource (CR)
CR是CRD定义的资源类型的实例：
- 遵循CRD定义的结构
- 可以通过kubectl创建、更新、删除
- 存储在etcd中
- 可以被控制器监控和协调
3. API Group和Version
CRD属于API Group和Version：
- API Group：资源的分组，如
webapp.my.domain - Version：资源的版本，如
v1、v1beta1 - 支持多版本共存和转换
4. Schema
Schema定义资源的结构：
- 使用OpenAPI v3规范
- 定义字段的类型、必填、默认值
- 支持复杂的嵌套结构
- 支持验证规则
CRD结构
基本CRD定义
yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: crontabs.stable.example.com
spec:
group: stable.example.com
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
cronSpec:
type: string
image:
type: string
replicas:
type: integer
scope: Namespaced
names:
plural: crontabs
singular: crontab
kind: CronTab
shortNames:
- ct完整CRD定义
yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: applications.app.example.com
labels:
app.kubernetes.io/name: application-crd
app.kubernetes.io/version: v1.0.0
annotations:
controller-gen.kubebuilder.io/version: v0.11.0
spec:
group: app.example.com
names:
kind: Application
listKind: ApplicationList
plural: applications
singular: application
shortNames:
- app
- apps
categories:
- all
- apps
scope: Namespaced
versions:
- name: v1
served: true
storage: true
deprecated: false
deprecationWarning: ""
schema:
openAPIV3Schema:
description: Application is the Schema for the applications API
type: object
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values.'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to.'
type: string
metadata:
type: object
spec:
description: ApplicationSpec defines the desired state of Application
type: object
required:
- image
properties:
affinity:
description: Affinity rules for pod scheduling
properties:
nodeAffinity:
description: Node affinity rules
properties:
preferredDuringSchedulingIgnoredDuringExecution:
items:
properties:
preference:
properties:
matchExpressions:
items:
properties:
key:
type: string
operator:
type: string
values:
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchFields:
items:
properties:
key:
type: string
operator:
type: string
values:
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
type: object
weight:
format: int32
type: integer
required:
- preference
- weight
type: object
type: array
requiredDuringSchedulingIgnoredDuringExecution:
properties:
nodeSelectorTerms:
items:
properties:
matchExpressions:
items:
properties:
key:
type: string
operator:
type: string
values:
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchFields:
items:
properties:
key:
type: string
operator:
type: string
values:
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
type: object
type: array
required:
- nodeSelectorTerms
type: object
type: object
podAffinity:
type: object
podAntiAffinity:
type: object
type: object
autoscaling:
description: Autoscaling configuration
properties:
enabled:
type: boolean
maxReplicas:
format: int32
maximum: 100
minimum: 1
type: integer
minReplicas:
format: int32
maximum: 100
minimum: 1
type: integer
targetCPUUtilizationPercentage:
format: int32
maximum: 100
minimum: 1
type: integer
targetMemoryUtilizationPercentage:
format: int32
maximum: 100
minimum: 1
type: integer
required:
- enabled
type: object
config:
additionalProperties:
type: string
description: Configuration key-value pairs
type: object
env:
description: Environment variables
items:
properties:
name:
type: string
value:
type: string
valueFrom:
properties:
configMapKeyRef:
properties:
key:
type: string
name:
type: string
optional:
type: boolean
required:
- key
type: object
fieldRef:
properties:
apiVersion:
type: string
fieldPath:
type: string
required:
- fieldPath
type: object
resourceFieldRef:
properties:
containerName:
type: string
divisor:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
resource:
type: string
required:
- resource
type: object
secretKeyRef:
properties:
key:
type: string
name:
type: string
optional:
type: boolean
required:
- key
type: object
type: object
required:
- name
type: object
type: array
image:
description: Container image
pattern: '^[a-zA-Z0-9._-]+(/[a-zA-Z0-9._-]+)*:[a-zA-Z0-9._-]+$'
type: string
imagePullPolicy:
description: Image pull policy
enum:
- Always
- Never
- IfNotPresent
type: string
imagePullSecrets:
description: Image pull secrets
items:
properties:
name:
type: string
required:
- name
type: object
type: array
ingress:
description: Ingress configuration
properties:
annotations:
additionalProperties:
type: string
type: object
enabled:
type: boolean
hosts:
items:
properties:
host:
type: string
paths:
items:
properties:
path:
type: string
pathType:
enum:
- Exact
- Prefix
- ImplementationSpecific
type: string
required:
- path
- pathType
type: object
type: array
required:
- host
type: object
type: array
tls:
items:
properties:
hosts:
items:
type: string
type: array
secretName:
type: string
required:
- secretName
type: object
type: array
required:
- enabled
type: object
livenessProbe:
description: Liveness probe configuration
properties:
exec:
properties:
command:
items:
type: string
type: array
required:
- command
type: object
failureThreshold:
format: int32
minimum: 1
type: integer
httpGet:
properties:
host:
type: string
httpHeaders:
items:
properties:
name:
type: string
value:
type: string
required:
- name
- value
type: object
type: array
path:
type: string
port:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
scheme:
type: string
required:
- port
type: object
initialDelaySeconds:
format: int32
minimum: 0
type: integer
periodSeconds:
format: int32
minimum: 1
type: integer
successThreshold:
format: int32
minimum: 1
type: integer
tcpSocket:
properties:
host:
type: string
port:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
required:
- port
type: object
terminationGracePeriodSeconds:
format: int64
type: integer
timeoutSeconds:
format: int32
minimum: 1
type: integer
type: object
nodeSelector:
additionalProperties:
type: string
description: Node selector for pod scheduling
type: object
port:
description: Container port
format: int32
maximum: 65535
minimum: 1
type: integer
readinessProbe:
description: Readiness probe configuration
properties:
exec:
properties:
command:
items:
type: string
type: array
required:
- command
type: object
failureThreshold:
format: int32
minimum: 1
type: integer
httpGet:
properties:
host:
type: string
httpHeaders:
items:
properties:
name:
type: string
value:
type: string
required:
- name
- value
type: object
type: array
path:
type: string
port:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
scheme:
type: string
required:
- port
type: object
initialDelaySeconds:
format: int32
minimum: 0
type: integer
periodSeconds:
format: int32
minimum: 1
type: integer
successThreshold:
format: int32
minimum: 1
type: integer
tcpSocket:
properties:
host:
type: string
port:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
required:
- port
type: object
terminationGracePeriodSeconds:
format: int64
type: integer
timeoutSeconds:
format: int32
minimum: 1
type: integer
type: object
replicas:
default: 1
description: Number of replicas
format: int32
maximum: 100
minimum: 1
type: integer
resources:
description: Resource requirements
properties:
limits:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
type: object
requests:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
type: object
type: object
service:
description: Service configuration
properties:
annotations:
additionalProperties:
type: string
type: object
port:
format: int32
maximum: 65535
minimum: 1
type: integer
targetPort:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
type:
enum:
- ClusterIP
- NodePort
- LoadBalancer
- ExternalName
type: string
required:
- type
type: object
tolerations:
description: Tolerations for pod scheduling
items:
properties:
effect:
type: string
key:
type: string
operator:
type: string
tolerationSeconds:
format: int64
type: integer
value:
type: string
type: object
type: array
volumeMounts:
description: Volume mounts
items:
properties:
mountPath:
type: string
mountPropagation:
type: string
name:
type: string
readOnly:
type: boolean
subPath:
type: string
subPathExpr:
type: string
required:
- mountPath
- name
type: object
type: array
volumes:
description: Volumes
items:
properties:
awsElasticBlockStore:
properties:
fsType:
type: string
partition:
format: int32
type: integer
readOnly:
type: boolean
volumeID:
type: string
required:
- volumeID
type: object
azureDisk:
properties:
cachingMode:
type: string
diskName:
type: string
diskURI:
type: string
fsType:
type: string
kind:
type: string
readOnly:
type: boolean
required:
- diskName
- diskURI
type: object
azureFile:
properties:
readOnly:
type: boolean
secretName:
type: string
shareName:
type: string
required:
- secretName
- shareName
type: object
cephfs:
properties:
monitors:
items:
type: string
type: array
path:
type: string
readOnly:
type: boolean
secretFile:
type: string
secretRef:
properties:
name:
type: string
type: object
user:
type: string
required:
- monitors
type: object
cinder:
properties:
fsType:
type: string
readOnly:
type: boolean
secretRef:
properties:
name:
type: string
type: object
volumeID:
type: string
required:
- volumeID
type: object
configMap:
properties:
defaultMode:
format: int32
type: integer
items:
items:
properties:
key:
type: string
mode:
format: int32
type: integer
path:
type: string
required:
- key
- path
type: object
type: array
name:
type: string
optional:
type: boolean
type: object
csi:
properties:
driver:
type: string
fsType:
type: string
nodePublishSecretRef:
properties:
name:
type: string
type: object
readOnly:
type: boolean
volumeAttributes:
additionalProperties:
type: string
type: object
required:
- driver
type: object
downwardAPI:
properties:
defaultMode:
format: int32
type: integer
items:
items:
properties:
fieldRef:
properties:
apiVersion:
type: string
fieldPath:
type: string
required:
- fieldPath
type: object
mode:
format: int32
type: integer
path:
type: string
resourceFieldRef:
properties:
containerName:
type: string
divisor:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
resource:
type: string
required:
- resource
type: object
required:
- path
type: object
type: array
type: object
emptyDir:
properties:
medium:
type: string
sizeLimit:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
type: object
ephemeral:
properties:
volumeClaimTemplate:
properties:
metadata:
type: object
spec:
properties:
accessModes:
items:
type: string
type: array
dataSource:
properties:
apiGroup:
type: string
kind:
type: string
name:
type: string
required:
- kind
- name
type: object
dataSourceRef:
properties:
apiGroup:
type: string
kind:
type: string
name:
type: string
required:
- kind
- name
type: object
resources:
properties:
limits:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
type: object
requests:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
type: object
type: object
selector:
properties:
matchExpressions:
items:
properties:
key:
type: string
operator:
type: string
values:
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
type: object
type: object
storageClassName:
type: string
volumeMode:
type: string
volumeName:
type: string
type: object
required:
- spec
type: object
required:
- volumeClaimTemplate
type: object
fc:
properties:
fsType:
type: string
lun:
format: int32
type: integer
readOnly:
type: boolean
targetWWNs:
items:
type: string
type: array
wwids:
items:
type: string
type: array
type: object
flexVolume:
properties:
driver:
type: string
fsType:
type: string
options:
additionalProperties:
type: string
type: object
readOnly:
type: boolean
secretRef:
properties:
name:
type: string
type: object
required:
- driver
type: object
flocker:
properties:
datasetName:
type: string
datasetUUID:
type: string
type: object
gcePersistentDisk:
properties:
fsType:
type: string
partition:
format: int32
type: integer
pdName:
type: string
readOnly:
type: boolean
required:
- pdName
type: object
gitRepo:
properties:
directory:
type: string
repository:
type: string
revision:
type: string
required:
- repository
type: object
glusterfs:
properties:
endpoints:
type: string
path:
type: string
readOnly:
type: boolean
required:
- endpoints
- path
type: object
hostPath:
properties:
path:
type: string
type:
type: string
required:
- path
type: object
iscsi:
properties:
chapAuthDiscovery:
type: boolean
chapAuthSession:
type: boolean
fsType:
type: string
initiatorName:
type: string
iqn:
type: string
iscsiInterface:
type: string
lun:
format: int32
type: integer
portals:
items:
type: string
type: array
readOnly:
type: boolean
secretRef:
properties:
name:
type: string
type: object
targetPortal:
type: string
required:
- iqn
- lun
- targetPortal
type: object
name:
type: string
nfs:
properties:
path:
type: string
readOnly:
type: boolean
server:
type: string
required:
- path
- server
type: object
persistentVolumeClaim:
properties:
claimName:
type: string
readOnly:
type: boolean
required:
- claimName
type: object
photonPersistentDisk:
properties:
fsType:
type: string
pdID:
type: string
required:
- pdID
type: object
portworxVolume:
properties:
fsType:
type: string
readOnly:
type: boolean
volumeID:
type: string
required:
- volumeID
type: object
projected:
properties:
defaultMode:
format: int32
type: integer
sources:
items:
properties:
configMap:
properties:
items:
items:
properties:
key:
type: string
mode:
format: int32
type: integer
path:
type: string
required:
- key
- path
type: object
type: array
name:
type: string
optional:
type: boolean
type: object
downwardAPI:
properties:
items:
items:
properties:
fieldRef:
properties:
apiVersion:
type: string
fieldPath:
type: string
required:
- fieldPath
type: object
mode:
format: int32
type: integer
path:
type: string
resourceFieldRef:
properties:
containerName:
type: string
divisor:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
resource:
type: string
required:
- resource
type: object
required:
- path
type: object
type: array
type: object
secret:
properties:
items:
items:
properties:
key:
type: string
mode:
format: int32
type: integer
path:
type: string
required:
- key
- path
type: object
type: array
name:
type: string
optional:
type: boolean
type: object
serviceAccountToken:
properties:
audience:
type: string
expirationSeconds:
format: int64
type: integer
path:
type: string
required:
- path
type: object
type: object
type: array
required:
- sources
type: object
quobyte:
properties:
group:
type: string
readOnly:
type: boolean
registry:
type: string
tenant:
type: string
user:
type: string
volume:
type: string
required:
- registry
- volume
type: object
rbd:
properties:
fsType:
type: string
image:
type: string
keyring:
type: string
monitors:
items:
type: string
type: array
pool:
type: string
readOnly:
type: boolean
secretRef:
properties:
name:
type: string
type: object
user:
type: string
required:
- image
- monitors
type: object
scaleIO:
properties:
fsType:
type: string
gateway:
type: string
protectionDomain:
type: string
readOnly:
type: boolean
secretRef:
properties:
name:
type: string
type: object
sslEnabled:
type: boolean
storageMode:
type: string
storagePool:
type: string
system:
type: string
volumeName:
type: string
required:
- gateway
- secretRef
- system
type: object
secret:
properties:
defaultMode:
format: int32
type: integer
items:
items:
properties:
key:
type: string
mode:
format: int32
type: integer
path:
type: string
required:
- key
- path
type: object
type: array
optional:
type: boolean
secretName:
type: string
type: object
storageos:
properties:
fsType:
type: string
readOnly:
type: boolean
secretRef:
properties:
apiVersion:
type: string
fieldPath:
type: string
kind:
type: string
name:
type: string
namespace:
type: string
resourceVersion:
type: string
uid:
type: string
type: object
volumeName:
type: string
volumeNamespace:
type: string
type: object
vsphereVolume:
properties:
fsType:
type: string
storagePolicyID:
type: string
storagePolicyName:
type: string
volumePath:
type: string
required:
- volumePath
type: object
required:
- name
type: object
type: array
required:
- image
- port
type: object
status:
description: ApplicationStatus defines the observed state of Application
properties:
availableReplicas:
format: int32
type: integer
conditions:
items:
properties:
lastTransitionTime:
format: date-time
type: string
message:
type: string
observedGeneration:
format: int64
type: integer
reason:
type: string
status:
type: string
type:
type: string
required:
- lastTransitionTime
- status
- type
type: object
type: array
x-kubernetes-list-map-keys:
- type
x-kubernetes-list-type: map
currentReplicas:
format: int32
type: integer
observedGeneration:
format: int64
type: integer
phase:
type: string
readyReplicas:
format: int32
type: integer
replicas:
format: int32
type: integer
unavailableReplicas:
format: int32
type: integer
updatedReplicas:
format: int32
type: integer
required:
- availableReplicas
- currentReplicas
- readyReplicas
- replicas
type: object
required:
- spec
type: object
subresources:
status: {}
scale:
specReplicasPath: .spec.replicas
statusReplicasPath: .status.replicas
labelSelectorPath: .status.labelSelector
additionalPrinterColumns:
- name: Replicas
type: integer
description: The number of replicas
jsonPath: .spec.replicas
- name: Ready
type: integer
description: The number of ready replicas
jsonPath: .status.readyReplicas
- name: Available
type: integer
description: The number of available replicas
jsonPath: .status.availableReplicas
- name: Phase
type: string
description: The phase of the application
jsonPath: .status.phase
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
selectableFields:
- jsonPath: .spec.image
- jsonPath: .status.phase实践示例
示例1：简单的CRD和CR
定义CRD
yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: websites.web.example.com
spec:
group: web.example.com
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
required:
- domain
- image
properties:
domain:
type: string
pattern: '^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$'
image:
type: string
replicas:
type: integer
minimum: 1
maximum: 10
default: 1
tls:
type: boolean
default: false
status:
type: object
properties:
url:
type: string
phase:
type: string
subresources:
status: {}
additionalPrinterColumns:
- name: Domain
type: string
jsonPath: .spec.domain
- name: Replicas
type: integer
jsonPath: .spec.replicas
- name: TLS
type: boolean
jsonPath: .spec.tls
- name: Phase
type: string
jsonPath: .status.phase
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
scope: Namespaced
names:
plural: websites
singular: website
kind: Website
shortNames:
- web创建CR
yaml
apiVersion: web.example.com/v1
kind: Website
metadata:
name: my-website
namespace: default
spec:
domain: my-website.example.com
image: nginx:1.25
replicas: 3
tls: truekubectl操作
bash
kubectl apply -f website-crd.yaml
kubectl get crd websites.web.example.com
kubectl apply -f my-website.yaml
kubectl get websites
kubectl describe website my-website
kubectl get website my-website -o yaml
kubectl patch website my-website --type merge -p '{"spec":{"replicas":5}}'
kubectl delete website my-website示例2：带验证的CRD
定义CRD
yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: databases.db.example.com
spec:
group: db.example.com
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
required:
- spec
properties:
spec:
type: object
required:
- version
- storageSize
- database
properties:
version:
type: string
enum:
- "13"
- "14"
- "15"
- "16"
storageSize:
type: string
pattern: '^[1-9][0-9]*Gi$'
storageClass:
type: string
database:
type: string
minLength: 1
maxLength: 63
pattern: '^[a-z_][a-z0-9_]*$'
replicas:
type: integer
minimum: 1
maximum: 10
default: 1
resources:
type: object
properties:
requests:
type: object
properties:
cpu:
type: string
pattern: '^[0-9]+m?$'
memory:
type: string
pattern: '^[0-9]+Mi?$'
limits:
type: object
properties:
cpu:
type: string
pattern: '^[0-9]+m?$'
memory:
type: string
pattern: '^[0-9]+Mi?$'
backup:
type: object
properties:
enabled:
type: boolean
schedule:
type: string
pattern: '^(@(annually|yearly|monthly|weekly|daily|hourly|reboot))|(@every (\d+(ns|us|ms|s|m|h))+)|((((\d+,)+\d+|(\d+(\/|-)\d+)|\d+|\*) ?){5,7})$'
retention:
type: integer
minimum: 1
maximum: 30
status:
type: object
properties:
phase:
type: string
enum:
- Creating
- Running
- Updating
- Failed
currentVersion:
type: string
currentReplicas:
type: integer
readyReplicas:
type: integer
lastBackup:
type: string
format: date-time
conditions:
type: array
items:
type: object
required:
- type
- status
properties:
type:
type: string
status:
type: string
enum:
- "True"
- "False"
- "Unknown"
reason:
type: string
message:
type: string
lastTransitionTime:
type: string
format: date-time
subresources:
status: {}
scale:
specReplicasPath: .spec.replicas
statusReplicasPath: .status.readyReplicas
additionalPrinterColumns:
- name: Version
type: string
jsonPath: .spec.version
- name: Storage
type: string
jsonPath: .spec.storageSize
- name: Replicas
type: integer
jsonPath: .spec.replicas
- name: Ready
type: integer
jsonPath: .status.readyReplicas
- name: Phase
type: string
jsonPath: .status.phase
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
scope: Namespaced
names:
plural: databases
singular: database
kind: Database
shortNames:
- db创建CR
yaml
apiVersion: db.example.com/v1
kind: Database
metadata:
name: my-database
namespace: default
spec:
version: "15"
storageSize: "50Gi"
storageClass: "fast-ssd"
database: "myapp"
replicas: 3
resources:
requests:
cpu: "500m"
memory: "1Gi"
limits:
cpu: "2000m"
memory: "4Gi"
backup:
enabled: true
schedule: "0 2 * * *"
retention: 7示例3：多版本CRD
定义CRD
yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: queues.messaging.example.com
spec:
group: messaging.example.com
versions:
- name: v1alpha1
served: true
storage: false
deprecated: true
deprecationWarning: "messaging.example.com/v1alpha1 Queue is deprecated; use messaging.example.com/v1 Queue instead"
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
required:
- name
properties:
name:
type: string
maxSize:
type: integer
ttl:
type: integer
status:
type: object
properties:
messageCount:
type: integer
subresources:
status: {}
- name: v1beta1
served: true
storage: false
deprecated: false
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
required:
- queueName
properties:
queueName:
type: string
maxSize:
type: integer
ttl:
type: integer
deadLetterQueue:
type: string
status:
type: object
properties:
messageCount:
type: integer
consumerCount:
type: integer
subresources:
status: {}
additionalPrinterColumns:
- name: Name
type: string
jsonPath: .spec.queueName
- name: Messages
type: integer
jsonPath: .status.messageCount
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
required:
- name
properties:
name:
type: string
minLength: 1
maxLength: 255
maxSize:
type: integer
minimum: 1
maximum: 1000000
ttl:
type: integer
minimum: 1
deadLetterQueue:
type: string
retryPolicy:
type: object
properties:
maxRetries:
type: integer
minimum: 0
maximum: 100
backoffMultiplier:
type: number
minimum: 1
maximum: 10
encryption:
type: object
properties:
enabled:
type: boolean
kmsKeyId:
type: string
status:
type: object
properties:
messageCount:
type: integer
consumerCount:
type: integer
phase:
type: string
conditions:
type: array
items:
type: object
properties:
type:
type: string
status:
type: string
message:
type: string
subresources:
status: {}
scale:
specReplicasPath: .spec.maxSize
statusReplicasPath: .status.messageCount
additionalPrinterColumns:
- name: Name
type: string
jsonPath: .spec.name
- name: Max Size
type: integer
jsonPath: .spec.maxSize
- name: Messages
type: integer
jsonPath: .status.messageCount
- name: Consumers
type: integer
jsonPath: .status.consumerCount
- name: Phase
type: string
jsonPath: .status.phase
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
scope: Namespaced
names:
plural: queues
singular: queue
kind: Queue
shortNames:
- q
conversion:
strategy: Webhook
webhook:
conversionReviewVersions: ["v1", "v1beta1"]
clientConfig:
service:
name: queue-conversion-webhook
namespace: messaging-system
path: /convert版本转换Webhook
go
package main
import (
"encoding/json"
"fmt"
"net/http"
apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
type ConversionReview struct {
apiextensionsv1.ConversionReview
}
func convertHandler(w http.ResponseWriter, r *http.Request) {
var review ConversionReview
if err := json.NewDecoder(r.Body).Decode(&review); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
response := apiextensionsv1.ConversionReview{
TypeMeta: metav1.TypeMeta{
Kind: "ConversionReview",
APIVersion: "apiextensions.k8s.io/v1",
},
Response: &apiextensionsv1.ConversionResponse{
UID: review.Request.UID,
ConvertedObjects: []runtime.Object{},
Result: metav1.Status{
Status: metav1.StatusSuccess,
},
},
}
for _, obj := range review.Request.Objects {
converted, err := convertObject(obj, review.Request.DesiredAPIVersion)
if err != nil {
response.Response.Result.Status = metav1.StatusFailure
response.Response.Result.Message = err.Error()
break
}
response.Response.ConvertedObjects = append(response.Response.ConvertedObjects, converted)
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(response)
}
func convertObject(obj runtime.RawExtension, targetVersion string) (runtime.Object, error) {
switch targetVersion {
case "messaging.example.com/v1":
return convertToV1(obj)
case "messaging.example.com/v1beta1":
return convertToV1beta1(obj)
case "messaging.example.com/v1alpha1":
return convertToV1alpha1(obj)
default:
return nil, fmt.Errorf("unsupported target version: %s", targetVersion)
}
}
func main() {
http.HandleFunc("/convert", convertHandler)
http.ListenAndServeTLS(":8443", "/certs/tls.crt", "/certs/tls.key", nil)
}自定义控制器
控制器框架
使用controller-runtime
go
package main
import (
"context"
"fmt"
"os"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/runtime/serializer"
"k8s.io/client-go/kubernetes/scheme"
"k8s.io/client-go/rest"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/log/zap"
webv1 "example.com/webhook/api/v1"
)
func init() {
_ = webv1.AddToScheme(scheme.Scheme)
}
type WebsiteReconciler struct {
client.Client
Scheme *runtime.Scheme
}
func (r *WebsiteReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
logger := ctrl.LoggerFrom(ctx)
website := &webv1.Website{}
if err := r.Get(ctx, req.NamespacedName, website); err != nil {
return ctrl.Result{}, client.IgnoreNotFound(err)
}
logger.Info("Reconciling Website", "name", website.Name)
return ctrl.Result{}, nil
}
func (r *WebsiteReconciler) SetupWithManager(mgr ctrl.Manager) error {
return ctrl.NewControllerManagedBy(mgr).
For(&webv1.Website{}).
Complete(r)
}
func main() {
ctrl.SetLogger(zap.New(zap.UseDevMode(true)))
mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
Scheme: scheme.Scheme,
MetricsBindAddress: ":8080",
Port: 9443,
})
if err != nil {
setupLog.Error(err, "unable to start manager")
os.Exit(1)
}
if err = (&WebsiteReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
}).SetupWithManager(mgr); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "Website")
os.Exit(1)
}
if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil {
setupLog.Error(err, "problem running manager")
os.Exit(1)
}
}资源验证
使用ValidatingAdmissionWebhook
Webhook定义
yaml
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
name: website-validator
webhooks:
- name: validate.website.web.example.com
rules:
- apiGroups: ["web.example.com"]
apiVersions: ["v1"]
operations: ["CREATE", "UPDATE"]
resources: ["websites"]
scope: "Namespaced"
clientConfig:
service:
name: website-validator
namespace: webhook-system
path: "/validate"
caBundle: LS0tLS...
admissionReviewVersions: ["v1"]
sideEffects: None
timeoutSeconds: 5Webhook实现
go
package main
import (
"encoding/json"
"fmt"
"net/http"
"strings"
admissionv1 "k8s.io/api/admission/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
webv1 "example.com/webhook/api/v1"
)
type AdmissionReview struct {
admissionv1.AdmissionReview
}
func validateHandler(w http.ResponseWriter, r *http.Request) {
var review AdmissionReview
if err := json.NewDecoder(r.Body).Decode(&review); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
response := admissionv1.AdmissionReview{
TypeMeta: metav1.TypeMeta{
Kind: "AdmissionReview",
APIVersion: "admission.k8s.io/v1",
},
Response: &admissionv1.AdmissionResponse{
UID: review.Request.UID,
Allowed: true,
},
}
website := &webv1.Website{}
if err := json.Unmarshal(review.Request.Object.Raw, website); err != nil {
response.Response.Allowed = false
response.Response.Result = &metav1.Status{
Status: metav1.StatusFailure,
Message: err.Error(),
}
} else if err := validateWebsite(website); err != nil {
response.Response.Allowed = false
response.Response.Result = &metav1.Status{
Status: metav1.StatusFailure,
Message: err.Error(),
}
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(response)
}
func validateWebsite(website *webv1.Website) error {
if website.Spec.Domain == "" {
return fmt.Errorf("domain is required")
}
if strings.Contains(website.Spec.Domain, " ") {
return fmt.Errorf("domain cannot contain spaces")
}
if website.Spec.Replicas < 1 || website.Spec.Replicas > 10 {
return fmt.Errorf("replicas must be between 1 and 10")
}
return nil
}
func main() {
http.HandleFunc("/validate", validateHandler)
http.ListenAndServeTLS(":8443", "/certs/tls.crt", "/certs/tls.key", nil)
}使用MutatingAdmissionWebhook
Webhook定义
yaml
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
name: website-mutator
webhooks:
- name: mutate.website.web.example.com
rules:
- apiGroups: ["web.example.com"]
apiVersions: ["v1"]
operations: ["CREATE", "UPDATE"]
resources: ["websites"]
scope: "Namespaced"
clientConfig:
service:
name: website-mutator
namespace: webhook-system
path: "/mutate"
caBundle: LS0tLS...
admissionReviewVersions: ["v1"]
sideEffects: None
timeoutSeconds: 5Webhook实现
go
package main
import (
"encoding/json"
"net/http"
admissionv1 "k8s.io/api/admission/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
webv1 "example.com/webhook/api/v1"
)
func mutateHandler(w http.ResponseWriter, r *http.Request) {
var review AdmissionReview
if err := json.NewDecoder(r.Body).Decode(&review); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
website := &webv1.Website{}
if err := json.Unmarshal(review.Request.Object.Raw, website); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
patches := []PatchOperation{}
if website.Spec.Replicas == 0 {
patches = append(patches, PatchOperation{
Op: "add",
Path: "/spec/replicas",
Value: 1,
})
}
if website.Spec.TLS == nil {
patches = append(patches, PatchOperation{
Op: "add",
Path: "/spec/tls",
Value: false,
})
}
patchBytes, _ := json.Marshal(patches)
response := admissionv1.AdmissionReview{
TypeMeta: metav1.TypeMeta{
Kind: "AdmissionReview",
APIVersion: "admission.k8s.io/v1",
},
Response: &admissionv1.AdmissionResponse{
UID: review.Request.UID,
Allowed: true,
Patch: patchBytes,
PatchType: func() *admissionv1.PatchType {
pt := admissionv1.PatchTypeJSONPatch
return &pt
}(),
},
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(response)
}
type PatchOperation struct {
Op string `json:"op"`
Path string `json:"path"`
Value interface{} `json:"value,omitempty"`
}
func main() {
http.HandleFunc("/mutate", mutateHandler)
http.ListenAndServeTLS(":8443", "/certs/tls.crt", "/certs/tls.key", nil)
}kubectl操作命令
CRD管理
bash
kubectl get crd
kubectl get crd -o wide
kubectl describe crd websites.web.example.com
kubectl get crd websites.web.example.com -o yaml
kubectl apply -f crd.yaml
kubectl edit crd websites.web.example.com
kubectl delete crd websites.web.example.com
kubectl api-resources | grep website
kubectl explain website.spec
kubectl explain website.statusCR管理
bash
kubectl get websites
kubectl get websites -o wide
kubectl get websites -o yaml
kubectl get websites -o json
kubectl describe website my-website
kubectl get website my-website -o jsonpath='{.spec.domain}'
kubectl get website my-website -o jsonpath='{.status.phase}'
kubectl apply -f my-website.yaml
kubectl create -f my-website.yaml
kubectl edit website my-website
kubectl patch website my-website --type merge -p '{"spec":{"replicas":5}}'
kubectl patch website my-website --type strategic -p '{"spec":{"tls":true}}'
kubectl delete website my-website
kubectl delete websites --all
kubectl get websites --selector=app=myapp
kubectl get websites --field-selector=status.phase=Running
kubectl scale website my-website --replicas=5
kubectl get websites --sort-by=.metadata.creationTimestamp
kubectl get websites -o custom-columns=NAME:.metadata.name,DOMAIN:.spec.domain,REPLICAS:.spec.replicas
kubectl get websites -o go-template='{{range .items}}{{.metadata.name}}: {{.spec.domain}}{{"\n"}}{{end}}'Webhook管理
bash
kubectl get validatingwebhookconfigurations
kubectl describe validatingwebhookconfiguration website-validator
kubectl get mutatingwebhookconfigurations
kubectl describe mutatingwebhookconfiguration website-mutator
kubectl logs -n webhook-system -l app=website-validator
kubectl get events -n webhook-system --sort-by='.lastTimestamp'故障排查指南
问题1：CRD创建失败
症状
bash
Error from server (BadRequest): error when creating "crd.yaml": CustomResourceDefinition.apiextensions.k8s.io "websites.web.example.com" is invalid: spec.names.singular: Invalid value: "website": must be unique排查步骤
bash
kubectl get crd | grep website
kubectl describe crd websites.web.example.com
kubectl logs -n kube-system -l component=kube-apiserver
kubectl api-resources | grep website解决方案
- 检查CRD名称是否已存在
- 验证API Group和Version格式
- 确认Schema定义正确
- 检查字段命名规范
问题2：CR创建失败
症状
bash
Error from server (BadRequest): error when creating "website.yaml": Website.web.example.com "my-website" is invalid: spec.domain: Invalid value: "my website": spec.domain in body should match '^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$'排查步骤
bash
kubectl explain website.spec
kubectl get crd websites.web.example.com -o jsonpath='{.spec.versions[0].schema.openAPIV3Schema}'
kubectl apply -f website.yaml --dry-run=client
kubectl apply -f website.yaml --dry-run=server解决方案
- 检查字段值是否符合Schema定义
- 验证必填字段是否提供
- 确认枚举值是否正确
- 检查正则表达式匹配
问题3：Webhook验证失败
症状
bash
Error from server (InternalError): error when creating "website.yaml": Internal error occurred: failed calling webhook "validate.website.web.example.com": Post "https://website-validator.webhook-system.svc:443/validate?timeout=5s": dial tcp 10.96.100.200:443: connect: connection refused排查步骤
bash
kubectl get svc -n webhook-system
kubectl get pods -n webhook-system -l app=website-validator
kubectl logs -n webhook-system -l app=website-validator
kubectl describe validatingwebhookconfiguration website-validator
kubectl exec -n webhook-system -it <pod-name> -- curl -k https://localhost:8443/validate解决方案
- 检查Webhook服务是否正常运行
- 验证证书配置是否正确
- 确认Webhook路径是否正确
- 检查网络策略限制
问题4：版本转换失败
症状
bash
Error from server: error when creating "queue.yaml": conversion webhook for messaging.example.com/v1, Kind=Queue failed: Post "https://queue-conversion-webhook.messaging-system.svc:443/convert?timeout=30s": no such host排查步骤
bash
kubectl get crd queues.messaging.example.com -o yaml | grep -A 10 conversion
kubectl get svc -n messaging-system
kubectl get pods -n messaging-system -l app=queue-conversion-webhook
kubectl logs -n messaging-system -l app=queue-conversion-webhook解决方案
- 检查转换Webhook服务配置
- 验证Webhook服务是否运行
- 确认版本转换逻辑正确
- 检查Webhook证书配置
问题5：状态更新失败
症状
bash
kubectl get website my-website -o yaml
status:
phase: ""排查步骤
bash
kubectl describe website my-website
kubectl get website my-website -o jsonpath='{.metadata.resourceVersion}'
kubectl auth can-i update websites/status --as=system:serviceaccount:default:default
kubectl logs -n controller-system -l app=website-controller解决方案
- 检查控制器是否正常运行
- 验证RBAC权限配置
- 确认状态子资源定义
- 检查控制器协调逻辑
最佳实践
1. CRD设计最佳实践
命名规范
yaml
metadata:
name: <plural>.<group>
spec:
group: <domain>.<organization>
names:
kind: <Kind>
plural: <plural>
singular: <singular>
shortNames:
- <short>Schema设计
yaml
schema:
openAPIV3Schema:
type: object
description: "Clear description of the resource"
properties:
spec:
type: object
description: "Specification of the resource"
required:
- requiredField
properties:
requiredField:
type: string
description: "Description of the field"
minLength: 1
maxLength: 255
optionalField:
type: string
description: "Optional field with default"
default: "default-value"2. 版本管理最佳实践
版本策略
yaml
versions:
- name: v1alpha1
served: true
storage: false
deprecated: true
deprecationWarning: "Use v1 instead"
- name: v1beta1
served: true
storage: false
- name: v1
served: true
storage: true版本转换
yaml
conversion:
strategy: Webhook
webhook:
conversionReviewVersions: ["v1", "v1beta1"]
clientConfig:
service:
name: conversion-webhook
namespace: system3. 验证最佳实践
Schema验证
yaml
properties:
email:
type: string
format: email
pattern: '^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'
port:
type: integer
minimum: 1
maximum: 65535
replicas:
type: integer
minimum: 1
maximum: 100
default: 1Webhook验证
go
func validateResource(obj runtime.Object) error {
switch o := obj.(type) {
case *MyResource:
if o.Spec.RequiredField == "" {
return fmt.Errorf("requiredField is required")
}
if err := validateEmail(o.Spec.Email); err != nil {
return err
}
}
return nil
}4. 状态管理最佳实践
状态定义
yaml
status:
properties:
phase:
type: string
enum:
- Pending
- Running
- Failed
- Unknown
conditions:
type: array
items:
type: object
required:
- type
- status
properties:
type:
type: string
status:
type: string
enum:
- "True"
- "False"
- "Unknown"
reason:
type: string
message:
type: string
lastTransitionTime:
type: string
format: date-time状态更新
go
func (r *MyReconciler) updateStatus(ctx context.Context, obj *MyResource) error {
obj.Status.ObservedGeneration = obj.Generation
obj.Status.LastUpdateTime = metav1.Now()
if obj.Status.Phase == "" {
obj.Status.Phase = "Pending"
}
return r.Status().Update(ctx, obj)
}5. 性能优化最佳实践
打印列优化
yaml
additionalPrinterColumns:
- name: Replicas
type: integer
description: "Number of replicas"
jsonPath: .spec.replicas
priority: 0
- name: Phase
type: string
description: "Current phase"
jsonPath: .status.phase
priority: 0
- name: Internal
type: string
jsonPath: .status.internalInfo
priority: 1选择器字段
yaml
selectableFields:
- jsonPath: .spec.image
- jsonPath: .status.phase6. 文档最佳实践
CRD文档
yaml
schema:
openAPIV3Schema:
type: object
description: |
MyResource is a custom resource that manages application deployments.
## Overview
MyResource provides a simplified interface for deploying applications
with built-in best practices for production workloads.
## Features
- Automatic scaling
- Rolling updates
- Health monitoring
- Resource management
## Example
```yaml
apiVersion: app.example.com/v1
kind: MyResource
metadata:
name: my-app
spec:
replicas: 3
image: nginx:1.25
```
properties:
spec:
type: object
description: "Specification of the desired behavior"总结
本章详细介绍了自定义资源的定义和管理方法：
- 基础概念: 掌握了CRD、CR、API Group等核心概念
- CRD定义: 学会了完整的CRD结构定义
- 验证机制: 理解了Schema验证和Webhook验证
- 版本管理: 掌握了多版本CRD和版本转换
- 控制器开发: 学会了自定义控制器的开发
- 故障排查: 掌握了常见问题的诊断和解决方法
自定义资源是Kubernetes扩展的核心机制，为Operator模式和高级应用管理提供了基础。
下一步学习
- Operator模式 - 深入学习Operator开发
- 自动扩缩容 - 配置HPA/VPA自动扩缩容
- 服务网格 - 学习Istio服务网格架构
- Helm Charts - 回顾Helm包管理器使用