Skip to content

ConfigMap与Secret

概述

ConfigMap和Secret是Kubernetes中用于配置管理的核心资源对象。ConfigMap用于存储非敏感的配置数据,而Secret用于存储敏感信息(如密码、证书、密钥等)。它们实现了配置与容器镜像的解耦,使应用配置更加灵活和安全。

核心概念

ConfigMap

  • 存储非敏感的配置数据(键值对)
  • 支持多种数据格式(配置文件、环境变量、命令行参数)
  • 可以被多个Pod共享使用
  • 更新后需要重启Pod才能生效

Secret

  • 存储敏感信息(密码、证书、密钥等)
  • 数据以Base64编码存储
  • 支持多种类型(Opaque、docker-registry、tls等)
  • 可以加密存储(etcd加密)
  • 挂载到Pod时存储在tmpfs(内存文件系统)中

使用场景

类型ConfigMapSecret
应用配置文件
环境变量
数据库连接字符串
API密钥
TLS证书
镜像仓库凭证

ConfigMap详解

创建方式

1. 从字面值创建

yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: default
data:
  app.name: "my-application"
  app.version: "1.0.0"
  app.mode: "production"
  database.host: "mysql.database.svc.cluster.local"
  database.port: "3306"

2. 从文件创建

yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config
  namespace: default
data:
  nginx.conf: |
    user nginx;
    worker_processes auto;
    error_log /var/log/nginx/error.log;
    pid /run/nginx.pid;
    
    events {
        worker_connections 1024;
    }
    
    http {
        log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                          '$status $body_bytes_sent "$http_referer" '
                          '"$http_user_agent" "$http_x_forwarded_for"';
        
        access_log  /var/log/nginx/access.log  main;
        
        sendfile            on;
        tcp_nopush          on;
        tcp_nodelay         on;
        keepalive_timeout   65;
        types_hash_max_size 2048;
        
        include             /etc/nginx/mime.types;
        default_type        application/octet-stream;
        
        include /etc/nginx/conf.d/*.conf;
    }

3. 从目录创建

bash
# 从目录创建ConfigMap
kubectl create configmap app-config --from-file=config-dir/

使用方式

1. 环境变量方式

yaml
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: my-app:1.0
    env:
    - name: APP_NAME
      valueFrom:
        configMapKeyRef:
          name: app-config
          key: app.name
    - name: APP_VERSION
      valueFrom:
        configMapKeyRef:
          name: app-config
          key: app.version
    - name: DATABASE_HOST
      valueFrom:
        configMapKeyRef:
          name: app-config
          key: database.host

2. 环境变量注入所有键值

yaml
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: my-app:1.0
    envFrom:
    - configMapRef:
        name: app-config
      prefix: CONFIG_

3. 挂载为文件

yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
spec:
  containers:
  - name: nginx
    image: nginx:1.21
    volumeMounts:
    - name: config-volume
      mountPath: /etc/nginx/nginx.conf
      subPath: nginx.conf
  volumes:
  - name: config-volume
    configMap:
      name: nginx-config

4. 挂载整个目录

yaml
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: my-app:1.0
    volumeMounts:
    - name: config-volume
      mountPath: /etc/app/config
      readOnly: true
  volumes:
  - name: config-volume
    configMap:
      name: app-config
      items:
      - key: app.properties
        path: app.properties
      - key: database.properties
        path: database.properties

Secret详解

Secret类型

类型说明用途
Opaque通用Secret存储任意数据
kubernetes.io/dockerconfigjsonDocker镜像仓库凭证拉取私有镜像
kubernetes.io/tlsTLS证书HTTPS配置
kubernetes.io/service-account-tokenServiceAccount令牌API认证
kubernetes.io/basic-auth基本认证用户名密码认证
kubernetes.io/ssh-authSSH认证SSH密钥认证

创建方式

1. Opaque类型Secret

yaml
apiVersion: v1
kind: Secret
metadata:
  name: db-secret
  namespace: default
type: Opaque
stringData:
  username: "admin"
  password: "StrongPassword123!"
  database: "appdb"
  host: "mysql.database.svc.cluster.local"
  port: "3306"

2. Docker镜像仓库Secret

yaml
apiVersion: v1
kind: Secret
metadata:
  name: docker-registry-secret
  namespace: default
type: kubernetes.io/dockerconfigjson
stringData:
  .dockerconfigjson: |
    {
      "auths": {
        "https://index.docker.io/v1/": {
          "username": "dockeruser",
          "password": "dockerpassword",
          "email": "user@example.com",
          "auth": "ZG9ja2VydXNlcjpkb2NrZXJwYXNzd29yZA=="
        },
        "registry.example.com": {
          "username": "admin",
          "password": "registrypassword",
          "auth": "YWRtaW46cmVnaXN0cnlwYXNzd29yZA=="
        }
      }
    }

3. TLS证书Secret

yaml
apiVersion: v1
kind: Secret
metadata:
  name: tls-secret
  namespace: default
type: kubernetes.io/tls
stringData:
  tls.crt: |
    -----BEGIN CERTIFICATE-----
    MIIDXTCCAkWgAwIBAgIJAJC1HiIAZAiUMA0GCSqGSIb3Qq87teleQge
    ...
    -----END CERTIFICATE-----
  tls.key: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIEpQIBAAKCAQEAz7Z8z9J5dX9Yv8Pq7Z8z9J5dX9Yv8Pq7Z8z9J5dX9Y
    ...
    -----END RSA PRIVATE KEY-----

4. SSH认证Secret

yaml
apiVersion: v1
kind: Secret
metadata:
  name: ssh-secret
  namespace: default
type: kubernetes.io/ssh-auth
stringData:
  ssh-privatekey: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIEpQIBAAKCAQEAz7Z8z9J5dX9Yv8Pq7Z8z9J5dX9Yv8Pq7Z8z9J5dX9Y
    ...
    -----END RSA PRIVATE KEY-----
  ssh-publickey: |
    ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHt...

使用方式

1. 环境变量方式

yaml
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: my-app:1.0
    env:
    - name: DB_USERNAME
      valueFrom:
        secretKeyRef:
          name: db-secret
          key: username
    - name: DB_PASSWORD
      valueFrom:
        secretKeyRef:
          name: db-secret
          key: password
    - name: DB_HOST
      valueFrom:
        secretKeyRef:
          name: db-secret
          key: host

2. 挂载为文件

yaml
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: my-app:1.0
    volumeMounts:
    - name: secret-volume
      mountPath: /etc/secrets
      readOnly: true
  volumes:
  - name: secret-volume
    secret:
      secretName: db-secret
      items:
      - key: username
        path: db-username
      - key: password
        path: db-password

3. 拉取私有镜像

yaml
apiVersion: v1
kind: Pod
metadata:
  name: private-image-pod
spec:
  imagePullSecrets:
  - name: docker-registry-secret
  containers:
  - name: app
    image: registry.example.com/my-app:1.0

4. Ingress使用TLS证书

yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-ingress
  namespace: default
spec:
  tls:
  - hosts:
    - app.example.com
    secretName: tls-secret
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app-service
            port:
              number: 80

kubectl操作命令

ConfigMap命令

bash
# 创建ConfigMap
kubectl create configmap app-config --from-literal=key1=value1 --from-literal=key2=value2
kubectl create configmap app-config --from-file=config.properties
kubectl create configmap app-config --from-file=config-dir/
kubectl apply -f configmap.yaml

# 查看ConfigMap
kubectl get configmap
kubectl get cm
kubectl describe cm app-config
kubectl get cm app-config -o yaml

# 编辑ConfigMap
kubectl edit cm app-config

# 删除ConfigMap
kubectl delete cm app-config

# 从ConfigMap创建Pod
kubectl create configmap app-config --from-literal=app.name=myapp --dry-run=client -o yaml | kubectl apply -f -

# 导出ConfigMap
kubectl get cm app-config -o jsonpath='{.data}'

Secret命令

bash
# 创建Secret
kubectl create secret generic db-secret --from-literal=username=admin --from-literal=password=StrongPassword123
kubectl create secret generic db-secret --from-file=username.txt --from-file=password.txt
kubectl create secret docker-registry docker-secret --docker-server=registry.example.com --docker-username=admin --docker-password=password --docker-email=admin@example.com
kubectl create secret tls tls-secret --cert=path/to/tls.crt --key=path/to/tls.key
kubectl apply -f secret.yaml

# 查看Secret
kubectl get secrets
kubectl describe secret db-secret
kubectl get secret db-secret -o yaml

# 解码Secret
kubectl get secret db-secret -o jsonpath='{.data.password}' | base64 --decode

# 编辑Secret
kubectl edit secret db-secret

# 删除Secret
kubectl delete secret db-secret

# 查看Secret类型
kubectl get secrets --field-selector type=Opaque

故障排查命令

bash
# 查看Pod环境变量
kubectl exec -it pod-name -- env | grep -i config

# 查看挂载的ConfigMap文件
kubectl exec -it pod-name -- ls -la /etc/config/
kubectl exec -it pod-name -- cat /etc/config/app.properties

# 查看挂载的Secret文件
kubectl exec -it pod-name -- ls -la /etc/secrets/
kubectl exec -it pod-name -- cat /etc/secrets/db-password

# 检查ConfigMap是否存在
kubectl get cm app-config -o jsonpath='{.data}'

# 检查Secret是否存在
kubectl get secret db-secret -o jsonpath='{.data}'

# 查看Pod事件
kubectl describe pod pod-name | grep -A 10 Events

真实场景实践示例

场景1:多环境配置管理

需求:为开发、测试、生产环境配置不同的应用参数。

yaml
# 1. 开发环境配置
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: development
data:
  APP_ENV: "development"
  LOG_LEVEL: "debug"
  DATABASE_URL: "jdbc:mysql://dev-mysql:3306/appdb"
  REDIS_HOST: "dev-redis"
  REDIS_PORT: "6379"
  API_TIMEOUT: "30"
  MAX_CONNECTIONS: "10"
---
apiVersion: v1
kind: Secret
metadata:
  name: app-secret
  namespace: development
type: Opaque
stringData:
  DATABASE_USERNAME: "dev_user"
  DATABASE_PASSWORD: "dev_password123"
  REDIS_PASSWORD: "redis_dev_pass"
  API_KEY: "dev_api_key_12345"
---
# 2. 测试环境配置
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: testing
data:
  APP_ENV: "testing"
  LOG_LEVEL: "info"
  DATABASE_URL: "jdbc:mysql://test-mysql:3306/appdb"
  REDIS_HOST: "test-redis"
  REDIS_PORT: "6379"
  API_TIMEOUT: "60"
  MAX_CONNECTIONS: "50"
---
apiVersion: v1
kind: Secret
metadata:
  name: app-secret
  namespace: testing
type: Opaque
stringData:
  DATABASE_USERNAME: "test_user"
  DATABASE_PASSWORD: "test_password456"
  REDIS_PASSWORD: "redis_test_pass"
  API_KEY: "test_api_key_67890"
---
# 3. 生产环境配置
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: production
data:
  APP_ENV: "production"
  LOG_LEVEL: "error"
  DATABASE_URL: "jdbc:mysql://prod-mysql:3306/appdb"
  REDIS_HOST: "prod-redis"
  REDIS_PORT: "6379"
  API_TIMEOUT: "120"
  MAX_CONNECTIONS: "200"
---
apiVersion: v1
kind: Secret
metadata:
  name: app-secret
  namespace: production
type: Opaque
stringData:
  DATABASE_USERNAME: "prod_user"
  DATABASE_PASSWORD: "prod_strong_password_789"
  REDIS_PASSWORD: "redis_prod_strong_pass"
  API_KEY: "prod_api_key_abcdef123456"
---
# 4. 应用部署(通用模板)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: app
        image: my-app:1.0
        ports:
        - containerPort: 8080
        envFrom:
        - configMapRef:
            name: app-config
        - secretRef:
            name: app-secret
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        resources:
          requests:
            cpu: 100m
            memory: 256Mi
          limits:
            cpu: 500m
            memory: 512Mi
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5

场景2:微服务配置中心

需求:为多个微服务提供统一的配置管理。

yaml
# 1. 数据库服务配置
apiVersion: v1
kind: ConfigMap
metadata:
  name: mysql-config
  namespace: microservices
data:
  my.cnf: |
    [mysqld]
    innodb_buffer_pool_size = 1G
    max_connections = 500
    query_cache_size = 0
    query_cache_type = 0
    slow_query_log = 1
    slow_query_log_file = /var/log/mysql/slow.log
    long_query_time = 2
    log_error = /var/log/mysql/error.log
    binlog_format = ROW
    server_id = 1
    log_bin = mysql-bin
    expire_logs_days = 7
---
apiVersion: v1
kind: Secret
metadata:
  name: mysql-secret
  namespace: microservices
type: Opaque
stringData:
  root-password: "MySQLRootPassword123!"
  replication-user: "repl_user"
  replication-password: "ReplPassword456!"
---
# 2. Redis服务配置
apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-config
  namespace: microservices
data:
  redis.conf: |
    bind 0.0.0.0
    port 6379
    daemonize no
    appendonly yes
    appendfsync everysec
    maxmemory 2gb
    maxmemory-policy allkeys-lru
    save 900 1
    save 300 10
    save 60 10000
    tcp-backlog 511
    tcp-keepalive 300
---
apiVersion: v1
kind: Secret
metadata:
  name: redis-secret
  namespace: microservices
type: Opaque
stringData:
  redis-password: "RedisStrongPassword789!"
---
# 3. 应用服务配置
apiVersion: v1
kind: ConfigMap
metadata:
  name: user-service-config
  namespace: microservices
data:
  application.yml: |
    server:
      port: 8080
      servlet:
        context-path: /api/users
    
    spring:
      datasource:
        url: jdbc:mysql://mysql:3306/userdb
        driver-class-name: com.mysql.cj.jdbc.Driver
        hikari:
          maximum-pool-size: 20
          minimum-idle: 5
          connection-timeout: 30000
      
      redis:
        host: redis
        port: 6379
        timeout: 3000
        lettuce:
          pool:
            max-active: 8
            max-idle: 8
            min-idle: 0
      
      jpa:
        hibernate:
          ddl-auto: update
        show-sql: false
        properties:
          hibernate:
            format_sql: true
            dialect: org.hibernate.dialect.MySQL8Dialect
    
    logging:
      level:
        root: INFO
        com.example.userservice: DEBUG
      pattern:
        console: "%d{yyyy-MM-dd HH:mm:ss} - %msg%n"
    
    management:
      endpoints:
        web:
          exposure:
            include: health,info,metrics
      metrics:
        export:
          prometheus:
            enabled: true
---
apiVersion: v1
kind: Secret
metadata:
  name: user-service-secret
  namespace: microservices
type: Opaque
stringData:
  application-secret.yml: |
    spring:
      datasource:
        username: user_service
        password: UserServicePassword123
      redis:
        password: RedisStrongPassword789!
      security:
        jwt:
          secret: JWTSecretKeyForUserService123456789
          expiration: 86400000
---
# 4. 部署User Service
apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service
  namespace: microservices
spec:
  replicas: 3
  selector:
    matchLabels:
      app: user-service
  template:
    metadata:
      labels:
        app: user-service
    spec:
      containers:
      - name: user-service
        image: user-service:1.0
        ports:
        - containerPort: 8080
        volumeMounts:
        - name: config-volume
          mountPath: /config
          readOnly: true
        - name: secret-volume
          mountPath: /config/secrets
          readOnly: true
        env:
        - name: SPRING_CONFIG_LOCATION
          value: "classpath:/application.yml,/config/application.yml,/config/secrets/application-secret.yml"
        resources:
          requests:
            cpu: 200m
            memory: 512Mi
          limits:
            cpu: 1000m
            memory: 1Gi
      volumes:
      - name: config-volume
        configMap:
          name: user-service-config
      - name: secret-volume
        secret:
          secretName: user-service-secret

场景3:应用配置热更新

需求:应用配置更新后,自动触发Pod滚动更新。

yaml
# 1. ConfigMap版本控制
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config-v1
  namespace: default
data:
  version: "1.0"
  feature.enabled: "false"
  cache.size: "100"
---
# 2. Deployment使用配置版本标签
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
        config-version: "v1"
    spec:
      containers:
      - name: app
        image: my-app:1.0
        envFrom:
        - configMapRef:
            name: app-config-v1
        volumeMounts:
        - name: config-volume
          mountPath: /etc/config
          readOnly: true
      volumes:
      - name: config-volume
        configMap:
          name: app-config-v1
---
# 3. 配置更新脚本
apiVersion: batch/v1
kind: Job
metadata:
  name: update-config
  namespace: default
spec:
  template:
    spec:
      serviceAccountName: config-updater
      containers:
      - name: kubectl
        image: bitnami/kubectl:latest
        command:
        - /bin/sh
        - -c
        - |
          # 创建新版本ConfigMap
          kubectl apply -f - <<EOF
          apiVersion: v1
          kind: ConfigMap
          metadata:
            name: app-config-v2
          data:
            version: "2.0"
            feature.enabled: "true"
            cache.size: "200"
          EOF
          
          # 更新Deployment使用新配置
          kubectl set image deployment/my-app app=my-app:1.0 --record
          kubectl patch deployment my-app -p '{"spec":{"template":{"metadata":{"labels":{"config-version":"v2"}},"spec":{"volumes":[{"name":"config-volume","configMap":{"name":"app-config-v2"}}]}}}}'
          
          # 等待滚动更新完成
          kubectl rollout status deployment/my-app
          
          # 删除旧版本ConfigMap
          kubectl delete configmap app-config-v1
      restartPolicy: OnFailure
---
# 4. Reloader自动更新(使用第三方工具)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: default
  annotations:
    config.reloader.stakater.com/reload: "app-config"
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: app
        image: my-app:1.0
        envFrom:
        - configMapRef:
            name: app-config

场景4:敏感信息加密管理

需求:使用外部密钥管理系统(如Vault)管理敏感信息。

yaml
# 1. ExternalSecrets配置(使用External Secrets Operator)
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: vault-backend
  namespace: default
spec:
  provider:
    vault:
      server: "https://vault.example.com"
      path: "secret"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "my-app-role"
---
# 2. 从Vault同步Secret
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: vault-secret
  namespace: default
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  target:
    name: app-secret
    creationPolicy: Owner
  data:
  - secretKey: database-password
    remoteRef:
      key: database
      property: password
  - secretKey: api-key
    remoteRef:
      key: api
      property: key
  - secretKey: jwt-secret
    remoteRef:
      key: jwt
      property: secret
---
# 3. SealedSecret配置(使用Sealed Secrets)
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: my-secret
  namespace: default
spec:
  encryptedData:
    database-password: AgBy3i4OJSWK+diTyGqc7U8z/ihPe...
    api-key: AgBy3i4OJSWK+diTyGqc7U8z/ihPe...
    jwt-secret: AgBy3i4OJSWK+diTyGqc7U8z/ihPe...
---
# 4. 应用使用Secret
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: app
        image: my-app:1.0
        env:
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: app-secret
              key: database-password
        - name: API_KEY
          valueFrom:
            secretKeyRef:
              name: app-secret
              key: api-key
        - name: JWT_SECRET
          valueFrom:
            secretKeyRef:
              name: app-secret
              key: jwt-secret

场景5:配置文件模板化

需求:根据环境变量动态生成配置文件。

yaml
# 1. ConfigMap模板
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-template
  namespace: default
data:
  nginx.conf.template: |
    user nginx;
    worker_processes {{WORKER_PROCESSES}};
    error_log /var/log/nginx/error.log {{LOG_LEVEL}};
    pid /run/nginx.pid;
    
    events {
        worker_connections {{WORKER_CONNECTIONS}};
    }
    
    http {
        log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                          '$status $body_bytes_sent "$http_referer" '
                          '"$http_user_agent"';
        
        access_log  /var/log/nginx/access.log  main;
        
        sendfile            on;
        tcp_nopush          on;
        keepalive_timeout   {{KEEPALIVE_TIMEOUT}};
        
        upstream backend {
            {{BACKEND_SERVERS}}
        }
        
        server {
            listen {{SERVER_PORT}};
            server_name {{SERVER_NAME}};
            
            location / {
                proxy_pass http://backend;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
            }
        }
    }
---
# 2. 环境变量配置
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-env-config
  namespace: default
data:
  WORKER_PROCESSES: "auto"
  LOG_LEVEL: "warn"
  WORKER_CONNECTIONS: "2048"
  KEEPALIVE_TIMEOUT: "65"
  SERVER_PORT: "80"
  SERVER_NAME: "example.com"
  BACKEND_SERVERS: |
    server backend1:8080;
    server backend2:8080;
    server backend3:8080;
---
# 3. InitContainer生成配置
apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
  namespace: default
spec:
  initContainers:
  - name: config-generator
    image: alpine:latest
    command:
    - /bin/sh
    - -c
    - |
      apk add --no-cache envsubst
      cat /templates/nginx.conf.template | envsubst > /config/nginx.conf
      echo "Configuration generated successfully"
    volumeMounts:
    - name: template-volume
      mountPath: /templates
      readOnly: true
    - name: config-volume
      mountPath: /config
    envFrom:
    - configMapRef:
        name: nginx-env-config
  containers:
  - name: nginx
    image: nginx:1.21
    ports:
    - containerPort: 80
    volumeMounts:
    - name: config-volume
      mountPath: /etc/nginx/nginx.conf
      subPath: nginx.conf
      readOnly: true
  volumes:
  - name: template-volume
    configMap:
      name: nginx-template
  - name: config-volume
    emptyDir: {}

故障排查指南

常见问题及解决方案

1. ConfigMap/Secret未挂载成功

症状

Warning  FailedMount  Unable to mount volumes for pod "app-pod": timeout expired waiting for volumes to attach or mount

排查步骤

bash
# 1. 检查ConfigMap/Secret是否存在
kubectl get configmap app-config
kubectl get secret app-secret

# 2. 查看Pod事件
kubectl describe pod app-pod

# 3. 检查挂载路径
kubectl exec -it app-pod -- ls -la /etc/config/

# 4. 查看Pod配置
kubectl get pod app-pod -o yaml | grep -A 10 volumes

可能原因

  • ConfigMap/Secret不存在
  • 名称拼写错误
  • 命名空间不匹配
  • 权限不足

解决方案

yaml
# 确保ConfigMap/Secret存在且在正确的命名空间
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: default  # 必须与Pod在同一个命名空间
data:
  key: value
---
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
  namespace: default  # 必须与ConfigMap在同一个命名空间
spec:
  containers:
  - name: app
    image: my-app:1.0
    volumeMounts:
    - name: config-volume
      mountPath: /etc/config
  volumes:
  - name: config-volume
    configMap:
      name: app-config  # 名称必须正确

2. 环境变量未注入

症状:应用读取不到环境变量

排查步骤

bash
# 1. 检查Pod环境变量
kubectl exec -it app-pod -- env | grep -i app

# 2. 检查ConfigMap内容
kubectl get configmap app-config -o yaml

# 3. 查看Pod配置
kubectl get pod app-pod -o yaml | grep -A 20 env

# 4. 检查应用日志
kubectl logs app-pod | grep -i environment

可能原因

  • ConfigMap键名错误
  • 环境变量名称冲突
  • ConfigMap未正确引用

解决方案

yaml
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: my-app:1.0
    env:
    - name: APP_NAME  # 环境变量名称
      valueFrom:
        configMapKeyRef:
          name: app-config
          key: app.name  # ConfigMap键名,必须完全匹配
    envFrom:
    - configMapRef:
        name: app-config
      prefix: CONFIG_  # 添加前缀避免冲突

3. Secret解码失败

症状

Error: couldn't get key data: illegal base64 data at input byte 4

排查步骤

bash
# 1. 查看Secret内容
kubectl get secret app-secret -o yaml

# 2. 尝试手动解码
kubectl get secret app-secret -o jsonpath='{.data.password}' | base64 --decode

# 3. 检查Secret类型
kubectl get secret app-secret -o jsonpath='{.type}'

# 4. 查看Secret详细信息
kubectl describe secret app-secret

可能原因

  • Secret数据未正确Base64编码
  • 使用了stringData而非data字段
  • 数据格式错误

解决方案

yaml
# 方式1:使用stringData(推荐)
apiVersion: v1
kind: Secret
metadata:
  name: app-secret
type: Opaque
stringData:  # 自动编码
  password: "MyPassword123"

# 方式2:使用data(手动编码)
apiVersion: v1
kind: Secret
metadata:
  name: app-secret
type: Opaque
data:  # 需要手动Base64编码
  password: TXlQYXNzd29yZDEyMw==  # echo -n "MyPassword123" | base64

4. 配置更新不生效

症状:更新ConfigMap后,应用未读取到新配置

排查步骤

bash
# 1. 检查ConfigMap是否更新
kubectl get configmap app-config -o yaml

# 2. 查看Pod挂载的配置文件
kubectl exec -it app-pod -- cat /etc/config/app.properties

# 3. 检查Pod创建时间
kubectl get pod app-pod -o yaml | grep creationTimestamp

# 4. 查看ConfigMap更新时间
kubectl describe configmap app-config | grep -A 5 Events

可能原因

  • 应用未重新加载配置
  • Pod未重启
  • 使用了subPath导致无法自动更新

解决方案

yaml
# 方式1:重启Pod
kubectl rollout restart deployment/my-app

# 方式2:使用版本标签触发滚动更新
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    metadata:
      annotations:
        config-hash: "v1"  # 更新此值触发滚动更新
    spec:
      containers:
      - name: app
        image: my-app:1.0
        volumeMounts:
        - name: config-volume
          mountPath: /etc/config
      volumes:
      - name: config-volume
        configMap:
          name: app-config

# 方式3:应用实现配置热加载
# 应用定期检查配置文件变化并重新加载

5. Secret权限问题

症状

Error: secrets "app-secret" is forbidden: User "system:serviceaccount:default:default" cannot get resource "secrets" in API group "" in the namespace "default"

排查步骤

bash
# 1. 检查ServiceAccount
kubectl get serviceaccount

# 2. 检查RBAC权限
kubectl auth can-i get secrets --as=system:serviceaccount:default:default

# 3. 查看Role和RoleBinding
kubectl get role,rolebinding

# 4. 检查Pod的ServiceAccount
kubectl get pod app-pod -o yaml | grep serviceAccountName

解决方案

yaml
# 创建ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-service-account
  namespace: default
---
# 创建Role
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: secret-reader
  namespace: default
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list", "watch"]
---
# 创建RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-secrets
  namespace: default
subjects:
- kind: ServiceAccount
  name: app-service-account
  namespace: default
roleRef:
  kind: Role
  name: secret-reader
  apiGroup: rbac.authorization.k8s.io
---
# Pod使用ServiceAccount
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  serviceAccountName: app-service-account
  containers:
  - name: app
    image: my-app:1.0

故障排查流程图

配置问题

检查资源存在性 → 不存在 → 创建ConfigMap/Secret
    ↓ 存在
检查命名空间 → 不匹配 → 修正命名空间
    ↓ 匹配
检查引用方式 → 错误 → 修正引用配置
    ↓ 正确
检查权限配置 → 权限不足 → 配置RBAC
    ↓ 权限正常
检查数据格式 → 格式错误 → 修正数据格式
    ↓ 格式正确
检查应用读取 → 应用问题 → 检查应用代码
    ↓ 正常
配置生效

最佳实践建议

1. 命名规范

yaml
# ConfigMap命名:{app-name}-config
metadata:
  name: user-service-config

# Secret命名:{app-name}-secret
metadata:
  name: user-service-secret

# 键名使用点分隔符
data:
  database.host: "mysql"
  database.port: "3306"
  cache.enabled: "true"

2. 敏感信息管理

yaml
# 不要在ConfigMap中存储敏感信息
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  database.host: "mysql"  # ✓ 非敏感信息
  database.port: "3306"   # ✓ 非敏感信息
  # database.password: "password123"  # ✗ 不要这样做!

---
# 敏感信息存储在Secret中
apiVersion: v1
kind: Secret
metadata:
  name: app-secret
stringData:
  database.password: "password123"  # ✓ 敏感信息存储在Secret中
  api.key: "secret_api_key"

3. 配置分层

yaml
# 基础配置(所有环境共享)
apiVersion: v1
kind: ConfigMap
metadata:
  name: base-config
  namespace: default
data:
  app.name: "my-app"
  app.version: "1.0.0"
---
# 环境特定配置
apiVersion: v1
kind: ConfigMap
metadata:
  name: env-config
  namespace: default
data:
  app.env: "production"
  log.level: "error"
---
# 应用使用多个ConfigMap
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: my-app:1.0
    envFrom:
    - configMapRef:
        name: base-config
    - configMapRef:
        name: env-config
    - secretRef:
        name: app-secret

4. 不可变配置

yaml
# 使用immutable字段防止意外修改
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: default
immutable: true  # 设置为不可变
data:
  app.name: "my-app"
  app.version: "1.0.0"
---
apiVersion: v1
kind: Secret
metadata:
  name: app-secret
  namespace: default
immutable: true  # 设置为不可变
type: Opaque
stringData:
  password: "StrongPassword123"

5. 配置验证

yaml
# 使用Admission Controller验证配置
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
  name: config-validation
spec:
  matchConstraints:
    resourceRules:
    - apiGroups: [""]
      apiVersions: ["v1"]
      operations: ["CREATE", "UPDATE"]
      resources: ["configmaps", "secrets"]
  validations:
  - expression: "object.data.size() <= 100"
    message: "ConfigMap/Secret cannot have more than 100 keys"
  - expression: "'password' in object.data == false"
    message: "ConfigMap should not contain 'password' key"

6. 配置加密

yaml
# 启用etcd加密
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
    - secrets
    providers:
    - aescbc:
        keys:
        - name: key1
          secret: <base64-encoded-secret>
    - identity: {}

7. 监控和告警

yaml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: config-alerts
  namespace: monitoring
spec:
  groups:
  - name: config
    rules:
    - alert: ConfigMapChanged
      expr: |
        kube_configmap_metadata_resource_version
      for: 0m
      labels:
        severity: info
      annotations:
        summary: "ConfigMap {{ $labels.configmap }} changed"
        description: "ConfigMap {{ $labels.configmap }} in namespace {{ $labels.namespace }} has been modified"
    
    - alert: SecretChanged
      expr: |
        kube_secret_metadata_resource_version
      for: 0m
      labels:
        severity: warning
      annotations:
        summary: "Secret {{ $labels.secret }} changed"
        description: "Secret {{ $labels.secret }} in namespace {{ $labels.namespace }} has been modified"

8. 配置备份

bash
#!/bin/bash
# backup-configs.sh

NAMESPACE="production"
BACKUP_DIR="/backups/configs"
DATE=$(date +%Y%m%d_%H%M%S)

# 创建备份目录
mkdir -p ${BACKUP_DIR}/${DATE}

# 备份所有ConfigMap
kubectl get configmaps -n ${NAMESPACE} -o yaml > ${BACKUP_DIR}/${DATE}/configmaps.yaml

# 备份所有Secret
kubectl get secrets -n ${NAMESPACE} -o yaml > ${BACKUP_DIR}/${DATE}/secrets.yaml

# 压缩备份
tar -czf ${BACKUP_DIR}/config-backup-${DATE}.tar.gz -C ${BACKUP_DIR} ${DATE}

# 删除临时目录
rm -rf ${BACKUP_DIR}/${DATE}

# 保留最近30天的备份
find ${BACKUP_DIR} -name "config-backup-*.tar.gz" -mtime +30 -delete

echo "Backup completed: ${BACKUP_DIR}/config-backup-${DATE}.tar.gz"

总结

核心要点

  1. 配置分离:将配置与镜像分离,实现灵活的配置管理
  2. 敏感信息保护:使用Secret存储敏感信息,启用加密存储
  3. 多环境管理:通过命名空间和ConfigMap实现多环境配置
  4. 配置热更新:合理设计配置更新策略,避免服务中断
  5. 安全最佳实践:使用RBAC、加密、不可变配置等安全措施

关键命令速查

bash
# ConfigMap管理
kubectl create configmap <name> --from-literal=key=value  # 创建ConfigMap
kubectl get configmap                                     # 查看ConfigMap
kubectl describe configmap <name>                         # 查看详情
kubectl delete configmap <name>                           # 删除ConfigMap

# Secret管理
kubectl create secret generic <name> --from-literal=key=value  # 创建Secret
kubectl get secrets                                            # 查看Secret
kubectl get secret <name> -o jsonpath='{.data.key}' | base64 --decode  # 解码Secret
kubectl delete secret <name>                                   # 删除Secret

# 故障排查
kubectl exec -it <pod> -- env                                 # 查看环境变量
kubectl exec -it <pod> -- cat /etc/config/file                # 查看配置文件
kubectl describe pod <pod>                                    # 查看Pod事件

下一步学习

参考资源