kubeadm安装集群系列-4.证书更新

证书更新

默认证书一年有效期

一旦证书过期，使用kubectl时会出现如下提示：`Unable to connect to the server: x509: certificate has expired or is not yet valid`

查看证书过期情况

 1 [root@k8s-test-master-1 ~]# kubeadm alpha certs check-expiration
 2 CERTIFICATE EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
 3 admin.conf Jul 28, 2020 05:41 UTC 364d no
 4 apiserver Jul 28, 2020 05:41 UTC 364d no
 5 apiserver-etcd-client Jul 28, 2020 05:41 UTC 364d no
 6 apiserver-kubelet-client Jul 28, 2020 05:41 UTC 364d no
 7 controller-manager.conf Jul 28, 2020 05:41 UTC 364d no
 8 etcd-healthcheck-client Jul 28, 2020 05:41 UTC 364d no
 9 etcd-peer Jul 28, 2020 05:41 UTC 364d no
10 etcd-server Jul 28, 2020 05:41 UTC 364d no
11 front-proxy-client Jul 28, 2020 05:41 UTC 364d no
12 scheduler.conf Jul 28, 2020 05:41 UTC 364d no
13 
14 # 查看根CA证书的有效期（十年）
15 [root@k8s-test-master-1 pki]# cd /etc/kubernetes/pki
16 [root@k8s-test-master-1 pki]# ls | grep ca.crt | xargs -I {} openssl x509 -text -in {} | grep "Not After"
17 Not After : Jul 26 05:41:23 2029 GMT
18 Not After : Jul 26 05:41:23 2029 GMT

证书目录结构

[root@k8s-test-master-1 pki]# pwd
/etc/kubernetes/pki
[root@k8s-test-master-1 pki]# tree .
.
├── apiserver.crt
├── apiserver-etcd-client.crt
├── apiserver-etcd-client.key
├── apiserver.key
├── apiserver-kubelet-client.crt
├── apiserver-kubelet-client.key
├── ca.crt
├── ca.key
├── etcd
│   ├── ca.crt
│   ├── ca.key
│   ├── healthcheck-client.crt
│   ├── healthcheck-client.key
│   ├── peer.crt
│   ├── peer.key
│   ├── server.crt
│   └── server.key
├── front-proxy-ca.crt
├── front-proxy-ca.key
├── front-proxy-client.crt
├── front-proxy-client.key
├── sa.key
└── sa.pub


1 directory, 22 files

Kubernetes 集群根证书

/etc/kubernetes/pki/ca.crt

/etc/kubernetes/pki/ca.key

由此根证书签发的证书有:

1,kube-apiserver 组件持有的服务端证书

　　/etc/kubernetes/pki/apiserver.crt

　　/etc/kubernetes/pki/apiserver.key

2,kubelet 组件持有的客户端证书

　　/etc/kubernetes/pki/apiserver-kubelet-client.crt

　　/etc/kubernetes/pki/apiserver-kubelet-client.key

kubelet 上一般不会明确指定服务端证书, 而是只指定 ca 根证书, 让 kubelet 根据本地主机信息自动生成服务端证书并保存到配置的cert-dir文件夹中。

汇聚层(aggregator)证书

/etc/kubernetes/pki/front-proxy-ca.crt

/etc/kubernetes/pki/front-proxy-ca.key

由此根证书签发的证书只有一组:

1,代理端使用的客户端证书, 用作代用户与 kube-apiserver 认证

/etc/kubernetes/pki/front-proxy-client.crt

/etc/kubernetes/pki/front-proxy-client.key

etcd 集群根证书

/etc/kubernetes/pki/etcd/ca.crt

/etc/kubernetes/pki/etcd/ca.key

由此根证书签发机构签发的证书有:

1,etcd server 持有的服务端证书

/etc/kubernetes/pki/etcd/server.crt

/etc/kubernetes/pki/etcd/server.key

2,peer 集群中节点互相通信使用的客户端证书

/etc/kubernetes/pki/etcd/peer.crt

/etc/kubernetes/pki/etcd/peer.key

3,pod 中定义 Liveness 探针使用的客户端证书

/etc/kubernetes/pki/etcd/healthcheck-client.crt

/etc/kubernetes/pki/etcd/healthcheck-client.key

4,配置在 kube-apiserver 中用来与 etcd server 做双向认证的客户端证书

/etc/kubernetes/pki/apiserver-etcd-client.crt

/etc/kubernetes/pki/apiserver-etcd-client.key

Serveice Account秘钥

这组的密钥对儿仅提供给 kube-controller-manager 使用. kube-controller-manager 通过 sa.key 对 token 进行签名, master 节点通过公钥 sa.pub 进行签名的验证.

API Server的authenticating环节支持多种身份校验方式：client cert、bearer token、static password auth等，这些方式中有一种方式通过authenticating（Kubernetes API Server会逐个方式尝试），那么身份校验就会通过。一旦API Server发现client发起的request使用的是service account token的方式，API Server就会自动采用signed bearer token方式进行身份校验。而request就会使用携带的service account token参与验证。该token是API Server在创建service account时用API server启动参数：–service-account-key-file的值签署(sign)生成的。如果–service-account-key-file未传入任何值，那么将默认使用–tls-private-key-file的值，即API Server的私钥（server.key）。

通过authenticating后，API Server将根据Pod username所在的group：system:serviceaccounts和system:serviceaccounts:(NAMESPACE)的权限对其进行authority 和admission control两个环节的处理。在这两个环节中，cluster管理员可以对service account的权限进行细化设置。

/etc/kubernetes/pki/sa.key

/etc/kubernetes/pki/sa.pub

kubeadm 创建的集群, kube-proxy ,flannel,coreDNS是以 pod 形式运行的, 在 pod 中, 直接使用 service account 与 kube-apiserver 进行认证, 此时就不需要再单独为 kube-proxy 创建证书

更新证书

生成集群配置的yaml文件

1 kubeadm config view > /root/kubeadm.yaml

kubeadm.yaml

 1 apiServer:
 2   extraArgs:
 3     authorization-mode: Node,RBAC
 4   timeoutForControlPlane: 4m0s
 5 apiVersion: kubeadm.k8s.io/v1beta2
 6 certificatesDir: /etc/kubernetes/pki
 7 clusterName: kubernetes-test
 8 controlPlaneEndpoint: 10.8.28.200:6443
 9 controllerManager: {}
10 dns:
11   type: CoreDNS
12 etcd:
13   local:
14     dataDir: /data/etcd
15 imageRepository: k8s.gcr.io
16 kind: ClusterConfiguration
17 kubernetesVersion: v1.15.1
18 networking:
19   dnsDomain: cluster.local
20   podSubnet: 192.168.0.0/16
21   serviceSubnet: 10.96.0.0/12
22 scheduler: {}

证书更新使用帮助

[root@k8s-test-master-1 ~]# kubeadm alpha certs renew --help
This command is not meant to be run on its own. See list of available subcommands.

Usage:
  kubeadm alpha certs renew [flags]
  kubeadm alpha certs renew [command]

Available Commands:
  admin.conf               Renew the certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself
  all                      Renew all available certificates
  apiserver                Renew the certificate for serving the Kubernetes API
  apiserver-etcd-client    Renew the certificate the apiserver uses to access etcd
  apiserver-kubelet-client Renew the certificate for the API server to connect to kubelet
  controller-manager.conf  Renew the certificate embedded in the kubeconfig file for the controller manager to use
  etcd-healthcheck-client  Renew the certificate for liveness probes to healtcheck etcd
  etcd-peer                Renew the certificate for etcd nodes to communicate with each other
  etcd-server              Renew the certificate for serving etcd
  front-proxy-client       Renew the certificate for the front proxy client
  scheduler.conf           Renew the certificate embedded in the kubeconfig file for the scheduler manager to use

更新证书操作

每个Master操作

kubeadm alpha certs renew all --config=/root/kubeadm.yaml
# (也可以逐个更新)
certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healtcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed
# 再次查询证书期限
[root@k8s-test-master-1 ~]# kubeadm alpha certs check-expiration
CERTIFICATE                EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
admin.conf                 Jul 29, 2020 06:47 UTC   364d            no
apiserver                  Jul 29, 2020 06:47 UTC   364d            no
apiserver-etcd-client      Jul 29, 2020 06:47 UTC   364d            no
apiserver-kubelet-client   Jul 29, 2020 06:47 UTC   364d            no
controller-manager.conf    Jul 29, 2020 06:47 UTC   364d            no
etcd-healthcheck-client    Jul 29, 2020 06:47 UTC   364d            no
etcd-peer                  Jul 29, 2020 06:47 UTC   364d            no
etcd-server                Jul 29, 2020 06:47 UTC   364d            no
front-proxy-client         Jul 29, 2020 06:47 UTC   364d            no
scheduler.conf             Jul 29, 2020 06:47 UTC   364d            no

# 在三台Master上执行重启kube-apiserver,kube-controller,kube-scheduler,etcd这4个容器，使证书生效
docker ps |grep -E 'k8s_kube-apiserver|k8s_kube-controller-manager|k8s_kube-scheduler|k8s_etcd_etcd' | awk -F ' ' '{print $1}' |xargs docker restart