结合初始化信息理解kubeadm init的工作流程
初始化信息
W0519 01:47:26.317891 2272 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io] [init] Using Kubernetes version: v1.18.0 [preflight] Running pre-flight checks [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/ [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Starting the kubelet [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "ca" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.10.0.1 10.0.0.50] [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [master localhost] and IPs [10.0.0.50 127.0.0.1 ::1] [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [master localhost] and IPs [10.0.0.50 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "sa" key and public key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" [kubeconfig] Writing "admin.conf" kubeconfig file [kubeconfig] Writing "kubelet.conf" kubeconfig file [kubeconfig] Writing "controller-manager.conf" kubeconfig file [kubeconfig] Writing "scheduler.conf" kubeconfig file [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" W0519 01:50:33.295075 2272 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC" [control-plane] Creating static Pod manifest for "kube-scheduler" W0519 01:50:33.296322 2272 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC" [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s [apiclient] All control plane components are healthy after 23.005020 seconds [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [kubelet] Creating a ConfigMap "kubelet-config-1.18" in namespace kube-system with the configuration for the kubelets in the cluster [upload-certs] Skipping phase. Please see --upload-certs [mark-control-plane] Marking the node master as control-plane by adding the label "node-role.kubernetes.io/master=''" [mark-control-plane] Marking the node master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule] [bootstrap-token] Using token: q1k6jx.wnw5mn8qqt0ia3wc [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 10.0.0.50:6443 --token q1k6jx.wnw5mn8qqt0ia3wc --discovery-token-ca-cert-hash sha256:c99d2b8e33ab852ec41e9a0fe816683d5d791d7afa8041a802b986cac8456553
1、Preflight Checks
- Linux 内核的版本必须是否是 3.10 以上
- Linux Cgroups 模块是否可用
- 机器的 hostname 是否标准在 Kubernetes 项目里，机器的名字以及一切存储在 Etcd 中的 API 对象，都必须使用标准的 DNS 命名（RFC 1123）。
- 用户安装的 kubeadm 和 kubelet 的版本是否匹配
- 机器上是不是已经安装了 Kubernetes 的二进制文件
- Kubernetes 的工作端口 10250/10251/10252 端口是不是已经被占用
- ip、mount 等 Linux 指令是否存在
- Docker 是否已经安装
等等
2、生成提供服务所需的证书和对应目录
Kubernetes 对外提供服务时，需要通过HTTPS访问kube-apiserver。 生成证书位置：/etc/kubernetes/pki/ca.{crt,key} kube-apiserver 请求 kubelet时，如：使用 kubectl 获取容器日志等 streaming 操作，也需要安全的连接 生成的证书位置：/etc/kubernetes/pki/apiserver-kubelet-client.{crt,key}
apiserver向etcd请求存储持久化信息时，需要的证书：/etc/kubernetes/pki/apiserver-etcd-client.{crt,key}
...
3、生成访问 kube-apiserver配置文件
为其他组件生成访问 kube-apiserver 所需的配置文件。
配置文件的路径：/etc/kubernetes/xxx.conf
[root@master kubelet]# ll /etc/kubernetes/*.conf -rw-------. 1 root root 5445 May 19 01:50 /etc/kubernetes/admin.conf -rw-------. 1 root root 5481 May 19 01:50 /etc/kubernetes/controller-manager.conf -rw-------. 1 root root 1857 May 19 01:50 /etc/kubernetes/kubelet.conf -rw-------. 1 root root 5433 May 19 01:50 /etc/kubernetes/scheduler.conf
这些文件记录了Master节点的IP地址、端口、证书信息、工作上下文环境，例如：
[root@master kubelet]# cat /etc/kubernetes/admin.conf apiVersion: v1 clusters: - cluster: certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJd01EVXhPREUzTlRBek1Gb1hEVE13TURVeE5qRTNOVEF6TUZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTWF6CjdKcEkrL0JjVkRlMUFpNy80WDFaYmE2b1IwcE8zb1MxVTVXREllRmJacEpaVlpUcEtFSDJUK0phSEZLTEU3Q3MKc1lNQ3R5aHMwSEdYeHkwY211N1lPR2N1N2JHcUgrajR2MDM4ckpLWm9XM0x5bkVtOUs1MXUvZmNDR0NsQ3NxTgplQWVWSWtQWUlkeWprZEkvckdKZWZQTG82M2xWSHI1Yi9LTUtETTNQQndiZlE3NFRWNS9rVE5ZdjY5VDNIdm4wClBDUUVnMXMxeFMvSk5PdTQxQy9mMVczNUxIZHYvVlg1YnI5WXN5cG1RSG1RRUQxU2xEa0xyQ28vZ016cTdlMEwKNnhHbEZtYUlYMkd6RmF5MGNyZVVGTkFtTUQwaGx5WHF0MnUrdndjSEFZZERGNzhSekRvY2ZEa2VGMGl0TnEvOQp0cWNNbUpCOUlOWjZ1b2ovcC9rQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFKSWJRME85dHlWTVdUaUVWYU9TdUQrWEFjTlMKRXkzT2JCR1NyM1RlTnA1bWFpazg0TjE3b2g5Z0FSQVhPMS8xckpPeWpIK1FrOE4zRnY2WXBzNVY0MktnTWx3SQplT0VFdy9YSlZGU3JkVi9ob0pUMUxyV1dGY1pqTis3eGw5bkJ0MmdyTXo2K1JxcDQ4alpPUWpIRytZcEJvNWkyCktvMWFKcHFGL0hzM2RzbytsVWNrWitRRS9NZ3ViVkloRE96ZUF6LzkwMFNlcC9XNkl2NTVkTUJsbTh1SEFNQmQKTEsvdlJ2K2J3bHdwZ1dhVVErNlFmL1ZMRzYxYXRiUmR4WW8wVWJDRUtyL3dxU1N5MURyQmRvOUFsKyt3c0JBWgpYa3YzSzFHL29zUGRIbnVnOUdjak1WZWFrcDhVYzlJVXVpREhoNVBUN1VtYmYyNmNWaEh0VG5mdjRoND0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo= server: https://10.0.0.50:6443 name: kubernetes contexts: - context: cluster: kubernetes user: kubernetes-admin name: kubernetes-admin@kubernetes current-context: kubernetes-admin@kubernetes kind: Config preferences: {} users: - name: kubernetes-admin user: client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM4akNDQWRxZ0F3SUJBZ0lJTHJPNTd3UDUyZkV3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TURBMU1UZ3hOelV3TXpCYUZ3MHlNVEExTVRneE56VXdNekphTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQTJ0U3I4ZzB6Qlp4aTFxTEYKU2pJd09NeVVYVXovbjFIcGNVL0hjQ0tSalQ0L1k5dUtZNVMrTWJXVVg3UkxZTXBzcWhKb1ZRSTNRYzdRRUw1TwpjdUlIVDFKbDIxVFh5Z3JlQlZNRGR4cEtCcnFZQll2YThkVjZpWlZEL1VieXJCSCt5SnA5WGxoWE83NEovOEZJCnNmVDltR3pPRENyS1VTOE1nU1lxTnI3elZLN3BVSEh1ZVlHMC9aTVhqSmF4NkFaeXdEU2FNUnBtRllLRkNEMVUKM0dqTVVJVmtqNFVEZHZRN1QxOExDK0grUFVhSzgyQ1p2MHR4b243cE9ZZ3JobmFFUEQ0WFZzV3g5OVA3Y24xUwpkSXFMUFNlUXhrMXk2N3orcnpYV3pSL0ZPVGJGaGdtWlNvaThaSmtyTmdmZFVSWlNoazdCcU5KWGlkdFdDQTRUCmJiV0lSd0lEQVFBQm95Y3dKVEFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFISjdQUEZjUFRDN3FBTnN2T3hqd2NUVXowK0FobGJ5cDhmVgpaTE0vdmJCUFRHOU9uRitNQkZieVBmR1pPN0FHcGRtREY3UTR5WGFyK3FIMHJiY3pXWmJqMW9WRkcwWXlUaDhECmFEZjd0RmdQMkJTUktqclpzODdJcUtEYXhCWUhmaG9SS1AxdFpUN1UyTUFQUXg3OUdUUjgxNmNIMzVidlp2d2kKTlpjUEU0NFBsNUhuUTN2U3E1RzMzcEJPSFd5cStUOTlEZUo1ci9oOHJVTnFidDN0cGhEaThLVys0Rkl4RlFKSQpqdGJ6c3RZSkR5YWRWNDFlV1d0RVYwTDdDWUovWWVCRi9PWWMwNU1CSVRIMkcvYmd1bDJwbjhQREkwUU5MWG5hCllUOTV5dm10RHhvWnpmYjdlYnV5UVVkZkRJcnB4YUprL2FOV2dkYi9jYldoNTUwODhQST0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo= client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBMnRTcjhnMHpCWnhpMXFMRlNqSXdPTXlVWFV6L24xSHBjVS9IY0NLUmpUNC9ZOXVLClk1UytNYldVWDdSTFlNcHNxaEpvVlFJM1FjN1FFTDVPY3VJSFQxSmwyMVRYeWdyZUJWTURkeHBLQnJxWUJZdmEKOGRWNmlaVkQvVWJ5ckJIK3lKcDlYbGhYTzc0Si84RklzZlQ5bUd6T0RDcktVUzhNZ1NZcU5yN3pWSzdwVUhIdQplWUcwL1pNWGpKYXg2QVp5d0RTYU1ScG1GWUtGQ0QxVTNHak1VSVZrajRVRGR2UTdUMThMQytIK1BVYUs4MkNaCnYwdHhvbjdwT1lncmhuYUVQRDRYVnNXeDk5UDdjbjFTZElxTFBTZVF4azF5Njd6K3J6WFd6Ui9GT1RiRmhnbVoKU29pOFpKa3JOZ2ZkVVJaU2hrN0JxTkpYaWR0V0NBNFRiYldJUndJREFRQUJBb0lCQVFDMW0ydjduSktzWkdYdQpoUFZBcHpnMzJ5aUI2ZVgyM2E3ajEvYkhEQmxKWTlDTjJlUVcwcG1wZlcxZW82MHU3YStTMFdYK3JyRVhEMERECnRIdzhnWExabEtOdGpCTHQzV2oyZURkVy85MVJpa2VoeXJod25OOXVFUTkwd2cyaFdlbmRwOERGcklEdzFyMUwKb0tmbzhFNEowcnFKaEhXVlBIdWZMd0kzbnU4b1pkdHFGdUtMQnBSWnNyQzNKWXZlRkRkODZtMWR4TmVWczc4Qwo3Q1VRZzVJOUp6Vkl3WDVoQVhZbWdtTm1YNDA3cnpHTERzYlpYN0JhNmlrTDJHSUNiQngzMlh0QXVCNnRBdEIyCk5ZWmNrSk9mZHlvV1BtWjU0V1JOZEpGYWlmZXBYZ2M4N28wTlhTV2Z0SDdNUEx5eWlYWmRwSVVMRllpbjhHL1IKbGR6WjhMNHhBb0dCQU9FNzZEdE04MTh3VFZCZkVxc0RZRVJpcnphQitUa1Q5TmhxNXRLSUtGb3JYYURTRk5sZwpvVUtSejBvdVZleTNuenpQa004cTc3b2t4YkFaR1Q2QnB4WVJzcWNaWlJTTzErOU1rVEZOZGhqSzhVM0ZoNnlFCjNJTDJPVURYNjNrTnprNm8rRyt6Z21WaStteFFKZERNVmhKbnNCQjVRMzFhdHNjYUtDWFROTm1KQW9HQkFQaTQKMmlDVzY2TVFMMHFpdjJPekZ5cCs1SlZ5MHh4OTcwV1dtTlZKNWY3dGh5SG1melR4aW51Nkl4c2Y4bG1NdGpjbApWSnJBcjBoOTRGMkl0aERORWtYSlZIaTNQK3lQUWh0QWs4WmJmdGJtTWEzcjZnNTlpNjN0S1p1SHZHWS9vVTBhCnpZams4Z29TNWZqTEZhQnZLaWpPWU5tN2VIeVR2ZFV5a0Iya2RHOVBBb0dBVVFWeTBib3BwZWxETnBFc3J2WGsKOEZTcmdLa2FsTXkzL0EzZ2dJVllOcTk0Mjd3V29lZWZ1c21tenFHQ2FVZlljVkNkWDlpcktjUEdsVVZDRG5rbgpPTW9mQVBzaW9GV09HZGZxTnRrTmpYZWJmQVY5ZTdMRGZCekVsYTNXVjlKK2owODdKenRrd2NIc0lZQm5TZ2ZuClFuR29KUlRxRVRMTG95Mm1tWXl6YXprQ2dZQTZnM0o5bkVQUFZ1MXBSNlJ6Rmh6ckdITTZYWXNnOXRlbHJXcEQKTTJGeWVmc0NsTEYwaVNhbE9RTXRUSFM5Y0ljbHJoaWJWNTFsRm9nRU9UZHIrSExHRERsZE5POUsvZUQxZkZuSApucHJXZjgxTU4yWVhCcDRueXRXeEMxdmRTamJ1WnRIWDFOVEVYZkhRZUNhY1djcTNVdVlpRXlLalhEYWF3NHg0CkRNcC9yUUtCZ0F6bCtjSDVDMkhOdUJZeGV4My9rMXM4d3lFQVdHTUcrU0NWRWd0QTNPaW5GUFp1SmhpN3pXcjEKaVZRL05UdDE2WGdZZFY4WVJidG5Wa25qKzNXckRCVVE1Qk5JVkUzMzNZMUFFL3NnS3VjS1NtQ2hCWFl2UEd4VApKYlVtTUROY1ZNeUd4UzdaS0t5TUQrSlZvVFI2ZlhqeHhKUzdFRW8vdEFmZ2xmaWZGcnBxCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
4、生成Master组件的Pod配置文件
为 kube-apiserver、kube-controller-manager、kube-scheduler 和 etcd 生成YAML文件。通过Static Pod的方法启动
组件的YAML文件路径：
[root@master ~]# ll /etc/kubernetes/manifests/
total 16
-rw-------. 1 root root 1855 May 19 01:50 etcd.yaml
-rw-------. 1 root root 2726 May 19 01:50 kube-apiserver.yaml
-rw-------. 1 root root 2594 May 19 01:50 kube-controller-manager.yaml
-rw-------. 1 root root 1149 May 19 01:50 kube-scheduler.yaml
kubelet 启动时，它会自动检查这个目录，加载所有的 Pod YAML 文件，然后启动它们
Master 容器启动后，kubeadm 会通过 localhost:6443/healthz 做健康检查
5、生成bootstrap token
Node节点可以使用这个token，通过kubeadm join的方式加入集群
6、保存配置信息
kubeadm 会将 Master 各个重要信息通过 ConfigMap 的方式保存到 Etcd 中，提供给 Node 节点使用
ConfigMap 的名字是 cluster-info
7、安装默认插件（附件）
CoreDNS和kube-proxy 这两个是必须安装的
kubeadm join 的工作流程
获取ConfigMap中cluster-info保存的地址、端口、证书，使用kubeadm init 生成的 bootstrap token通过"安全模式"访问 kube-apiserver，