kubeadm搭建高可用k8s平台（多master）

环境确认

172.27.0.4 master
172.27.0.11 master
172.27.0.17 master
系统版本：centos7.6
k8s版本：1.19.3（最新）
docker版本：1.19（最新）
确保firewalld和selinux都关闭

第一步：下载k8s和docker的yum源(3台)

k8s源

[root@VM-0-4-centos yum.repos.d]# cat /etc/yum.repos.d/kubernetes.repo 
[kubernetes]
name = kubernetes
baseurl = https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled = 1
gpgcheck = 1
gpgkey = https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

docker-ce源

[root@VM-0-4-centos yum.repos.d]# wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

第二步：下载kubelet，kubeadm，kubectl，docker-ce(3台)

[root@VM-0-4-centos yum.repos.d]# yum install kubelet kubeadm kubectl docker-ce -y

第三步：给docker写入镜像加速(3台)

cat > /etc/docker/daemon.json <<EOF
{
    "registry-mirrors":["https://registry.docker-cn.com","https://l10nt4hq.mirror.aliyuncs.com"]
}
EOF

第四步：使加速镜像生效

systemctl daemon-reload && systemctl restart docker

第五步：创建高可用集群

您可以使用 --kubernetes-version 标志来设置要使用的 Kubernetes 版本。建议将
kubeadm、kebelet、kubectl 和 Kubernetes 的版本匹配。
这个 --control-plane-endpoint 标志应该被设置成负载均衡器的地址或 DNS 和端口
这个 --upload-certs 标志用来将在所有控制平面实例之间的共享证书上传到集群。如果正好相反，你更喜欢手动地通过控制平面节点或者使用自动化工具复制证书，请删除此标志.

[root@VM-0-4-centos yum.repos.d]# kubeadm init --control-plane-endpoint "172.27.0.4:6443" --upload-certs --image-repository registry.aliyuncs.com/google_containers
W1106 14:56:21.949737   16390 configset.go:348] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[init] Using Kubernetes version: v1.19.3
[preflight] Running pre-flight checks
    [WARNING Service-Docker]: docker service is not enabled, please run 'systemctl enable docker.service'
    [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

...............

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:

  kubeadm join 172.27.0.4:6443 --token 09qgun.0knjwo2de6cb3jui 
    --discovery-token-ca-cert-hash sha256:9e3062cf17679336d984b16ea384ab287f6ae2a5e184e4c07a8eb4e512ca7bf6 
    --control-plane --certificate-key bdc888f3d751989a3f111a8f3ea8c313380dfd0fc6d719bff4c6a5715efba5a6

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 172.27.0.4:6443 --token 09qgun.0knjwo2de6cb3jui 
    --discovery-token-ca-cert-hash sha256:9e3062cf17679336d984b16ea384ab287f6ae2a5e184e4c07a8eb4e512ca7bf6 
[root@VM-0-4-centos yum.repos.d]#

第七步：在其他两个机器上执行上述kubeadm join的命令，上面的为添加master，下面的是添加node

kubeadm join 172.27.0.4:6443 --token 09qgun.0knjwo2de6cb3jui     --discovery-token-ca-cert-hash sha256:9e3062cf17679336d984b16ea384ab287f6ae2a5e184e4c07a8eb4e512ca7bf6     --control-plane --certificate-key bdc888f3d751989a3f111a8f3ea8c313380dfd0fc6d719bff4c6a5715efba5a6 
[preflight] Running pre-flight checks
    [WARNING Service-Docker]: docker service is not enabled, please run 'systemctl enable docker.service'
    [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
.....................
To start administering your cluster from this node, you need to run the following as a regular user:

    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config

Run 'kubectl get nodes' to see this node join the cluster.

第八步：按照提示完成三条命令

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

第九步：验证,可以看到3台都是master

以此类推

[root@VM-0-4-centos yum.repos.d]# kubectl get nodes
NAME             STATUS     ROLES    AGE     VERSION
vm-0-11-centos   NotReady   master   3m52s   v1.19.3
vm-0-17-centos   NotReady   master   3m10s   v1.19.3
vm-0-4-centos    NotReady   master   11m     v1.19.3