问题描述
在k8s master节点按步骤创建成功后，执行从节点的加入操作，发现一直超时无法加入
问题日志
[root@node02 kubernetes-1.14]# kubeadm join 10.0.0.121:6443 --token oxqn4k.2olj80b59gndjmdj --discovery-token-unsafe-skip-ca-verification
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.14" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Activating the kubelet service
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
[kubelet-check] Initial timeout of 40s passed.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
error execution phase kubelet-start: timed out waiting for the condition
问题解决
1.按网上查了并不是token过期的原因
2.反复操作过程中，发现master非常卡，后来执行命令都无法返回结果了
kubeadm token list
failed to list bootstrap tokens: Get https://10.0.0.121:6443/api/v1/namespaces/kube-system/secrets?fieldSelector=type%3Dbootstrap.kubernetes.io%2Ftoken: net/http: TLS handshake timeout
3.判断可能是内存不足
[root@node01 ~]# free -h
total used free shared buff/cache available
Mem: 972M 524M 61M 50M 386M 76M
Swap: 0B 0B 0B
- 只有1G内存
4.于是打开虚拟机设置，增加1G内存
[root@node01 ~]# free -h
total used free shared buff/cache available
Mem: 1.9G 552M 595M 50M 848M 1.2G
Swap: 0B 0B 0B
发现master机流畅多了
5.重新执行节点加入，成功加入
kubeadm join 10.0.0.121:6443 --token befijo.vtrzxgw2ds7m5bak --discovery-token-ca-cert-hash sha256:5ea9439afdd166995f92d9d42a6fcb73aa8c92e5af5570850a709273f361b5b0
[kubelet-start] Activating the kubelet service
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details
[root@node01 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
node01 Ready master 20h v1.14.1
node02 Ready <none> 30m v1.14.1
问题总结
- 主节点由于内存不足无法正常服务导致从节点无法加入
- 又是一起简单的原因导致难查的BUG
- 由此可知k8s部署主节点至少需要分配2G内存