签发二级CA,部署公网K8S集群,安装calico网络插件

一、机器及环境准备
二、系统初始配置
三、添加国内镜像源
四、升级系统 && 安装软件
五、自建CA并签发二级CA
六、使用kubeadm部署K8S集群
七、安装网络插件

一、机器及环境准备

master节点三台,worker节点一台,公网负载均衡一个

主机名 内网IP 公网IP 域名 解析地址 系统版本 前端端口 后端端口 后端服务器组
master1 172.30.0.3       CentOS 7.6 64bit      
master2 172.30.0.4       CentOS 7.6 64bit      
master3 172.30.0.5       CentOS 7.6 64bit      
worker-0001 172.30.0.1       CentOS 7.6 64bit      
负载均衡 172.30.0.10 1.1.1       6444 6443 master1-3
域名     xxx.yyy.com 1.1.1.1        

 

 

 

 

 

 

 

 

 

二、系统初始配置

1、关闭swap

swapoff -a
sed -ri 's/.*swap.*/#&/' /etc/fstab

2、关闭所有防火墙

systemctl stop firewalld
systemctl disable firewalld

3、禁用SELINUX

setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

4、内核配置

cat << EOF > /etc/sysctl.d/kubernetes.conf
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
vm.swappiness=0
vm.overcommit_memory=1
vm.panic_on_oom=0
EOF
modprobe br_netfilter
modprobe ip_vs
sysctl -p /etc/sysctl.d/kubernetes.conf

5、kube-proxy开启ipvs的前置配置

cat << EOF >/etc/sysconfig/modules/ipvs.modules 
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4

三、添加国内镜像源

1、添加Docker-ce源

yum install -y yum-utils
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

2、添加K8S源

cat << EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

四、升级系统 && 安装软件

1、升级安装

yum update  -y
yum install -y epel-release
yum install -y conntrack ipvsadm ipset jq sysstat curl iptables
yum install -y device-mapper-persistent-data lvm2
yum install -y container-selinux
yum install -y docker-ce
yum install -y kubelet kubeadm kubectl

2、配置 cgroupdriver

修改或创建/etc/docker/daemon.json,加入下面内容:

{
  "exec-opts": ["native.cgroupdriver=systemd"]
}

3、启动docker、设置docker、kubelet开机自启(这里先不要开启kubelet服务)

systemctl enable docker && systemctl start docker
systemctl enable kubelet

4、验证 docker cgroup

执行以下命令, 预期输出 Cgroup Driver: systemd

docker info |grep Cgroup

五、自建CA并签发二级CA

1、创建根CA

1.1、依次创建如下目录:

mkdir -p /opt/ca/root
mkdir /opt/ca/root/key

1.2、vim /opt/ca/root/openssl.cnf

[ ca ]
default_ca	= CA_default
 
[ CA_default ]
dir		    = /opt/ca/root
certs		= $dir/certs
crl_dir		= $dir/crl
database	= $dir/index.txt
new_certs_dir	= $dir/newcerts
certificate	= $dir/key/ca.crt
serial		= $dir/serial
crlnumber	= $dir/crlnumber
crl		    = $dir/crl.pem
private_key	= $dir/key/ca.key
RANDFILE	= $dir/key/.rand
unique_subject	= no
 
x509_extensions	= usr_cert
copy_extensions = copy
 
name_opt 	= ca_default
cert_opt 	= ca_default
 
default_days	= 3650
default_crl_days= 30
default_md	= sha256
preserve	= no
policy		= policy_ca
 
[ policy_ca ]
countryName		= supplied
stateOrProvinceName	= supplied
organizationName	= supplied
organizationalUnitName	= supplied
commonName		= supplied
emailAddress		= optional
 
[ req ]
default_bits		= 2048
default_keyfile 	= privkey.pem
distinguished_name	= req_distinguished_name
attributes		= req_attributes
x509_extensions	= v3_ca
string_mask = utf8only
utf8 = yes
prompt                  = no
 
[ req_distinguished_name ]
countryName			= CN
stateOrProvinceName		= beijing
localityName			= beijing
organizationName        = Global XXX CA Inc
organizationalUnitName	= XXX Root CA
commonName			= Global XXX Root CA
 
[ usr_cert ]
basicConstraints = CA:TRUE
 
[ v3_ca ]
basicConstraints = CA:TRUE
 
[ req_attributes ]

1.3、创建如下目录及文件

mkdir /opt/ca/root/newcerts
touch /opt/ca/root/index.txt
touch /opt/ca/root/index.txt.attr
echo 01 > /opt/ca/root/serial

1.4、创建根CA私钥

openssl genrsa -out /opt/ca/root/key/ca.key 2048

1.5、创建根CA证书请求文件

openssl req -new -key /opt/ca/root/key/ca.key -out /opt/ca/root/key/ca.csr -config /opt/ca/root/openssl.cnf

1.6、自签根CA证书

openssl ca -selfsign -in /opt/ca/root/key/ca.csr -out /opt/ca/root/key/ca.crt -config /opt/ca/root/openssl.cnf

1.7、查看证书信息(可选)

openssl x509 -text -in /opt/ca/root/key/ca.crt

经过以上几个步骤,就生成了根CA的相关证书和私钥,可以用于签发其他的CA(二级CA),不可签发服务器证书

2、创建二级CA

2.1、创建如下目录

mkdir -p /opt/ca/agent/key

2.2、vim /opt/ca/agent/openssl.cnf

[ ca ]
default_ca	= CA_default
 
[ CA_default ]
dir		    = /opt/ca/agent
certs		= $dir/certs
crl_dir		= $dir/crl
database	= $dir/index.txt
new_certs_dir	= $dir/newcerts
certificate	= $dir/key/ca.crt
serial		= $dir/serial
crlnumber	= $dir/crlnumber
crl		    = $dir/crl.pem
private_key	= $dir/key/ca.key
RANDFILE	= $dir/key/.rand
unique_subject	= no
 
x509_extensions	= usr_cert
copy_extensions = copy
 
name_opt 	= ca_default
cert_opt 	= ca_default
 
default_days	= 3650
default_crl_days= 30
default_md	= sha256
preserve	= no
policy		= policy_ca
 
[ policy_ca ]
countryName		= supplied
stateOrProvinceName	= supplied
organizationName	= supplied
organizationalUnitName	= supplied
commonName		= supplied
emailAddress		= optional
 
[ req ]
default_bits		= 2048
default_keyfile 	= privkey.pem
distinguished_name	= req_distinguished_name
attributes		= req_attributes
x509_extensions	= v3_ca
string_mask = utf8only
utf8 = yes
prompt = no
 
[ req_distinguished_name ]
countryName			= CN
stateOrProvinceName		= Guangdong
localityName			= Guangzhou
organizationName        = Global XXX CA Inc
organizationalUnitName	= Google 2020 CA
commonName			= Google 2020 CA
 
[ usr_cert ]
basicConstraints = CA:FALSE
 
[ v3_ca ]
basicConstraints        = CA:TRUE
 
[ req_attributes ]

2.3、创建如下目录及文件

mkdir /opt/ca/agent/newcerts
touch /opt/ca/agent/index.txt
touch /opt/ca/agent/index.txt.attr
echo 01 > /opt/ca/agent/serial

2.4、创建二级CA私钥

openssl genrsa -out /opt/ca/agent/key/ca.key 2048

2.5、创建二级CA证书请求文件

openssl req -new -key /opt/ca/agent/key/ca.key -out /opt/ca/agent/key/ca.csr -config /opt/ca/agent/openssl.cnf

2.6、使用根CA签发二级CA

openssl ca -in /opt/ca/agent/key/ca.csr -out /opt/ca/agent/key/ca.crt -config /opt/ca/root/openssl.cnf

2.7、查看证书信息(可选)

openssl x509 -text -in /opt/ca/agent/key/ca.crt

经过以上几个步骤,就生成了一个二级CA,这个二级CA可以签发服务器证书(不能签发其他的CA)

3、配置kubeadm使用自定义证书

3.1、创建目录

mkdir  /etc/kubernetes/pki/

3.2、将二级CA证书及私钥复制到pki目录下,kubeadm初始化中将自动使用此CA签发

cp  /opt/ca/agent/key/ca.crt   /etc/kubernetes/pki/
cp  /opt/ca/agent/key/ca.key  /etc/kubernetes/pki/

六、使用kubeadm部署K8S集群

1、打印kubeadm配置文件(master1节点操作)

kubeadm config print init-defaults > kubeadm-confi

2、编辑kubeadm配置文件(master1节点操作)

将advertiseAddress选项中的ip地址改为master1的地址;

添加controlPlaneEndpoint配置,地址为域名xxx.yyy.com,端口为负载均衡映射6443的前端端口6444;

将容器镜像仓库改为阿里仓库;

将kubeproxy的mode改为ipvs。

3、初始化master1节点

kubeadm init --config=kubeadm-config.yml --upload-certs

4、master1节点初始化完成后,逐个将master02、master03、worker01加入集群

第一段为配置kubectl;

第二段为master2、master3执行加入集群;

第三段为worker-0001执行加入集群。

5、如果添加工作节点的命令遗忘,可重新生成token,并输出显示

kubeadm token create --print-join-command --ttl 0

     添加master节点则在 kubeadm join 中增加 --control-plane --certificate-key 参数

七、安装网络插件

此处网络插件选用 calico

1、获取calico网络的yaml文件

curl  https://docs.projectcalico.org/v3.14/manifests/calico.yaml -o calico.yaml

2、部署flannel网络

kubectl apply -f calico.yaml

3、等待一段时间后,查看所有pod的状态为Running、所有节点的状态是Ready,则集群正常

kubectl get pods -n kube-system

kubectl get nodes -o wide

 

附:验证证书是否是某CA机构颁发

openssl verify -CAfile /etc/kubernetes/pki/ca.crt  kubelet-client-current.pem

  回显如下则kubelet-client-current.pem 证书是 /etc/kubernetes/pki/ca.crt 该CA机构颁发

kubelet-client-current.pem: OK

 查看证书详情

openssl x509 -in /etc/kubernetes/pki/apiserver.crt -text -noout

  

原文地址:https://www.cnblogs.com/dongming/p/13841860.html