- service到底是什么？
k8s的service定义了一个服务的访问入口地址，前端的应用通过这个入口地址访问其背后的一组由pod副本组成的集群实例。来自外部的访问请求被负载均衡到后端的各个容器应用上。Service与其后端Pod副本集群之间则是通过Label Selector来实现对接。
RC的作用相当于保证Service的服务能力和服务质量始终处于预期的标准。Service定义可以基于POST方式。请求apiserver创建新的实例。
例如：
1.创建一个新的nginx pod
[root@wf-01 ~]# kubectl run nginx-deploy --image=nginx:1.14-alpine --port=80 --replicas=1
2.查看创建的pod
[root@wf-01 ~]# kubectl get pod |grep nginx
nginx-deploy-84cbfc56b6-7v59n 1/1 Running 0 4m1s
3.查看deploy
[root@wf-01 ~]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 1/1 1 1 4m28s
4.查看是否可以访问
[root@wf-01 ~]# curl 172.20.0.64
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
- 这时我们访问的IP为Pod的IP，那么问题来了，如果我把Pod删除，则deploy控制器会重新启动一个pod以实现第一次创建Pod时指定的副本数为1的条件。这个时候pod ip会发生变化。显而这个不是我们期望的。所以需要创建一个service来保证pod的IP的变化对pod客户端(其它pod，集群内部客户端)来说是透明的。总而言之，service为pods提供固定的访问端点。
- 创建service, service有很多类型。如ClusterIP，NodePort，LoadBalancer或者ExternalName。默认为ClusterIP,意思是这个services IP只能被集群内的Pod客户端访问。
# 创建service
[root@wf-01 ~]# kubectl expose deployment nginx-deploy --name=nginx --port=80 --target-port=80 --protocol=TCP
service/nginx exposed
# 查看nginx的service
[root@wf-01 ~]# kubectl get svc |grep nginx
nginx ClusterIP 10.68.23.129 <none> 80/TCP 18s
# 访问
[root@wf-01 ~]# curl 10.68.23.129
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
- 如果集群部署了coredns，则集群内的节点DNS为coredns的地址。则可以通过service name直接解析到对应的IP
# 集群的coredns地址
[root@wf-01 ~]# kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.68.0.2 <none> 53/UDP,53/TCP,9153/TCP 5d18h
- 创建一个client(使用busybox)来验证是否可以实现dns功能
[root@wf-01 ~]# kubectl run client --image=busybox --replicas=1 -it --restart=Never
# 查看解析的dns地址
~ # cat /etc/resolv.conf
nameserver 10.68.0.2
search default.svc.cluster.local. svc.cluster.local. cluster.local.
options ndots:5
~ # wget -O - -q http://nginx:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
可以看到client获取的dns为default.svc.cluster.local。nginx服务的完整域名为: nginx.default.svc.cluster.local
- 手动删除一个nginx pod，deploy控制器会自动创建一个新的nginx pod调度至对应的node节点。这时使用busybox再次解析nginx，查看service是否生效
[root@wf-01 ~]# kubectl delete pod nginx-deploy-84cbfc56b6-7v59n
pod "nginx-deploy-84cbfc56b6-7v59n" deleted
[root@wf-01 ~]# kubectl get pod |grep nginx
nginx-deploy-84cbfc56b6-hcmwv 1/1 Running 0 30s
# 在busybox容器中执行下面操作查看
~ # wget -O - -q http://nginx:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
service深入理解
service是一个iptables或ipvs规则。使用标签选择器，将访问service_ip:port所有的都调度至相应的pod后端。
怎么知道service关联到哪些pod？
[root@wf-01 ~]# kubectl describe svc nginx
Name: nginx
Namespace: default
Labels: run=nginx-deploy
Annotations: <none>
Selector: run=nginx-deploy
Type: ClusterIP
IP: 10.68.23.129
Port: <unset> 80/TCP
TargetPort: 80/TCP
Endpoints: 172.20.0.66:80
Session Affinity: None
Events: <none>
其中上面的Endpoints:170.20.0.66:80就是关联的pods，而Selector: run=nginx=deploy就是选择相应的pod进行关联。下面查看nginx pod的标签是否为"run=nginx-deploy"
[root@wf-01 ~]# kubectl describe pods nginx-deploy-84cbfc56b6-hcmwv
Name: nginx-deploy-84cbfc56b6-hcmwv
Namespace: default
Node: 192.168.30.79/192.168.30.79
Start Time: Tue, 14 May 2019 10:53:11 +0800
Labels: pod-template-hash=84cbfc56b6
run=nginx-deploy
Annotations: <none>
Status: Running
IP: 172.20.0.66
Controlled By: ReplicaSet/nginx-deploy-84cbfc56b6
如果对应的pod资源被删除，则services后端关联的pod_ip会动态变化。我们也可以去改service的ip。则kube-dns内对应的几率也会动态变化。
- 一个创建好的deployment是可以动态改变副本数。也就是意味着可以实现动态扩容
# 创建一个双副本的deployment
[root@wf-01 ~]# kubectl run myapp --image=ikubernetes/myapp:v1 --replicas=2
deployment.apps/myapp created
# 查看deployment的副本信息
[root@wf-01 ~]# kubectl get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
myapp 2/2 2 2 37s
# 查看pod的详情
[root@wf-01 ~]# kubectl get pod -o wide |grep myapp
myapp-9b4987d5-jngkk 1/1 Running 0 89s 172.20.0.68 192.168.30.79 <none> <none>
myapp-9b4987d5-tcg4p 1/1 Running 0 89s 172.20.0.67 192.168.30.79 <none> <none>
- 使用busybox访问查看是否生效
~ # wget -O - -q 172.20.0.67
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
~ # wget -O - -q 172.20.0.68
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
~ # wget -O - -q 172.20.0.67/hostname.html
myapp-9b4987d5-tcg4p
~ # wget -O - -q 172.20.0.68/hostname.html
myapp-9b4987d5-jngkk
- 我们创建一个service来提供固定访问端点
[root@wf-01 ~]# kubectl expose deployment myapp --name=myapp --port=80
service/myapp exposed
[root@wf-01 ~]# kubectl get svc |grep myapp
myapp ClusterIP 10.68.20.152 <none> 80/TCP 12s
- 客户端再次访问，并测试负载均衡的效果
~ # wget -O - -q myapp
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
~ # wget -O - -q myapp.default.svc.cluster.local.
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
~ # while true;do wget -O - -q myapp/hostname.html; sleep 1;done
myapp-9b4987d5-tcg4p
myapp-9b4987d5-tcg4p
myapp-9b4987d5-tcg4p
myapp-9b4987d5-tcg4p
myapp-9b4987d5-jngkk
myapp-9b4987d5-jngkk
myapp-9b4987d5-jngkk
myapp-9b4987d5-jngkk
myapp-9b4987d5-jngkk
myapp-9b4987d5-tcg4p
myapp-9b4987d5-tcg4p
myapp-9b4987d5-jngkk
myapp-9b4987d5-tcg4p
myapp-9b4987d5-tcg4p
myapp-9b4987d5-tcg4p
- 验证service到底是什么？
[root@wf-01 ~]# iptables -vnL -t nat
Chain PREROUTING (policy ACCEPT 1 packets, 78 bytes)
pkts bytes target prot opt in out source destination
43607 6509K KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
689 43750 DOCKER all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
481 30744 CNI-HOSTPORT-DNAT all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT 1 packets, 78 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 7 packets, 420 bytes)
pkts bytes target prot opt in out source destination
1070K 64M KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
2950 211K DOCKER all -- * * 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
563K 34M CNI-HOSTPORT-DNAT all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT 7 packets, 420 bytes)
pkts bytes target prot opt in out source destination
696K 42M CNI-HOSTPORT-MASQ all -- * * 0.0.0.0/0 0.0.0.0/0 /* CNI portfwd requiring masquerade */
17 1126 MASQUERADE all -- * !docker0 172.17.0.0/16 0.0.0.0/0
1074K 65M KUBE-POSTROUTING all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes postrouting rules */
205K 12M RETURN all -- * * 172.20.0.0/16 172.20.0.0/16
49 3391 MASQUERADE all -- * * 172.20.0.0/16 !224.0.0.0/4
4 240 RETURN all -- * * !172.20.0.0/16 172.20.0.0/24
0 0 MASQUERADE all -- * * !172.20.0.0/16 172.20.0.0/16
0 0 CNI-fdae7f8826f90d050b196ad2 all -- * * 172.20.0.0/16 0.0.0.0/0 /* name: "mynet" id: "f8addbe803325c3cbfaa04f8635c3958eb01d98e08d2aabb6831ea429c627237" */
Chain CNI-DN-461bdacb794105fd5230f (1 references)
pkts bytes target prot opt in out source destination
0 0 CNI-HOSTPORT-SETMARK tcp -- * * 172.20.0.63 0.0.0.0/0 tcp dpt:3306
0 0 CNI-HOSTPORT-SETMARK tcp -- * * 127.0.0.1 0.0.0.0/0 tcp dpt:3306
0 0 DNAT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:3306 to:172.20.0.63:3306
Chain CNI-HOSTPORT-DNAT (2 references)
pkts bytes target prot opt in out source destination
0 0 CNI-DN-461bdacb794105fd5230f tcp -- * * 0.0.0.0/0 0.0.0.0/0 /* dnat name: "cbr0" id: "eb4ac0f03261276c559f1b981aa7c3e8fca1c0882bb36e53517d62905f500ea9" */ multiport dports 3306
Chain CNI-HOSTPORT-MASQ (1 references)
pkts bytes target prot opt in out source destination
0 0 MASQUERADE all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x2000/0x2000
Chain CNI-HOSTPORT-SETMARK (2 references)
pkts bytes target prot opt in out source destination
0 0 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 /* CNI portfwd masquerade mark */ MARK or 0x2000
service_ip存在于每个节点的iptables和ipvs。所以导致无法ping通。但是可以正常使用。如果想在集群外部访问到myapp，可以修改service类型为NodePort。
[root@wf-01 ~]# kubectl edit svc myapp
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2019-05-14T03:14:41Z"
labels:
run: myapp
name: myapp
namespace: default
resourceVersion: "863716"
selfLink: /api/v1/namespaces/default/services/myapp
uid: 69f9d888-75f6-11e9-8cb1-0050569931cd
spec:
clusterIP: 10.68.20.152
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: myapp
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}