k8s中安装redis6集群

注意:第二步不用操作了,yaml文件仅供参考

1.创建Namespace

kubectl apply -f Namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: cluster-redis

2.创建PersistentVolumeClaim #这一步不用操作了

kubectl apply -f PersistentVolumeClaim.yaml

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc1
  namespace: cluster-redis
spec:
  accessModes: 
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: "storage"
---  
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc2
  namespace: cluster-redis
spec:
  accessModes: 
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: "storage"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc3
  namespace: cluster-redis
spec:
  accessModes: 
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: "storage"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc4
  namespace: cluster-redis
spec:
  accessModes: 
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: "storage"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc5
  namespace: cluster-redis
spec:
  accessModes: 
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: "storage"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc6
  namespace: cluster-redis
spec:
  accessModes: 
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: "storage"

3.创建statefulset

kubectl apply -f ConfigMap.yaml
kubectl apply -f StatefulSet.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-cluster-configmap
  namespace: cluster-redis
data:
  update-node.sh: |
    #!/bin/sh
    REDIS_NODES="/data/nodes.conf"
    sed -i -e "/myself/ s/[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}/${POD_IP}/" ${REDIS_NODES}
    exec "$@"
  redis.conf: |+
    cluster-enabled yes
    cluster-require-full-coverage no
    cluster-node-timeout 15000
    cluster-config-file /data/nodes.conf
    cluster-migration-barrier 1
    appendonly yes
    protected-mode no

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-pod
  namespace: cluster-redis
spec:
  serviceName: redis-cluster-service
  replicas: 6
  selector:
    matchLabels:
      app: redis-pod
  template:
    metadata:
      labels:
        app: redis-pod
    spec:
      containers:
      - name: redis
        image: redis:6.2.1
        ports:
        - containerPort: 6379
          name: client
        - containerPort: 16379
          name: gossip
        command: ["/conf/update-node.sh", "redis-server", "/conf/redis.conf"]
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        volumeMounts:
        - name: conf
          mountPath: /conf
          readOnly: false
        - name: data
          mountPath: /data
          readOnly: false
      volumes:
      - name: conf
        configMap:
          name: redis-cluster-configmap
          defaultMode: 0755
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteMany" ]
      resources:
        requests:
          storage: 1Gi
      storageClassName: "storage" # 注意这个,事先创建好存储卷

4.创建service

kubectl apply -f Service.yaml

apiVersion: v1
kind: Service
metadata:
  name: redis-cluster-service
  namespace: cluster-redis
spec:
  type: ClusterIP
  ports:
  - port: 6379
    targetPort: 6379
    name: client
  - port: 16379
    targetPort: 16379
    name: gossip
  selector:
    app: redis-pod

5.初始化 Redis Cluster
注意如下这一步,分开来看:
(1) 命令: kubectl get pods -n cluster-redis -l app=redis-pod -o jsonpath='{range.items[*]}{.status.podIP}:6379 '|awk '{print $1" "$2" "$3" "$4" "$5" "$6" "}'
这个命令是获取到StatefulSet.yaml文件中生成的redis pod ip,最后那个awk是过滤到不需要的信息
比如执行kubectl get pods -n cluster-redis -l app=redis-pod -o jsonpath='{range.items[*]}{.status.podIP}:6379 '得到的结果如下:
10.0.2.122:6379 10.0.0.65:6379 10.0.2.123:6379 10.0.0.66:6379 10.0.0.69:6379 10.0.2.125:6379 :6379
可以看到最后有个多余的:6379是没用的,awk的作用就是去掉这个
(2) 命令:kubectl exec -it redis-pod-0 -n cluster-redis -- redis-cli --cluster create (pod ip) --cluster-replicas 1
这个命令是登录到其中一个pod中创建集群使用的

kubectl exec -it redis-pod-0 -n cluster-redis -- redis-cli --cluster create `kubectl get pods -n cluster-redis -l app=redis-pod -o jsonpath='{range.items[*]}{.status.podIP}:6379 '|awk '{print $1" "$2" "$3" "$4" "$5" "$6" "}'` --cluster-replicas 1

6.验证集群部署

kubectl exec -it redis-pod-0 -n cluster-redis -- redis-cli cluster info
for x in $(seq 0 5); do echo "redis-pod-$x"; kubectl exec redis-pod-$x -n cluster-redis -- redis-cli role; echo; done

7.注意事项
(1)StatefulSet类型的资源不能通过修改'replicas', 'template', and 'updateStrategy'等字段来进行更新,否则就会报错,具体如下:

The StatefulSet "redis-pod" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden

(2)生成的pod编号是从0开始的,依次是 redis-pod-0, redis-pod-1, redis-pod-2, redis-pod-3, redis-pod-4, redis-pod-5.
当删除一个pod时则会自动生成一个同名的pod.当然了,新生成的pod ip会发生变化

(3)查看pod对应的DNS域名

# for x in $(seq 0 5); do kubectl exec redis-pod-$x -n cluster-redis -- hostname -f; done
redis-pod-0.redis-cluster-service.cluster-redis.svc.cluster.local
redis-pod-1.redis-cluster-service.cluster-redis.svc.cluster.local
redis-pod-2.redis-cluster-service.cluster-redis.svc.cluster.local
redis-pod-3.redis-cluster-service.cluster-redis.svc.cluster.local
redis-pod-4.redis-cluster-service.cluster-redis.svc.cluster.local
redis-pod-5.redis-cluster-service.cluster-redis.svc.cluster.local

8.使用
项目中代码连接redis集群的话,使用的参数是: ClusterIP:6379