Kubernetes 搭建 nacos 集群

一、为nacos-server创建service服务

# cat nacos-server-test3-svc.yaml 
apiVersion: v1
kind: Service
metadata:
  name: nacos-server-test3-svc
  namespace: test3
  labels:
    app: nacos-server-test3
spec:
  externalTrafficPolicy: Cluster
  ports:
  - name: 8848-tcp
    port: 8848
    targetPort: 8848
    protocol: TCP
  - name: 7848-tcp
    port: 7848
    targetPort: 7848
    protocol: TCP
  selector:
    app: nacos-server-test3
  type: NodePort

二、可以使用configmap挂载配置文件

挂载application.properties配置文件主要是让Nacos集群暴露metrics数据，给prometheus进行监控，访问地址是{ip}:8848/nacos/actuator/prometheus；若不需要监控可省略此步。

# cat nacos-server-test3-cm.yaml 
apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-test3-configmap
  namespace: test3
  labels:
    app: nacos-server-test3
data:
  application.properties: |
    server.servlet.contextPath=${SERVER_SERVLET_CONTEXTPATH:/nacos}
    server.contextPath=/nacos
    server.port=${NACOS_APPLICATION_PORT:8848}
    spring.datasource.platform=${SPRING_DATASOURCE_PLATFORM:""}
    nacos.cmdb.dumpTaskInterval=3600
    nacos.cmdb.eventTaskInterval=10
    nacos.cmdb.labelTaskInterval=300
    nacos.cmdb.loadDataAtStart=false
    db.num=${MYSQL_DATABASE_NUM:1}
    db.url.0=jdbc:mysql://${MYSQL_SERVICE_HOST}:${MYSQL_SERVICE_PORT:3306}/${MYSQL_SERVICE_DB_NAME}?${MYSQL_SERVICE_DB_PARAM:characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true}
    db.url.1=jdbc:mysql://${MYSQL_SERVICE_HOST}:${MYSQL_SERVICE_PORT:3306}/${MYSQL_SERVICE_DB_NAME}?${MYSQL_SERVICE_DB_PARAM:characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true}
    db.user=${MYSQL_SERVICE_USER}
    db.password=${MYSQL_SERVICE_PASSWORD}
    nacos.core.auth.system.type=${NACOS_AUTH_SYSTEM_TYPE:nacos}
    nacos.core.auth.default.token.expire.seconds=${NACOS_AUTH_TOKEN_EXPIRE_SECONDS:18000}
    nacos.core.auth.default.token.secret.key=${NACOS_AUTH_TOKEN:SecretKey012345678901234567890123456789012345678901234567890123456789}
    nacos.core.auth.caching.enabled=${NACOS_AUTH_CACHE_ENABLE:false}
    nacos.core.auth.enable.userAgentAuthWhite=${NACOS_AUTH_USER_AGENT_AUTH_WHITE_ENABLE:false}
    nacos.core.auth.server.identity.key=${NACOS_AUTH_IDENTITY_KEY:serverIdentity}
    nacos.core.auth.server.identity.value=${NACOS_AUTH_IDENTITY_VALUE:security}
    server.tomcat.accesslog.enabled=${TOMCAT_ACCESSLOG_ENABLED:false}
    server.tomcat.accesslog.pattern=%h %l %u %t "%r" %s %b %D
    server.tomcat.basedir=
    nacos.security.ignore.urls=${NACOS_SECURITY_IGNORE_URLS:/,/error,/**/*.css,/**/*.js,/**/*.html,/**/*.map,/**/*.svg,/**/*.png,/**/*.ico,/console-fe/public/**,/v1/auth/**,/v1/console/health/**,/actuator/**,/v1/console/server/**}
    management.metrics.export.elastic.enabled=false
    management.metrics.export.influx.enabled=false
    nacos.naming.distro.taskDispatchThreadCount=10
    nacos.naming.distro.taskDispatchPeriod=200
    nacos.naming.distro.batchSyncKeyCount=1000
    nacos.naming.distro.initDataRatio=0.9
    nacos.naming.distro.syncRetryDelay=5000
    nacos.naming.data.warmup=true
    management.endpoints.web.exposure.include=*

三、使用Statefulset的方式部署nacos-server集群

配置参数参考链接：https://hub.docker.com/r/nacos/nacos-server

数据源使用的是MySQL所以并未挂载磁盘，如果是单机部署也可以使用deployment

# cat nacos-server-test3-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos-server-test3
  namespace: test3
  labels:
    app: nacos-server-test3
spec:
  serviceName: nacos-server-test3-svc      # 为上面service的名称
  replicas: 3                              # 集群副本数量为3
  selector:
    matchLabels:
      app: nacos-server-test3
  template:
    metadata:
      labels:
        app: nacos-server-test3
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - nacos-server-test3
              topologyKey: "kubernetes.io/hostname"
      volumes:                             # 挂载configmap配置文件，name中不可以包含小数点
      - name: application-properties
        configMap:
          name: nacos-test3-configmap
      containers:
      - name: nacos-server-test3
        imagePullPolicy: IfNotPresent
        image: nacos/nacos-server:1.4.1
        resources:
          requests:
            cpu: 500m
            memory: 2048Mi
          limits:
            cpu: 2000m
            memory: 2058Mi 
        ports:
        - containerPort: 8848
          name: nacos-port
        readinessProbe:
          tcpSocket:
            port: 8848
          initialDelaySeconds: 30
          periodSeconds: 5
        livenessProbe:
          tcpSocket:
            port: 8848
          initialDelaySeconds: 30
          failureThreshold: 30
          periodSeconds: 10
        volumeMounts:                  # 使用subPath的方式将configmap挂载到/home/nacos/conf/application.properties文件
        - name: application-properties
          mountPath: /home/nacos/conf/application.properties
          subPath: "application.properties"
        env:                           # 一些环境变量，可参考上面的链接
        - name: NACOS_REPLICAS
          value: "3"
        - name: MYSQL_SERVICE_HOST
          value: "10.3.100.10"
        - name: MYSQL_SERVICE_PORT
          value: "3306"
        - name: MYSQL_SERVICE_DB_NAME
          value: "nacos"
        - name: MYSQL_SERVICE_USER
          value: "nacos"
        - name: MYSQL_SERVICE_PASSWORD
          value: "nacos123456"
        - name: NACOS_APPLICATION_PORT
          value: "8848"
        - name: MODE
          value: "cluster"
        - name: PREFER_HOST_MODE
          value: "hostname"
        - name: SPRING_DATASOURCE_PLATFORM
          value: "mysql"
        - name: NACOS_SERVERS        # conf/cluster.conf配置文件信息：podname.servicename.namespace.svc.cluster.local
          value: "nacos-server-test3-0.nacos-server-test3-svc.test3.svc.cluster.local:8848 nacos-server-test3-1.nacos-server-test3-svc.test3.svc.cluster.local:8848 nacos-server-test3-2.nacos-server-test3-svc.test3.svc.cluster.local:8848"

若是部署nacos单机环境则需要修改MODE参数，并把副本数修改为1：

        - name: MODE
          value: "standalone"

四、创建服务

访问的时候访问NodePort的IP加端口访问，或者再配置ingress

# kubectl apply -f nacos-server-test3-svc.yaml 

# kubectl apply -f nacos-server-test3-cm.yaml 

# kubectl apply -f nacos-server-test3-statefulset.yaml

# kubectl get pod -n test3
NAME                   READY   STATUS    RESTARTS   AGE
nacos-server-test3-0   1/1     Running   0          50m
nacos-server-test3-1   1/1     Running   0          51m
nacos-server-test3-2   1/1     Running   0          52m