09 saltstack生产实例--Haproxy+keepalived

1、如何理解haproxy+keepalived

https://www.cnblogs.com/zs-wei/p/9213961.html

HaProxy可以通过监听一个统一的端口对外提供能力，然后内部进行分发，相当于nginx

何保证前端程序使用单个ip又能保证后端的实际处理机器为多台，这就是KeepAlived的作用。

2、pkg、haproxy、keepalived的安装

视频配套的资源：https://github.com/unixhot/saltbook-code

1、pkg

init.sls依赖包安装

[root@linux-node1 /srv/salt/prod/modules]# cat pkg/init.sls 
pkg-init:
  pkg.installed:
    - names:
      - gcc
      - gcc-c++
      - glibc
      - make
      - autoconf
      - openssl
      - openssl-devel

2、haproxy

安装文件 init.sls

[root@linux-node1 /srv/salt/prod/modules]# cat haproxy/init.sls 
include:
  - modules.pkg.init

haproxy-install:
  file.managed:
    - name: /usr/local/src/haproxy-1.5.3.tar.gz
    - source: salt://modules/haproxy/files/haproxy-1.5.3.tar.gz
    - mode: 755
    - user: root
    - group: root

  cmd.run:
    - name: cd /usr/local/src && tar zxf haproxy-1.5.3.tar.gz && cd haproxy-1.5.3 && make TARGET=linux26 PREFIX=/usr/local/haproxy && make install PREFIX=/usr/local/haproxy
    - unless: test -d /usr/local/haproxy
    - require:
      - pkg: pkg-init
      - file: haproxy-install

/etc/init.d/haproxy:
  file.managed:
    - source: salt://modules/haproxy/files/haproxy.init
    - mode: 755
    - user: root
    - group: root
    - require:
      - cmd: haproxy-install

net.ipv4.ip_nonlocal_bind:
  sysctl.present:
    - value: 1

haproxy-config-dir:
  file.directory:
    - name: /etc/haproxy
    - mode: 755
    - user: root
    - group: root

haproxy-init:
  cmd.run:
    - name: chkconfig --add haproxy
    - unless: chkconfig --list | grep haproxy
    - require:
      - file: /etc/init.d/haproxy

内核参数

默认不能监听不在本地的ipv4

file配置

3、keepalived

配置文件 init.sls

[root@linux-node1 /srv/salt/prod/modules]# cat keepalived/init.sls 
include:
  - modules.pkg.init

keepalived-install:
  file.managed:
    - name: /usr/local/src/keepalived-1.2.17.tar.gz
    - source: salt://modules/keepalived/files/keepalived-1.2.17.tar.gz
    - mode: 755
    - user: root
    - group: root
  cmd.run:
    - name: cd /usr/local/src && tar zxf keepalived-1.2.17.tar.gz && cd keepalived-1.2.17 && ./configure --prefix=/usr/local/keepalived --disable-fwmark && make && make install
    - unless: test -d /usr/local/keepalived
    - require:
      - file: keepalived-install

/etc/sysconfig/keepalived:
  file.managed:
    - source: salt://modules/keepalived/files/keepalived.sysconfig
    - mode: 644
    - user: root
    - group: root

/etc/init.d/keepalived:
  file.managed:
    - source: salt://modules/keepalived/files/keepalived.init
    - mode: 755
    - user: root
    - group: root

keepalived-init:
  cmd.run:
    - name: chkconfig --add keepalived
    - unless: chkconfig --list | grep keepalived
    - require:
      - file: /etc/init.d/keepalived

/etc/keepalived:
  file.directory:
    - user: root
    - group: root

配置文件

4、test启动

修改index主页

# node1
[root@linux-node1 /srv/salt/prod]# cd /var/www/html/
[root@linux-node1 /var/www/html]# echo "linux node11111" > index.html

# node2
[root@linux-node2 /etc/my.cnf.d]# cd /var/www/html/
[root@linux-node2 /var/www/html]# echo "linux node2222" > index.htm

[root@linux-node1 ~]# salt * state.sls modules.haproxy.init saltenv=prod

[root@linux-node1 /srv/salt/prod/modules]# salt * state.sls modules.keepalived.init saltenv=prod

3、如何使用负载均衡

每个业务一个负载均衡

每个主机的负载均衡

负载均衡对内对外

作业

管理 Haproy+Keepalive 配置文件
必须使用jianja模板，配置不一样

4、haproxy-outside的配置

1、git上面的源代码

2、服务 sls

加入到系统服务中

[root@linux-node1 /srv/salt/prod]# cat lb-outside/haproxy-outside.sls
include:
  - modules.haproxy.init

haproxy-service:
  file.managed:
    - name: /etc/haproxy/haproxy.cfg
    - source: salt://lb-outside/files/haproxy-outside.cfg
    - user: root
    - group: root
    - mode: 644
  service.running:
    - name: haproxy
    - enable: True
    - reload: True
    - require:
      - cmd: haproxy-init
    - watch:
      - file: haproxy-service

3、配置file

[root@linux-node1 /srv/salt/prod]# cat lb-outside/files/haproxy-outside.cfg 
global
maxconn 100000
chroot /usr/local/haproxy
uid 99  
gid 99 
daemon
nbproc 1 
pidfile /usr/local/haproxy/logs/haproxy.pid 
log 127.0.0.1 local3 info

defaults
option http-keep-alive
maxconn 100000
mode http
timeout connect 5000ms
timeout client  50000ms
timeout server 50000ms

listen stats
mode http
bind 0.0.0.0:8888
stats enable
stats uri     /haproxy-status 
stats auth    haproxy:saltstack

frontend frontend_www_example_com
    bind 192.168.194.131:80
    mode http
    option httplog
    log global
    default_backend backend_www_example_com

backend backend_www_example_com
    option forwardfor header X-REAL-IP
    option httpchk HEAD / HTTP/1.0
    balance source
    server web-node1  192.168.194.131:8080 check inter 2000 rise 30 fall 15
    server web-node2  192.168.194.132:8080 check inter 2000 rise 30 fall 15

配置文件解释

# 全局
global
maxconn 100000
chroot /usr/local/haproxy
uid 99  
gid 99 
daemon
nbproc 1 
pidfile /usr/local/haproxy/logs/haproxy.pid 
log 127.0.0.1 local3 info

# 默认配置
defaults
option http-keep-alive
maxconn 100000
mode http
timeout connect 5000ms
timeout client  50000ms
timeout server 50000ms

# 状态页面
listen stats
mode http
bind 0.0.0.0:8888     # 端口8888
stats enable
stats uri     /haproxy-status 　　# url
stats auth    haproxy:saltstack　　# 用户名：密码

# 前端
frontend frontend_www_example_com
    bind 192.168.194.131:80
    mode http
    option httplog
    log global
    default_backend backend_www_example_com

# 前后端backend统一  backend_www_example_com
# 后端
backend backend_www_example_com
    option forwardfor header X-REAL-IP
    option httpchk HEAD / HTTP/1.0
    balance source
    server web-node1  192.168.194.131:8080 check inter 2000 rise 30 fall 15
    server web-node2  192.168.194.132:8080 check inter 2000 rise 30 fall 15

4、test

[root@linux-node1 /srv/salt/prod/lb-outside]# salt * state.sls lb-outside.haproxy-outside saltenv=prod

用户名haproxy

密码saltstack

5、haproxy-keepalived的配置

1、sls

使用jinja模板

[root@linux-node1 /srv/salt/prod/lb-outside]# cat haproxy-outside-keepalived.sls 
include:
  - modules.keepalived.init

keepalived-server:
  file.managed:
    - name: /etc/keepalived/keepalived.conf
    - source: salt://lb-outside/files/haproxy-outside-keepalived.conf
    - mode: 644
    - user: root
    - group: root
    - template: jinja
    {% if grains['fqdn'] == 'linux-node1.example.com' %}
    - ROUTEID: haproxy_ha
    - STATEID: MASTER
    - PRIORITYID: 150
    {% elif grains['fqdn'] == 'linux-node2.example.com' %}
    - ROUTEID: haproxy_ha
    - STATEID: BACKUP
    - PRIORITYID: 100
    {% endif %}

  service.running:
    - name: keepalived
    - enable: True
    - watch:
      - file: keepalived-server

master，backup

优先级不同 150,100

2、file

[root@linux-node1 /srv/salt/prod/lb-outside]# vim files/haproxy-outside-keepalived.conf 
! Configuration File for keepalived
global_defs {
   notification_email {
     saltstack@example.com
   }
   notification_email_from keepalived@example.com
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id {{ROUTEID}}   # 环境有多个haproxy要有不同的route id
}

vrrp_instance haproxy_ha {
state {{STATEID}}
interface eth0
    virtual_router_id 36    # 虚拟id
priority {{PRIORITYID}}     # 角色
    advert_int 1            # 优先级id
authentication {
auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
       192.168.194.131
    }
}

网卡名称 eth0

虚拟ip

3、test启动

[root@linux-node1 /srv/salt/prod]# salt * state.sls lb-outside.haproxy-outside-keepalived saltenv=prod

node1抢到144ip了

断开这个node1的keepalived，

node2抢到地址

4、top

[root@linux-node1 /srv/salt/prod/modules]# salt * state.highstate

redis，单独拿出来，多个业务都在用

之前配置过

https://www.cnblogs.com/venicid/p/11276232.html#_label7_0

master直接安装的

redis不对

redis路径

6、我们的课程体系

全运维自动化的课程：

OpenStack创建虚拟机

配置服务

zabbix去监控

saltstack部署这样的架构

代码自动化的布置上去

ELK日志的自动化收集

docker