今天这里来实现keepalived加lvs的dr模式，实现高可用。
keepalived用来管理lvs。
环境：
| 主机名 | IP | 系统 | 角色 |
| tiandong63 | 192.168.199.3 | rhel6.5、ipvsadm、keepalived | lvs server1（MASTER） |
| tiandong64 | 192.168.199.4 | rhel6.5、ipvsadm、keepalived | lvs server2（BACKUP） |
| tiandong65 |
rip：192.168.199.5 DG：192.168.199.1 vip：192.168.199.111 |
rhel7.4 | realserver1 |
| tiandong66 |
rip：192.168.199.6 DG：192.168.199.1 vip：192.168.199.111 |
rhel7.3 | realserver2 |
实战：
lvs server配置
[root@tiandong63 ~]# yum install ipvsadm keepalived -y
[root@tiandong64 ~]# yum install ipvsadm keepalived -y
[root@tiandong63 ~]# /etc/init.d/keepalived start
[root@tiandong64 ~]# /etc/init.d/keepalived start
[root@tiandong63 ~]# rpm -ql keepalived #查看keepalived的安装路径
[root@tiandong63 ~]# vim /etc/keepalived/keepalived.conf
1 ! Configuration File for keepalived
2
3 global_defs { #全局定义
4 notification_email { #定义邮件
5 root@localhost #定义邮件地址
6 }
7 notification_email_from root@localhost #定义邮件地址
8 smtp_server localhost #邮件服务器
9 smtp_connect_timeout 30 #邮件超时时间
10 router_id tiandong63 #router_id可以自己定义，但是必须唯一
11 }
12
13 vrrp_instance apache { #定义vrr组
14 state MASTER #vrrp实例的角色，MASTER必须大写
15 interface eth0 #对外访问的网络接口，和自己的一致
16 virtual_router_id 51 #虚拟路由器id必须和从的一致
17 priority 100 #主从优先级，主的要高于从
18 advert_int 1 #广播周期秒数
19 authentication {
20 auth_type PASS
21 auth_pass 1111
22 }
23 virtual_ipaddress {
24 192.168.199.111 #vip地址，真实环境这里应该是公网ip
25 }
26 }
27
28 virtual_server 192.168.199.111 80 { #:虚拟VIP地址 与 端口，DR架构WEB端口要和虚拟端口监听一致。否则将无法访问
29 delay_loop 6 #健康检查时间间隔，单位是秒
30 lb_algo rr #lvs算法
31 lb_kind DR #lvs的模式
32 nat_mask 255.255.255.0
33 protocol TCP #使用TCP协议
34 real_server 192.168.199.5 80 { #真实的ip
35 weight 1
36 TCP_CHECK {
37 connect_timeout 10 #连接超时时间
38 nb_get_retry 3
39 delay_before_retry 3
40 connect_port 80 #连接端口为80，要和上面的保持一致
41 }
42 }
43 real_server 192.168.199.6 80 { #真实的ip
44 weight 1
45 TCP_CHECK {
46 connect_timeout 10
47 nb_get_retry 3
48 delay_before_retry 3
49 connect_port 80
50 }
51 }
52 }
标红的就是配置文件需要修改的地方，在从上必须修改以下几个位置，其他配置一样，把配置文件拷贝到从上，然后修改：
10 router_id tiandong64
14 state BACKUP
17 priority 90
realserver配置（1和2上面都得配置，直接执行脚本就可以了。）
[root@tiandong65 ~]# more lvsdr.sh
#!/bin/bash
VIP=192.168.199.111
source /etc/init.d/functions
case $1 in
start)
echo 'start LVS of RealServer DR'
/sbin/ifconfig lo:1 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev lo:1
echo '1' > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo '2' > /proc/sys/net/ipv4/conf/lo/arp_announce
echo '1' > /proc/sys/net/ipv4/conf/all/arp_ignore
echo '2' > /proc/sys/net/ipv4/conf/all/arp_announce
;;
stop)
/sbin/ifconfig lo:1 down
echo 'Close LVS of RealServer DR'
echo '0' > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo '0' > /proc/sys/net/ipv4/conf/lo/arp_announce
echo '0' > /proc/sys/net/ipv4/conf/all/arp_ignore
echo '0' > /proc/sys/net/ipv4/conf/all/arp_announce
;;
*)
echo "Usage:$0 (start|stop)"
exit 1
esac
[root@tiandong65 ~]# ./lvsdr.sh start
测试：
此时lvs server的主为192.168.199.3，192.168.199.4是闲置的
可以在从上面测试：
[root@tiandong64 ~]# curl 192.168.199.111
this is 192.168.199.6
[root@tiandong64 ~]# curl 192.168.199.111
this is 192.168.199.5
[root@tiandong64 ~]# curl 192.168.199.111
this is 192.168.199.6
[root@tiandong64 ~]# curl 192.168.199.111
this is 192.168.199.5
在主上查看连接状态：
[root@tiandong63 ~]# ipvsadm -ln --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 192.168.199.111:80 4 23 0 1908 0
-> 192.168.199.5:80 2 11 0 928 0
-> 192.168.199.6:80 2 12 0 980 0
测试lvs server是否会负载：
停了主（192.168.199.3）上面的keepalived，看一下从（192.168.199.4）上面的是否会开启，是否会正常转发：
[root@tiandong63 ~]# /etc/init.d/keepalived stop 停止主上面的keepalived
Stopping keepalived: [ OK ]
[root@tiandong64 ~]# ip a 在192.168.199.4上面查看
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:c7:20:71 brd ff:ff:ff:ff:ff:ff
inet 192.168.199.4/24 brd 192.168.199.255 scope global eth0
inet 192.168.199.111/32 scope global eth0 #vip已经飘过来了
inet6 fe80::20c:29ff:fec7:2071/64 scope link
valid_lft forever preferred_lft forever
在192.168.199.3上面测试：（此时该主机为从）
[root@tiandong63 ~]# curl 192.168.199.111
this is 192.168.199.5
[root@tiandong63 ~]# curl 192.168.199.111
this is 192.168.199.6
[root@tiandong63 ~]# curl 192.168.199.111
this is 192.168.199.5
[root@tiandong63 ~]# curl 192.168.199.111
this is 192.168.199.6
在192.168.199.4上面查看连接状态：
[root@tiandong64 ~]# ipvsadm -ln --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 192.168.199.111:80 4 22 0 1848 0
-> 192.168.199.5:80 2 10 0 872 0
-> 192.168.199.6:80 2 12 0 976 0
当主上的keepalived恢复的话看一下状态：
当主恢复了之后，vip有飘到了192.168.199.3上面，因为优先级比较高。
[root@tiandong63 ~]# /etc/init.d/keepalived start
[root@tiandong63 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:38:0b:14 brd ff:ff:ff:ff:ff:ff
inet 192.168.199.3/24 brd 192.168.199.255 scope global eth0
inet 192.168.199.111/32 scope global eth0
inet6 fe80::20c:29ff:fe38:b14/64 scope link
valid_lft forever preferred_lft forever
测试realserver出现故障的现象：
当一台realserver的Apache服务出现故障时，是否会充lvs中剔除，
[root@tiandong63 ~]# ipvsadm -ln 正常情况下的状态。
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.199.111:80 rr
-> 192.168.199.5:80 Route 1 0 0
-> 192.168.199.6:80 Route 1 0 0
当有一台realserver出现故障：
[root@tiandong66 ~]# systemctl stop httpd 一台服务器的Apache服务故障了
[root@tiandong63 ~]# ipvsadm -ln 此时查看只有一台realsever了。
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.199.111:80 rr
-> 192.168.199.5:80 Route 1 0 0
可以查看lvs server上的日志：
[root@tiandong63 ~]# tail -f /var/log/messages 把故障的主机移除了。
Dec 16 12:52:50 tiandong63 Keepalived_vrrp[3571]: VRRP_Instance(apache) Received lower prio advert, forcing new election
Dec 16 12:52:51 tiandong63 Keepalived_vrrp[3571]: VRRP_Instance(apache) Entering MASTER STATE
Dec 16 12:52:51 tiandong63 Keepalived_vrrp[3571]: VRRP_Instance(apache) setting protocol VIPs.
Dec 16 12:52:51 tiandong63 Keepalived_vrrp[3571]: VRRP_Instance(apache) Sending gratuitous ARPs on eth0 for 192.168.199.111
Dec 16 12:52:51 tiandong63 Keepalived_healthcheckers[3570]: Netlink reflector reports IP 192.168.199.111 added
Dec 16 12:52:56 tiandong63 Keepalived_vrrp[3571]: VRRP_Instance(apache) Sending gratuitous ARPs on eth0 for 192.168.199.111
Dec 16 12:58:35 tiandong63 Keepalived_healthcheckers[3570]: TCP connection to [192.168.199.6]:80 failed !!!
Dec 16 12:58:35 tiandong63 Keepalived_healthcheckers[3570]: Removing service [192.168.199.6]:80 from VS [192.168.199.111]:80
Dec 16 12:58:35 tiandong63 Keepalived_healthcheckers[3570]: Remote SMTP server [0.0.0.0]:25 connected.
Dec 16 12:58:35 tiandong63 Keepalived_healthcheckers[3570]: SMTP alert successfully sent.
当real server恢复之后：
[root@tiandong66 ~]# systemctl start httpd 恢复Apache服务
[root@tiandong63 ~]# tail -f /var/log/messages 查看日志，把机器加入到lvs中了
Dec 16 12:58:35 tiandong63 Keepalived_healthcheckers[3570]: SMTP alert successfully sent.
Dec 16 13:01:35 tiandong63 Keepalived_healthcheckers[3570]: TCP connection to [192.168.199.6]:80 success.
Dec 16 13:01:35 tiandong63 Keepalived_healthcheckers[3570]: Adding service [192.168.199.6]:80 to VS [192.168.199.111]:80
Dec 16 13:01:35 tiandong63 Keepalived_healthcheckers[3570]: Remote SMTP server [0.0.0.0]:25 connected.
Dec 16 13:01:35 tiandong63 Keepalived_healthcheckers[3570]: SMTP alert successfully sent.
^C
[root@tiandong63 ~]# ipvsadm -ln #查看
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.199.111:80 rr
-> 192.168.199.5:80 Route 1 0 0
-> 192.168.199.6:80 Route 1 0 0
OK了，有什么问题随时欢迎讨论指教！！！！！
QQ：1127000383
192.168.199.7