一、LVS-dr的基本原理
direct routing，它通过修改请求报文的目标MAC地址进行转发。
请求报文经由director发送至RS，那么就不能让除Director外的RS响应，因此，有三种解决方案：1.在上游路由器进行IP和MAC的绑定，2.在RS上做arptables，3.修改RS主机内核参数，也就是说vip别名在环回口，修改的内核参数能使vip对发来的请求不做处理。
响应报文不经由director，而是有vip直接响应cip，其中vip通过rip的MAC和gateway直接响应cip。
dr的基本规则：
（1）保证前端路由器将目标IP为VIP的请求报文发送给director
解决方案：
静态绑定
arptables
修改RS主机内核的参数
（2）RS的RIP可以使用私有地址；但也可以使用公网地址
（3）RS跟Director必须在同一物理网络中
（4）请求报文经由Director调度，但响应报文一定不能经由Director；
（5）不支持端口映射
（6）RS可以大多数OS
（7）RS的网关不能指向DIP
二、实验环境，局域网环境
Client：本机windows7 Director：CentOS 7.1 RealServer：node1，node2均为CentOS6.7 Director与RealServer的所有IP均为同一网段
拓扑如下：
三、配置
director： [root@localhost ~]# ifconfig ens33:0 192.168.1.15/32 broadcast 192.168.1.15 up [root@localhost ~]# route add -host 192.168.1.15 dev ens33:0 RS： node1： [root@jymlinux ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore [root@jymlinux ~]# echo 1 > /proc/sys/net/ipv4/conf/eth2/arp_ignore [root@jymlinux ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce [root@jymlinux ~]# echo 2 > /proc/sys/net/ipv4/conf/eth2/arp_announce [root@jymlinux ~]# ifconfig lo:0 192.168.1.15/32 broadcast 192.168.1.15 up [root@jymlinux ~]# route add -host 192.168.1.15 dev lo:0 node2： [root@jymlinux ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore [root@jymlinux ~]# echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore [root@jymlinux ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce [root@jymlinux ~]# echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce [root@jymlinux ~]# ifconfig lo:0 192.168.1.15/32 broadcast 192.168.1.15 up [root@jymlinux ~]# route add -host 192.168.1.15 dev lo:0 或写脚本并给予执行权限设置内核参数 [root@jymlinux ~]# vim lvsdrka.sh #!/bin/bash # case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/eth2/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/eth2/arp_announce ;; stop) echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/eth2/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/eth2/arp_announce ;; esac [root@jymlinux ~]# chmod +x lvsdrka.sh [root@jymlinux ~]# ./lvsdrka.sh start [root@jymlinux ~]# cat /proc/sys/net/ipv4/conf/all/arp_ignore 1 [root@jymlinux ~]# cat /proc/sys/net/ipv4/conf/all/arp_announce 2
两个内核参数：
arp_announce：arp通告
0：通告全部IP 默认
1：尽量避免将非本网络的地址通告给网络中的其他地址
2：总是用最佳本地地址通告网络
arp_ignore：arp响应
0：无论从哪个接口请求的，只要主机有这个地址，就会响应 默认
1：从哪个接口请求的地址，就从哪个接口的地址响应
2,3,4,,5,6,7,8,有9个级别，其余不常用
在lvs-dr中，我们一般使用arp_ignore=1 arp_announce=2
测试web服务
[root@localhost ~]# curl http://192.168.1.20 <h1>this is node1 <h1> [root@localhost ~]# curl http://192.168.1.21 <h1>this is node2 <h1>
配置集群规则
[root@localhost ~]# ipvsadm -A -t 192.168.1.15:80 -s rr [root@localhost ~]# ipvsadm -a -t 192.168.1.15:80 -r 192.168.1.20 -g [root@localhost ~]# ipvsadm -a -t 192.168.1.15:80 -r 192.168.1.21 -g
四、配置http与https双集群服务
此处网段改为192.168.3.0网段。
1、创建私有CA（以Director主机为例） [root@localhost ~]# cd /etc/pki/CA [root@localhost CA]# (umask 077; openssl genrsa -out private/cakey.pem 2048) Generating RSA private key, 2048 bit long modulus ..................+++ .......+++ e is 65537 (0x10001) [root@localhost CA]# touch index.txt [root@localhost CA]# echo 01 > serial [root@localhost CA]# openssl req -new -x509 -key private/cakey.pem -out cacert.pem -days 365 You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:CN State or Province Name (full name) []:XJ Locality Name (eg, city) [Default City]:XJ Organization Name (eg, company) [Default Company Ltd]:JJ Organizational Unit Name (eg, section) []:Ops Common Name (eg, your name or your server's hostname) []:CA Email Address []:ca.admin.com 2、RS主机申请证书 [root@jymlinux ~]# cd /etc/httpd/ [root@jymlinux httpd]# mkdir ssl [root@jymlinux httpd]# cd ssl [root@jymlinux ssl]# (umask 077; openssl genrsa -out httpd.key 2048) Generating RSA private key, 2048 bit long modulus ..............+++ ..................................+++ e is 65537 (0x10001) [root@jymlinux ssl]# openssl req -new -key httpd.key -out httpd.csr You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:CN State or Province Name (full name) []:XJ Locality Name (eg, city) [Default City]:XJ Organization Name (eg, company) [Default Company Ltd]:JJ Organizational Unit Name (eg, section) []:Ops Common Name (eg, your name or your server's hostname) []:CA Email Address []:rs1.admin.com Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: 3、将申请的证书发往CA [root@jymlinux ssl]# scp httpd.csr root@192.168.3.10:/root The authenticity of host '192.168.3.10 (192.168.3.10)' can't be established. RSA key fingerprint is ef:85:f8:aa:1c:de:41:5a:fd:93:8d:9f:83:f7:a2:ff. Are you sure you want to continue connecting (yes/no)? y Please type 'yes' or 'no': yes Warning: Permanently added '192.168.3.10' (RSA) to the list of known hosts. root@192.168.3.10's password: httpd.csr 100% 1013 1.0KB/s 00:00 4、CA签署并发证 [root@localhost CA]# openssl ca -in /root/httpd.csr -out /root/httpd.crt Using configuration from /etc/pki/tls/openssl.cnf Check that the request matches the signature Signature ok Certificate Details: Serial Number: 1 (0x1) Validity Not Before: Nov 9 13:48:21 2016 GMT Not After : Nov 9 13:48:21 2017 GMT Subject: countryName = CN stateOrProvinceName = XJ organizationName = JJ organizationalUnitName = Ops commonName = CA emailAddress = rs1.admin.com X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: 7E:FA:3A:6F:89:28:EF:D1:CF:5C:42:75:50:7B:C6:99:1D:98:91:B6 X509v3 Authority Key Identifier: keyid:91:9D:0E:8E:86:45:09:DE:C3:3F:63:61:C2:3D:CB:E1:E3:1C:F1:B6 Certificate is to be certified until Nov 9 13:48:21 2017 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated [root@localhost CA]# scp /root/httpd.crt root@192.168.3.20:/etc/httpd/ssl/ The authenticity of host '192.168.3.20 (192.168.3.20)' can't be established. RSA key fingerprint is e5:84:6c:f7:c0:60:3d:0b:39:b6:1e:12:0d:48:8b:07. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.3.20' (RSA) to the list of known hosts. root@192.168.3.20's password: httpd.crt 100% 4482 4.4KB/s 00:00 4、安装mod-ssl [root@jymlinux ~]# yum install mod_ssl 5、修改ssl的配置文件 [root@jymlinux ~]# cd /etc/httpd/conf.d/ [root@jymlinux conf.d]# vim ssl.conf DocumentRoot "/var/www/html" #启用 SSLCertificateFile /etc/httpd/ssl/httpd.crt #修改证书以及密钥的所在路径 SSLCertificateKeyFile /etc/httpd/ssl/httpd.key 6、重启httpd服务 [root@jymlinux conf.d]# service httpd restart 7、Director配置ipvsadm规则 #使用iptables在PREROUTING链上的MARK标记将http与https标记为一组 [root@localhost ~]# iptables -t mangle -A PREROUTING -d 192.168.3.15 -p tcp --dport 80 -j MARK --set-mark 10 [root@localhost ~]# iptables -t mangle -A PREROUTING -d 192.168.3.15 -p tcp --dport 443 -j MARK --set-mark 10 [root@localhost ~]# ipvsadm -A -f 10 -s rr [root@localhost ~]# ipvsadm -a -f 10 -r 192.168.3.20 -g [root@localhost ~]# ipvsadm -a -f 10 -r 192.168.3.21 -g