环路拓扑
组成拓扑结构的脚本
构成连通脚本
ip netns add ns1 ovs-vsctl add-br br1 ovs-vsctl add-port br1 tap1 -- set Interface tap1 type=internal ip link set tap1 netns ns1 ip netns exec ns1 ip addr add 1.1.1.1/24 dev tap1 ip netns exec ns1 ip link set tap1 up ip netns add ns2 ovs-vsctl add-br br2 ovs-vsctl add-port br2 tap2 -- set Interface tap2 type=internal ip link set tap2 netns ns2 ip netns exec ns2 ip addr add 1.1.1.2/24 dev tap2 ip netns exec ns2 ip link set tap2 up ip netns add ns3 ovs-vsctl add-br br3 ovs-vsctl add-port br3 tap3 -- set Interface tap3 type=internal ip link set tap3 netns ns3 ip netns exec ns3 ip addr add 1.1.1.3/24 dev tap3 ip netns exec ns3 ip link set tap3 up ip link add br1-br2 type veth peer name br2-br1 ip link add br1-br3 type veth peer name br3-br1 ip link add br2-br3 type veth peer name br3-br2 ip link set dev br1-br2 up ip link set dev br2-br1 up ip link set dev br1-br3 up ip link set dev br3-br1 up ip link set dev br2-br3 up ip link set dev br3-br2 up #连接br1 <-> br2 ovs-vsctl add-port br1 br1-br2 ovs-vsctl add-port br2 br2-br1 #连接br1<->br3 ovs-vsctl add-port br1 br1-br3 ovs-vsctl add-port br3 br3-br1 #连接br2<->br3 一半 ovs-vsctl add-port br2 br2-br3
拓扑清理脚本
ip link del br1-br2 type veth peer name br2-br1 ip link del br1-br3 type veth peer name br3-br1 ip link del br2-br3 type veth peer name br3-br2 ovs-vsctl del-br br1 ovs-vsctl del-br br2 ovs-vsctl del-br br3 ip netns del ns1 ip netns del ns2 ip netns del ns3
上述脚本执行完成后br3 <-> br2 未完全连接，因此还未组成环路
此时测试连通性：
连通脚本的测试命令
ip netns exec ns1 ping -c 3 1.1.1.2 ip netns exec ns1 ping -c 4 1.1.1.3
测试结果
root@controller-VirtualBox:~# ip netns exec ns1 ping -c 3 1.1.1.2 PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data. 64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=0.225 ms 64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.049 ms 64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.051 ms --- 1.1.1.2 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2031ms rtt min/avg/max/mdev = 0.049/0.108/0.225/0.082 ms root@controller-VirtualBox:~# ip netns exec ns1 ping -c 4 1.1.1.3 PING 1.1.1.3 (1.1.1.3) 56(84) bytes of data. 64 bytes from 1.1.1.3: icmp_seq=1 ttl=64 time=0.219 ms 64 bytes from 1.1.1.3: icmp_seq=2 ttl=64 time=0.051 ms 64 bytes from 1.1.1.3: icmp_seq=3 ttl=64 time=0.046 ms 64 bytes from 1.1.1.3: icmp_seq=4 ttl=64 time=0.050 ms --- 1.1.1.3 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3065ms rtt min/avg/max/mdev = 0.046/0.091/0.219/0.074 ms
将br2和br3连通组成二层环路
ovs-vsctl add-port br3 br3-br2
环路后进行ping连通测试结果,时通时不通
root@controller-VirtualBox:~# ip netns exec ns1 ping -c 30 1.1.1.2 PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data. 64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=0.436 ms --- 1.1.1.2 ping statistics --- 18 packets transmitted, 1 received, 94% packet loss, time 17402ms rtt min/avg/max/mdev = 0.436/0.436/0.436/0.000 ms
环路情况下分析
-
查询接口名称和接口索引的对应关系
root@controller-VirtualBox:~# ovs-ofctl show br1 OFPT_FEATURES_REPLY (xid=0x2): dpid:0000bee79fe0e948 n_tables:254, n_buffers:0 capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst 1(tap1): addr:ff:7f:00:00:36:38 config: PORT_DOWN state: LINK_DOWN speed: 0 Mbps now, 0 Mbps max 2(br1-br2): addr:b2:86:95:fa:4b:b9 config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max 3(br1-br3): addr:12:a5:67:28:40:f1 config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max LOCAL(br1): addr:be:e7:9f:e0:e9:48 config: PORT_DOWN state: LINK_DOWN speed: 0 Mbps now, 0 Mbps max OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0 root@controller-VirtualBox:~# ovs-ofctl show br2 OFPT_FEATURES_REPLY (xid=0x2): dpid:00007a10ec6b6942 n_tables:254, n_buffers:0 capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst 1(tap2): addr:ff:7f:00:00:36:38 config: PORT_DOWN state: LINK_DOWN speed: 0 Mbps now, 0 Mbps max 2(br2-br1): addr:0e:dd:29:5c:73:ec config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max 3(br2-br3): addr:4a:73:b5:67:af:fe config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max LOCAL(br2): addr:7a:10:ec:6b:69:42 config: PORT_DOWN state: LINK_DOWN speed: 0 Mbps now, 0 Mbps max OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0 root@controller-VirtualBox:~# ovs-ofctl show br3 OFPT_FEATURES_REPLY (xid=0x2): dpid:0000462c85b99a4d n_tables:254, n_buffers:0 capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst 1(tap3): addr:ff:7f:00:00:36:38 config: PORT_DOWN state: LINK_DOWN speed: 0 Mbps now, 0 Mbps max 2(br3-br1): addr:ca:ed:37:ef:7d:21 config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max 3(br3-br2): addr:ea:55:a1:2e:d9:d1 config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max LOCAL(br3): addr:46:2c:85:b9:9a:4d config: PORT_DOWN state: LINK_DOWN speed: 0 Mbps now, 0 Mbps max OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0 -
查看ns1的arp表项，发现已经无法学习到1.1.1.2 mac地址
root@controller-VirtualBox:~# ip netns exec ns1 arp -n Address HWtype HWaddress Flags Mask Iface 1.1.1.3 ether 72:e4:76:db:2e:c5 C tap1 1.1.1.2 (incomplete) tap1
-
分别查看br1、br2、br3的mac转发表，发现br1和br2表项已经错乱
root@controller-VirtualBox:~# ovs-appctl fdb/show br1 port VLAN MAC Age 2 0 3a:52:a3:d6:67:12 0 3 0 b2:86:95:fa:4b:b9 0 3 0 02:a1:bd:ae:56:75 0 root@controller-VirtualBox:~# ovs-appctl fdb/show br2 port VLAN MAC Age 3 0 3a:52:a3:d6:67:12 0 2 0 b2:86:95:fa:4b:b9 0 2 0 02:a1:bd:ae:56:75 0 root@controller-VirtualBox:~# ovs-appctl fdb/show br3 port VLAN MAC Age 2 0 3a:52:a3:d6:67:12 0 3 0 b2:86:95:fa:4b:b9 0 2 0 02:a1:bd:ae:56:75 0表项错乱说明：
br1连接ns1的tap1本应该从接口1学习到，结果从接口三学习到
br2连接ns2的tap2本应该从接口1学习到，结果从接口三学习到开启ovs stp解决上述二层环路问题
ovs-vsctl list bridge查看stp默认是关闭的
开启stp
ovs-vsctl set Bridge br1 stp_enable=true ovs-vsctl set Bridge br2 stp_enable=true ovs-vsctl set Bridge br3 stp_enable=true
经过测试网络恢复正常
root@controller-VirtualBox:~# ip netns exec ns1 ping -c 2 1.1.1.2 PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data. 64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=0.272 ms 64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.057 ms --- 1.1.1.2 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1023ms rtt min/avg/max/mdev = 0.057/0.164/0.272/0.108 ms root@controller-VirtualBox:~# ip netns exec ns1 ping -c 3 1.1.1.3 PING 1.1.1.3 (1.1.1.3) 56(84) bytes of data. 64 bytes from 1.1.1.3: icmp_seq=1 ttl=64 time=0.463 ms 64 bytes from 1.1.1.3: icmp_seq=2 ttl=64 time=0.068 ms 64 bytes from 1.1.1.3: icmp_seq=3 ttl=64 time=0.050 ms
stp研究
ovs-vsctl list bridge 获取到如下信息
br3 stp_root_path_cost=”0”
br2 stp_root_path_cost=”2”
br1 stp_root_path_cost=”2”‘
在br3-br1处抓包，并且令开窗口ns1 ping ns2
窗口1： ns1 ping ns2
root@controller-VirtualBox:~# ping 1.1.1.2 PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data. 64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=0.502 ms 64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.078 ms
窗口2：抓取到报文流量经过了br3
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on br3-br1, link-type EN10MB (Ethernet), capture size 262144 bytes 15:08:46.538492 IP 1.1.1.1 > 1.1.1.2: ICMP echo request, id 4525, seq 3, length 64 15:08:46.538524 IP 1.1.1.2 > 1.1.1.1: ICMP echo reply, id 4525, seq 3, length 64
通过测试得知上述环形网络，已经被逻辑修剪为如下拓扑图，从此解决了ovs二层环路问题
组成拓扑结构的脚本
stp相关命令总结
-
开启br1 stp
ovs-vsctl set bridge br1 tp_enable=ture
-
关闭br1 stp
ovs-vsctl set Bridge br1 stp_enable=false
-
设置br1 stp cost
ovs-vsctl set Port br2-br1 other_config:stp-path-cost=100
-
查询br1 stp状态
ovs-vsctl get bridge br1 stp_enable
-
设置br1的stp优先级
ovs-vsctl set bridge br1 other_config:stp-priority=0x7800
-
移除br1 stp设置
ovs-vsctl clear bridge br100 other_config