转载地址:http://blog.csdn.net/dylloveyou/article/details/70196632
本篇文章介绍如何规划及创建linuxbridge+vxlan网络,实现实例间及实例与外部的通讯。读者应对OpenStack及其网络服务Neutron有初步的了解。
1.规划网络
部署节点为一个controller节点(包含网络节点),两个compute节点。controller节点有3个网卡,分别为eth0(管理和API网络,CIDR为192.168.128.0/24)、eth1(租户网络,CIDR为10.10.10.0/24)、eth2(外部网络,CIDR为11.11.11.0/24,不设置IP);compute节点有2个网卡,分别为eth0(管理和API网络,CIDR为192.168.128.0/24)、eth1(租户网络,CIDR为10.10.10.0/24)。
这里用VMware Workstation下面的三个虚拟机分别作为controller、compute1、compute2节点,其网卡设置为:
- 每个主机的eth0网卡作为 API&Management 网络,使用的是“NAT模式”,可以连接到外网下载OpenStack软件
- 每个主机的eth1网卡作为 Tenant(VM) 网络,是承载VxLAN的底层网络,使用的是“仅主机模式 ”
- controller节点也作为网络节点,需要模拟OpenStack的外网,增加网卡eth2,这里也选择“仅主机模式”
具体IP设置为:
controller节点
# The primary network interfaceauto eth0iface eth0 inet staticaddress 192.168.128.31netmask 255.255.255.0gateway 192.168.128.2# The provider network interfaceauto eth1iface eth1 inet staticaddress 10.10.10.10netmask 255.255.255.0# The external network interfaceauto eth2iface eth2 inet manual
compute1节点
auto eth0iface eth0 inet staticaddress 192.168.128.32netmask 255.255.255.0gateway 192.168.128.2auto eth1iface eth1 inet staticaddress 10.10.10.11netmask 255.255.255.0
compute2节点
auto eth0iface eth0 inet staticaddress 192.168.128.34netmask 255.255.255.0gateway 192.168.128.2auto eth1iface eth1 inet staticaddress 10.10.10.12netmask 255.255.255.0
Neutron网络方案为:
- Neutron ML2的Type Driver为vxlan
- Neutron ML2的Mechanism Driver为LinuxBridge
- Neutron L2 Agent为LinuxBridge
2.创建虚拟网络
修改配置文件
/etc/neutron/plugins/ml2/ml2_conf.ini
/etc/neutron/l3_agent.ini
/etc/neutron/dhcp_agent.ini
controller节点 /etc/neutron/plugins/ml2/linuxbridge_agent.ini
compute节点 /etc/neutron/plugins/ml2/linuxbridge_agent.ini
securitygroup、vxlan的配置同上(注意local_ip是eth1网卡的IP地址)
配置文件修改后,需要重启网络相关服务。
登陆到dashboard,可以查看网络服务信息。
controller节点启动了 neturon-metadata-agent、neutron-l3-agent、neutron-linuxbridge-agent、neutron-dhcp-agent
compute节点启动了 neutron-linuxbridge-agent
创建vxlan100网络,网络地址为172.16.100.0/24
创建vxlan101网络,网络地址为172.16.101.0/24
创建外部网络(外部网络为flat网络),物理网络为external(和前面的配置一致,否则报错),网络地址为11.11.11.0/24
创建路由器,把vxlan网络和外部网络连通
我们的网络环境就准备好了。
3.测试网络
创建3个实例,2个选择vxlan100网络,1个选择vxlan101网络。
其中:test1、test3被调度到compute1节点,test2被调度到compute2节点。
查看网络拓扑。两个vxlan网络均通过路由器连接到了外网,并且两个vxlan网络也通过路由器连接。所以理论上test1、test2、test3能互通,并且也能连通外部网络。
为了保证外部能ping通以及能ssh登陆到实例,需要在安全组那加两条规则
下面测试网络的连通性:
同一vxlan:172.16.100.11 ping 172.16.100.13
不同vxlan:172.16.100.11 ping 172.16.101.12
外网:172.16.100.11 ping 11.11.11.111 (确保外网相同网段有一台机器)
目前的网络结构如下
查看各个节点上面的虚拟网络设备
controller节点
root@controller:~bridge name bridge id STP enabled interfacesbrq02444a0c-47 8000.2a7c30cba574 no tap3de43a36-db tapd018e93e-83 vxlan-100brq662b5cb3-38 8000.000c29c2468d no eth2 tapec56cb87-d6brq68a5be3b-16 8000.467e8814dbf9 no tapbc8ecf37-95 tapc095787f-94 vxlan-101virbr0 8000.52540057a3c4 yes virbr0-nic
compute1节点
root@compute1:~bridge name bridge id STP enabled interfacesbrq02444a0c-47 8000.7e309ef03102 no tapa4604188-24 vxlan-100brq68a5be3b-16 8000.12614635535c no tapf830c325-3a vxlan-101virbr0 8000.52540066ffc3 yes virbr0-nic
compute2节点
root@compute2:~bridge name bridge id STP enabled interfacesbrq02444a0c-47 8000.721f3ae01492 no tap73ed9130-1e vxlan-100brq68a5be3b-16 8000.8a1f5b8c25ce no vxlan-101virbr0 8000.52540066ffc3 yes virbr0-nic
controller节点1个Router、2个DHCP分别在各自的namespace下
root@controller:~qdhcp-02444a0c-47d1-48ef-9de7-a8614a5fd799qdhcp-68a5be3b-1696-4bd6-9785-1c9954a714e9qrouter-a88fa0ad-5e9d-4f63-9044-82d247c73617
可以通过exec查看router细节
root@controller:~1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: qr-bc8ecf37-95@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000 link/ether fa:16:3e:b8:46:02 brd ff:ff:ff:ff:ff:ff inet 172.16.101.1/24 brd 172.16.101.255 scope global qr-bc8ecf37-95 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:feb8:4602/64 scope link valid_lft forever preferred_lft forever3: qr-d018e93e-83@if11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000 link/ether fa:16:3e:81:ec:b1 brd ff:ff:ff:ff:ff:ff inet 172.16.100.1/24 brd 172.16.100.255 scope global qr-d018e93e-83 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe81:ecb1/64 scope link valid_lft forever preferred_lft forever4: qg-ec56cb87-d6@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether fa:16:3e:01:36:c6 brd ff:ff:ff:ff:ff:ff inet 11.11.11.12/24 brd 11.11.11.255 scope global qg-ec56cb87-d6 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe01:36c6/64 scope link valid_lft forever preferred_lft forever
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
查看router的路由
root@controller:~Kernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Ifacedefault 11.11.11.1 0.0.0.0 UG 0 0 0 qg-ec56cb87-d611.11.11.0 * 255.255.255.0 U 0 0 0 qg-ec56cb87-d6172.16.100.0 * 255.255.255.0 U 0 0 0 qr-d018e93e-83172.16.101.0 * 255.255.255.0 U 0 0 0 qr-bc8ecf37-95
router的iptables
root@controller:~# ip netns exec qrouter-a88fa0ad-5e9d-4f63-9044-82d247c73617 iptables -t nat -S-P PREROUTING ACCEPT-P INPUT ACCEPT-P OUTPUT ACCEPT-P POSTROUTING ACCEPT-N neutron-l3-agent-OUTPUT-N neutron-l3-agent-POSTROUTING-N neutron-l3-agent-PREROUTING-N neutron-l3-agent-float-snat-N neutron-l3-agent-snat-N neutron-postrouting-bottom-A PREROUTING -j neutron-l3-agent-PREROUTING-A OUTPUT -j neutron-l3-agent-OUTPUT-A POSTROUTING -j neutron-l3-agent-POSTROUTING-A POSTROUTING -j neutron-postrouting-bottom-A neutron-l3-agent-POSTROUTING ! -i qg-ec56cb87-d6 ! -o qg-ec56cb87-d6 -m conntrack ! --ctstate DNAT -j ACCEPT-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat-A neutron-l3-agent-snat -o qg-ec56cb87-d6 -j SNAT --to-source 11.11.11.12-A neutron-l3-agent-snat -m mark ! --mark 0x2/0xffff -m conntrack --ctstate DNAT -j SNAT --to-source 11.11.11.12-A neutron-postrouting-bottom -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-l3-agent-snat
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
可以通过exec查看dhcp细节
vxlan100的dhcp
root@controller:~1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: ns-3de43a36-db@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000 link/ether fa:16:3e:d5:60:09 brd ff:ff:ff:ff:ff:ff inet 172.16.100.10/24 brd 172.16.100.255 scope global ns-3de43a36-db valid_lft forever preferred_lft forever inet 169.254.169.254/16 brd 169.254.255.255 scope global ns-3de43a36-db valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fed5:6009/64 scope link valid_lft forever preferred_lft forever
vxlan101的dhcp
root@controller:~1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: ns-c095787f-94@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000 link/ether fa:16:3e:8a:e7:fb brd ff:ff:ff:ff:ff:ff inet 172.16.101.10/24 brd 172.16.101.255 scope global ns-c095787f-94 valid_lft forever preferred_lft forever inet 169.254.169.254/16 brd 169.254.255.255 scope global ns-c095787f-94 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe8a:e7fb/64 scope link valid_lft forever preferred_lft forever
再看下更详细的网络结构
controller节点
compute节点
如果需要从外部网络访问虚拟机,则要添加Floating IP
test1添加floating IP为11.11.11.13
测试网络连通:
外网:11.11.11.111 ping 11.11.11.13
root@ubuntu:~1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:0d:7d:0f brd ff:ff:ff:ff:ff:ff inet 192.168.128.10/24 brd 192.168.128.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe0d:7d0f/64 scope link valid_lft forever preferred_lft forever3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:0d:7d:19 brd ff:ff:ff:ff:ff:ff inet 192.168.1.10/24 brd 192.168.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe0d:7d19/64 scope link valid_lft forever preferred_lft forever4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:0d:7d:23 brd ff:ff:ff:ff:ff:ff inet 11.11.11.111/24 brd 11.11.11.255 scope global eth2 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe0d:7d23/64 scope link valid_lft forever preferred_lft foreverroot@ubuntu:~PING 11.11.11.13 (11.11.11.13) 56(84) bytes of data.64 bytes from 11.11.11.13: icmp_seq=1 ttl=63 time=9.12 ms64 bytes from 11.11.11.13: icmp_seq=2 ttl=63 time=3.40 ms64 bytes from 11.11.11.13: icmp_seq=3 ttl=63 time=2.74 ms64 bytes from 11.11.11.13: icmp_seq=4 ttl=63 time=1.82 ms
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
外网:11.11.11.111 ssh登录到 11.11.11.13
root@ubuntu:~cirros@11.11.11.13's password: $ ip a1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast qlen 1000 link/ether fa:16:3e:08:df:ad brd ff:ff:ff:ff:ff:ff inet 172.16.100.11/24 brd 172.16.100.255 scope global eth0 inet6 fe80::f816:3eff:fe08:dfad/64 scope link valid_lft forever preferred_lft forever
查看controller节点router的变化
root@controller:~1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: qr-bc8ecf37-95@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000 link/ether fa:16:3e:b8:46:02 brd ff:ff:ff:ff:ff:ff inet 172.16.101.1/24 brd 172.16.101.255 scope global qr-bc8ecf37-95 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:feb8:4602/64 scope link valid_lft forever preferred_lft forever3: qr-d018e93e-83@if11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000 link/ether fa:16:3e:81:ec:b1 brd ff:ff:ff:ff:ff:ff inet 172.16.100.1/24 brd 172.16.100.255 scope global qr-d018e93e-83 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe81:ecb1/64 scope link valid_lft forever preferred_lft forever4: qg-ec56cb87-d6@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether fa:16:3e:01:36:c6 brd ff:ff:ff:ff:ff:ff inet 11.11.11.12/24 brd 11.11.11.255 scope global qg-ec56cb87-d6 valid_lft forever preferred_lft forever inet 11.11.11.13/32 brd 11.11.11.13 scope global qg-ec56cb87-d6 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe01:36c6/64 scope link valid_lft forever preferred_lft forever
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
iptables
root@controller:~# ip netns exec qrouter-a88fa0ad-5e9d-4f63-9044-82d247c73617 iptables -t nat -S-P PREROUTING ACCEPT-P INPUT ACCEPT-P OUTPUT ACCEPT-P POSTROUTING ACCEPT-N neutron-l3-agent-OUTPUT-N neutron-l3-agent-POSTROUTING-N neutron-l3-agent-PREROUTING-N neutron-l3-agent-float-snat-N neutron-l3-agent-snat-N neutron-postrouting-bottom-A PREROUTING -j neutron-l3-agent-PREROUTING-A OUTPUT -j neutron-l3-agent-OUTPUT-A POSTROUTING -j neutron-l3-agent-POSTROUTING-A POSTROUTING -j neutron-postrouting-bottom-A neutron-l3-agent-OUTPUT -d 11.11.11.13/32 -j DNAT --to-destination 172.16.100.11-A neutron-l3-agent-POSTROUTING ! -i qg-ec56cb87-d6 ! -o qg-ec56cb87-d6 -m conntrack ! --ctstate DNAT -j ACCEPT-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697-A neutron-l3-agent-PREROUTING -d 11.11.11.13/32 -j DNAT --to-destination 172.16.100.11-A neutron-l3-agent-float-snat -s 172.16.100.11/32 -j SNAT --to-source 11.11.11.13-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat-A neutron-l3-agent-snat -o qg-ec56cb87-d6 -j SNAT --to-source 11.11.11.12-A neutron-l3-agent-snat -m mark ! --mark 0x2/0xffff -m conntrack --ctstate DNAT -j SNAT --to-source 11.11.11.12-A neutron-postrouting-bottom -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-l3-agent-snat
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
这样我们就创建并且测试了LinuxBridge+vxlan网络,验证了网络的连通性。并对Neutron虚拟网络实现的细节进行了描述,希望对大家有帮助。
在Neutron的学习总结过程中,参考了网络上面大量有价值的文档,在这里对无私分享的同学们表示衷心感谢!尤其是《每天5分钟学习OpenStack》,写的非常详细,本篇文档主要参考了里面的内容,在这里重点推荐:)。