Neutron总结-linuxbridge+vxlan网络

来源:互联网 发布:win8安装版iso镜像mac 编辑:程序博客网 时间:2024/06/04 17:56

转载地址:http://blog.csdn.net/dylloveyou/article/details/70196632

本篇文章介绍如何规划及创建linuxbridge+vxlan网络,实现实例间及实例与外部的通讯。读者应对OpenStack及其网络服务Neutron有初步的了解。

1.规划网络

部署节点为一个controller节点(包含网络节点),两个compute节点。controller节点有3个网卡,分别为eth0(管理和API网络,CIDR为192.168.128.0/24)、eth1(租户网络,CIDR为10.10.10.0/24)、eth2(外部网络,CIDR为11.11.11.0/24,不设置IP);compute节点有2个网卡,分别为eth0(管理和API网络,CIDR为192.168.128.0/24)、eth1(租户网络,CIDR为10.10.10.0/24)。
这里写图片描述

这里用VMware Workstation下面的三个虚拟机分别作为controller、compute1、compute2节点,其网卡设置为:

  1. 每个主机的eth0网卡作为 API&Management 网络,使用的是“NAT模式”,可以连接到外网下载OpenStack软件
  2. 每个主机的eth1网卡作为 Tenant(VM) 网络,是承载VxLAN的底层网络,使用的是“仅主机模式 ”
  3. controller节点也作为网络节点,需要模拟OpenStack的外网,增加网卡eth2,这里也选择“仅主机模式” 
    这里写图片描述

具体IP设置为: 
controller节点

# The primary network interfaceauto eth0iface eth0 inet staticaddress 192.168.128.31netmask 255.255.255.0gateway 192.168.128.2# The provider network interfaceauto eth1iface eth1 inet staticaddress 10.10.10.10netmask 255.255.255.0# The external network interfaceauto eth2iface eth2 inet manual
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16

compute1节点

# The primary network interfaceauto eth0iface eth0 inet staticaddress 192.168.128.32netmask 255.255.255.0gateway 192.168.128.2# The provider network interfaceauto eth1iface eth1 inet staticaddress 10.10.10.11netmask 255.255.255.0
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12

compute2节点

# The primary network interfaceauto eth0iface eth0 inet staticaddress 192.168.128.34netmask 255.255.255.0gateway 192.168.128.2# The provider network interfaceauto eth1iface eth1 inet staticaddress 10.10.10.12netmask 255.255.255.0
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12

Neutron网络方案为:

  1. Neutron ML2的Type Driver为vxlan
  2. Neutron ML2的Mechanism Driver为LinuxBridge
  3. Neutron L2 Agent为LinuxBridge

这里写图片描述

2.创建虚拟网络

修改配置文件 
/etc/neutron/plugins/ml2/ml2_conf.ini 
这里写图片描述

/etc/neutron/l3_agent.ini 
这里写图片描述

/etc/neutron/dhcp_agent.ini 
这里写图片描述

controller节点 /etc/neutron/plugins/ml2/linuxbridge_agent.ini 
这里写图片描述

compute节点 /etc/neutron/plugins/ml2/linuxbridge_agent.ini 
securitygroup、vxlan的配置同上(注意local_ip是eth1网卡的IP地址) 
这里写图片描述

配置文件修改后,需要重启网络相关服务。

登陆到dashboard,可以查看网络服务信息。 
controller节点启动了 neturon-metadata-agent、neutron-l3-agent、neutron-linuxbridge-agent、neutron-dhcp-agent 
compute节点启动了 neutron-linuxbridge-agent 
这里写图片描述

创建vxlan100网络,网络地址为172.16.100.0/24 
这里写图片描述

创建vxlan101网络,网络地址为172.16.101.0/24 
这里写图片描述

创建外部网络(外部网络为flat网络),物理网络为external(和前面的配置一致,否则报错),网络地址为11.11.11.0/24 
这里写图片描述

创建路由器,把vxlan网络和外部网络连通 
这里写图片描述

我们的网络环境就准备好了。

3.测试网络

创建3个实例,2个选择vxlan100网络,1个选择vxlan101网络。 
其中:test1、test3被调度到compute1节点,test2被调度到compute2节点。 
这里写图片描述

查看网络拓扑。两个vxlan网络均通过路由器连接到了外网,并且两个vxlan网络也通过路由器连接。所以理论上test1、test2、test3能互通,并且也能连通外部网络。 
这里写图片描述

为了保证外部能ping通以及能ssh登陆到实例,需要在安全组那加两条规则 
这里写图片描述

下面测试网络的连通性: 
同一vxlan:172.16.100.11 ping 172.16.100.13 
这里写图片描述

不同vxlan:172.16.100.11 ping 172.16.101.12 
这里写图片描述

外网:172.16.100.11 ping 11.11.11.111 (确保外网相同网段有一台机器) 
这里写图片描述

目前的网络结构如下 
这里写图片描述

查看各个节点上面的虚拟网络设备 
controller节点

root@controller:~# brctl showbridge name bridge id       STP enabled interfacesbrq02444a0c-47      8000.2a7c30cba574   no      tap3de43a36-db                            tapd018e93e-83                            vxlan-100brq662b5cb3-38      8000.000c29c2468d   no      eth2                            tapec56cb87-d6brq68a5be3b-16      8000.467e8814dbf9   no      tapbc8ecf37-95                            tapc095787f-94                            vxlan-101virbr0      8000.52540057a3c4   yes     virbr0-nic
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11

compute1节点

root@compute1:~# brctl showbridge name bridge id       STP enabled interfacesbrq02444a0c-47      8000.7e309ef03102   no      tapa4604188-24                            vxlan-100brq68a5be3b-16      8000.12614635535c   no      tapf830c325-3a                            vxlan-101virbr0      8000.52540066ffc3   yes     virbr0-nic
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7

compute2节点

root@compute2:~# brctl showbridge name bridge id       STP enabled interfacesbrq02444a0c-47      8000.721f3ae01492   no      tap73ed9130-1e                            vxlan-100brq68a5be3b-16      8000.8a1f5b8c25ce   no      vxlan-101virbr0      8000.52540066ffc3   yes     virbr0-nic
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6

controller节点1个Router、2个DHCP分别在各自的namespace下

root@controller:~# ip netnsqdhcp-02444a0c-47d1-48ef-9de7-a8614a5fd799qdhcp-68a5be3b-1696-4bd6-9785-1c9954a714e9qrouter-a88fa0ad-5e9d-4f63-9044-82d247c73617
  • 1
  • 2
  • 3
  • 4

可以通过exec查看router细节

root@controller:~# ip netns exec qrouter-a88fa0ad-5e9d-4f63-9044-82d247c73617 ip a1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: qr-bc8ecf37-95@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000    link/ether fa:16:3e:b8:46:02 brd ff:ff:ff:ff:ff:ff    inet 172.16.101.1/24 brd 172.16.101.255 scope global qr-bc8ecf37-95       valid_lft forever preferred_lft forever    inet6 fe80::f816:3eff:feb8:4602/64 scope link        valid_lft forever preferred_lft forever3: qr-d018e93e-83@if11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000    link/ether fa:16:3e:81:ec:b1 brd ff:ff:ff:ff:ff:ff    inet 172.16.100.1/24 brd 172.16.100.255 scope global qr-d018e93e-83       valid_lft forever preferred_lft forever    inet6 fe80::f816:3eff:fe81:ecb1/64 scope link        valid_lft forever preferred_lft forever4: qg-ec56cb87-d6@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000    link/ether fa:16:3e:01:36:c6 brd ff:ff:ff:ff:ff:ff    inet 11.11.11.12/24 brd 11.11.11.255 scope global qg-ec56cb87-d6       valid_lft forever preferred_lft forever    inet6 fe80::f816:3eff:fe01:36c6/64 scope link        valid_lft forever preferred_lft forever
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25

查看router的路由

root@controller:~# ip netns exec qrouter-a88fa0ad-5e9d-4f63-9044-82d247c73617 routeKernel IP routing tableDestination     Gateway         Genmask         Flags Metric Ref    Use Ifacedefault         11.11.11.1      0.0.0.0         UG    0      0        0 qg-ec56cb87-d611.11.11.0      *               255.255.255.0   U     0      0        0 qg-ec56cb87-d6172.16.100.0    *               255.255.255.0   U     0      0        0 qr-d018e93e-83172.16.101.0    *               255.255.255.0   U     0      0        0 qr-bc8ecf37-95
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7

router的iptables

root@controller:~# ip netns exec qrouter-a88fa0ad-5e9d-4f63-9044-82d247c73617 iptables -t nat -S-P PREROUTING ACCEPT-P INPUT ACCEPT-P OUTPUT ACCEPT-P POSTROUTING ACCEPT-N neutron-l3-agent-OUTPUT-N neutron-l3-agent-POSTROUTING-N neutron-l3-agent-PREROUTING-N neutron-l3-agent-float-snat-N neutron-l3-agent-snat-N neutron-postrouting-bottom-A PREROUTING -j neutron-l3-agent-PREROUTING-A OUTPUT -j neutron-l3-agent-OUTPUT-A POSTROUTING -j neutron-l3-agent-POSTROUTING-A POSTROUTING -j neutron-postrouting-bottom-A neutron-l3-agent-POSTROUTING ! -i qg-ec56cb87-d6 ! -o qg-ec56cb87-d6 -m conntrack ! --ctstate DNAT -j ACCEPT-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat-A neutron-l3-agent-snat -o qg-ec56cb87-d6 -j SNAT --to-source 11.11.11.12-A neutron-l3-agent-snat -m mark ! --mark 0x2/0xffff -m conntrack --ctstate DNAT -j SNAT --to-source 11.11.11.12-A neutron-postrouting-bottom -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-l3-agent-snat
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21

可以通过exec查看dhcp细节 
vxlan100的dhcp

root@controller:~# ip netns exec qdhcp-02444a0c-47d1-48ef-9de7-a8614a5fd799 ip a1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: ns-3de43a36-db@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000    link/ether fa:16:3e:d5:60:09 brd ff:ff:ff:ff:ff:ff    inet 172.16.100.10/24 brd 172.16.100.255 scope global ns-3de43a36-db       valid_lft forever preferred_lft forever    inet 169.254.169.254/16 brd 169.254.255.255 scope global ns-3de43a36-db       valid_lft forever preferred_lft forever    inet6 fe80::f816:3eff:fed5:6009/64 scope link        valid_lft forever preferred_lft forever
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15

vxlan101的dhcp

root@controller:~# ip netns exec qdhcp-68a5be3b-1696-4bd6-9785-1c9954a714e9 ip a1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: ns-c095787f-94@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000    link/ether fa:16:3e:8a:e7:fb brd ff:ff:ff:ff:ff:ff    inet 172.16.101.10/24 brd 172.16.101.255 scope global ns-c095787f-94       valid_lft forever preferred_lft forever    inet 169.254.169.254/16 brd 169.254.255.255 scope global ns-c095787f-94       valid_lft forever preferred_lft forever    inet6 fe80::f816:3eff:fe8a:e7fb/64 scope link        valid_lft forever preferred_lft forever
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15

再看下更详细的网络结构 
controller节点 
这里写图片描述

compute节点 
这里写图片描述

如果需要从外部网络访问虚拟机,则要添加Floating IP 
test1添加floating IP为11.11.11.13 
这里写图片描述

测试网络连通: 
外网:11.11.11.111 ping 11.11.11.13

root@ubuntu:~# ip a1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000    link/ether 00:0c:29:0d:7d:0f brd ff:ff:ff:ff:ff:ff    inet 192.168.128.10/24 brd 192.168.128.255 scope global eth0       valid_lft forever preferred_lft forever    inet6 fe80::20c:29ff:fe0d:7d0f/64 scope link        valid_lft forever preferred_lft forever3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000    link/ether 00:0c:29:0d:7d:19 brd ff:ff:ff:ff:ff:ff    inet 192.168.1.10/24 brd 192.168.1.255 scope global eth1       valid_lft forever preferred_lft forever    inet6 fe80::20c:29ff:fe0d:7d19/64 scope link        valid_lft forever preferred_lft forever4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000    link/ether 00:0c:29:0d:7d:23 brd ff:ff:ff:ff:ff:ff    inet 11.11.11.111/24 brd 11.11.11.255 scope global eth2       valid_lft forever preferred_lft forever    inet6 fe80::20c:29ff:fe0d:7d23/64 scope link        valid_lft forever preferred_lft foreverroot@ubuntu:~# ping 11.11.11.13PING 11.11.11.13 (11.11.11.13) 56(84) bytes of data.64 bytes from 11.11.11.13: icmp_seq=1 ttl=63 time=9.12 ms64 bytes from 11.11.11.13: icmp_seq=2 ttl=63 time=3.40 ms64 bytes from 11.11.11.13: icmp_seq=3 ttl=63 time=2.74 ms64 bytes from 11.11.11.13: icmp_seq=4 ttl=63 time=1.82 ms
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31

外网:11.11.11.111 ssh登录到 11.11.11.13

root@ubuntu:~# ssh cirros@11.11.11.13cirros@11.11.11.13's password: $ ip a1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast qlen 1000    link/ether fa:16:3e:08:df:ad brd ff:ff:ff:ff:ff:ff    inet 172.16.100.11/24 brd 172.16.100.255 scope global eth0    inet6 fe80::f816:3eff:fe08:dfad/64 scope link        valid_lft forever preferred_lft forever
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13

查看controller节点router的变化

root@controller:~# ip netns exec qrouter-a88fa0ad-5e9d-4f63-9044-82d247c73617 ip a1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: qr-bc8ecf37-95@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000    link/ether fa:16:3e:b8:46:02 brd ff:ff:ff:ff:ff:ff    inet 172.16.101.1/24 brd 172.16.101.255 scope global qr-bc8ecf37-95       valid_lft forever preferred_lft forever    inet6 fe80::f816:3eff:feb8:4602/64 scope link        valid_lft forever preferred_lft forever3: qr-d018e93e-83@if11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000    link/ether fa:16:3e:81:ec:b1 brd ff:ff:ff:ff:ff:ff    inet 172.16.100.1/24 brd 172.16.100.255 scope global qr-d018e93e-83       valid_lft forever preferred_lft forever    inet6 fe80::f816:3eff:fe81:ecb1/64 scope link        valid_lft forever preferred_lft forever4: qg-ec56cb87-d6@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000    link/ether fa:16:3e:01:36:c6 brd ff:ff:ff:ff:ff:ff    inet 11.11.11.12/24 brd 11.11.11.255 scope global qg-ec56cb87-d6       valid_lft forever preferred_lft forever    inet 11.11.11.13/32 brd 11.11.11.13 scope global qg-ec56cb87-d6       valid_lft forever preferred_lft forever    inet6 fe80::f816:3eff:fe01:36c6/64 scope link        valid_lft forever preferred_lft forever
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27

iptables

root@controller:~# ip netns exec qrouter-a88fa0ad-5e9d-4f63-9044-82d247c73617 iptables -t nat -S-P PREROUTING ACCEPT-P INPUT ACCEPT-P OUTPUT ACCEPT-P POSTROUTING ACCEPT-N neutron-l3-agent-OUTPUT-N neutron-l3-agent-POSTROUTING-N neutron-l3-agent-PREROUTING-N neutron-l3-agent-float-snat-N neutron-l3-agent-snat-N neutron-postrouting-bottom-A PREROUTING -j neutron-l3-agent-PREROUTING-A OUTPUT -j neutron-l3-agent-OUTPUT-A POSTROUTING -j neutron-l3-agent-POSTROUTING-A POSTROUTING -j neutron-postrouting-bottom-A neutron-l3-agent-OUTPUT -d 11.11.11.13/32 -j DNAT --to-destination 172.16.100.11-A neutron-l3-agent-POSTROUTING ! -i qg-ec56cb87-d6 ! -o qg-ec56cb87-d6 -m conntrack ! --ctstate DNAT -j ACCEPT-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697-A neutron-l3-agent-PREROUTING -d 11.11.11.13/32 -j DNAT --to-destination 172.16.100.11-A neutron-l3-agent-float-snat -s 172.16.100.11/32 -j SNAT --to-source 11.11.11.13-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat-A neutron-l3-agent-snat -o qg-ec56cb87-d6 -j SNAT --to-source 11.11.11.12-A neutron-l3-agent-snat -m mark ! --mark 0x2/0xffff -m conntrack --ctstate DNAT -j SNAT --to-source 11.11.11.12-A neutron-postrouting-bottom -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-l3-agent-snat
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24

这样我们就创建并且测试了LinuxBridge+vxlan网络,验证了网络的连通性。并对Neutron虚拟网络实现的细节进行了描述,希望对大家有帮助。

在Neutron的学习总结过程中,参考了网络上面大量有价值的文档,在这里对无私分享的同学们表示衷心感谢!尤其是《每天5分钟学习OpenStack》,写的非常详细,本篇文档主要参考了里面的内容,在这里重点推荐:)。


原创粉丝点击