DVR SNAT避免东西向流量的一种方案
来源:互联网 发布:大闹天宫进阶数据 编辑:程序博客网 时间:2024/06/05 15:08
http://www.ibm.com/developerworks/cn/cloud/library/1509_xuwei_dvr/
http://www.cnblogs.com/xingyun/archive/2015/10/25/4876083.html
http://www.weixinla.com/document/25263296.html
有个初步的想法,不知道是否可行;
每个有snat需求的vnet所在的计算节点上创建个共用的snat nm,占用一个openstack管理的外部ip供作snat使用;
多个租户可能会共享一个外部ip,一个租户的vm如果分散在多个计算节点上,会用掉多个外部ip;
每个计算节点针对SNAT维护一个公用的snat nm,每个租户创建network时,当前的计算节点没有snatnm时则创建它,这个snatnm占用的ip可以被当前计算节点的所有虚拟机共用;
每个虚拟网络在当前计算节点上都有内部vlan标识的,公用的snat nm上面有一个qg口和br-ex互联,下面有个turnk口和br-int2互联,针对每个虚拟网络创建一个eth:vlan的虚拟接口用于和虚拟网络的Vroute通信,他们之间走内部IP的静态路由(169.169.254.0的地址)
ovs作为DVR:
As part of my work in OpenDaylight, we are looking at creating a router using Open vSwitch... Why? Well OpenStack requires some limited L3 capabilities and we think that we can handle those in a distributed router.
Test Topology
My test topology looks like this:
We have a host in an external network 172.16.1.0/24
, one host in an internal network 10.10.10.0/24
and two hosts in another internal network 10.10.20.0/24
.
As such, The hosts in the 10.x.x.x
range should be able to speak to each other, but should not be able to speak to external hosts.
The host 10.10.10.2
has a floating IP of 172.16.1.10
and should be reachable on this address from the external 172.16.1.0/24
network. To do this, we'll use DNAT for traffic from 172.16.1.2 -> 172.16.1.10
and SNAT for traffic back from10.10.10.2 -> 172.16.1.2
If you'd like to recreate this topology you can checkout the OpenDaylight OVSDB project source on GitHub and:
vagrant up mininetvagrant ssh mininetcd /vagrant/resources/mininetsudo mn --custom topo.py --topo l3
The Pipeline
Our router is implemented using the following pipeline:
Table 0 - Classifier
In this table we work out what traffic is interesting for us before pushing it further along the pipeline
Table 100 - ACL
While we don't use this table today, the idea would be to filter traffic in this table (or series of tables) and to then resubmit to the classifier once we have scrubbed them.
Table 105 - ARP Responder
In this table we use some OVS-Jitsu to take an incoming ARP Request and turn it in to an ARP reply
Table 5 - L3 Rewrite
In this table, we make any L3 modifications we need to before a packet is routed
Table 10 - L3 Routing
The L3 routing table is where the routing magic happens. Here we modify the Source MAC address, Decrement the TTL and push to the L2 tables for forwarding
Table 15 - L3 Forwarding
In the L3 forwarding table we resolve a destination IP address to the correct MAC address for L2 forwarding.
Table 20 - L2 Rewrites
While we aren't using this table in this example, typically we would push/pop any L2 encapsulations here like a VLAN tag or a VXLAN/GRE Tunnel ID.
Table 25 - L2 Forwarding
In this table we do our L2 lookup and forward out the correct port. We also handle L2 BUM traffic here using OpenFlow Groups - no VLANs!
The Flows
To program the flows, paste the following in to mininet:
## Set Bridge to use OpenFlow 1.3sh ovs-vsctl set Bridge s1 "protocols=OpenFlow13"## Create Groupssh ovs-ofctl add-group -OOpenFlow13 s1 group_id=1,type=all,bucket=output:1sh ovs-ofctl add-group -OOpenFlow13 s1 group_id=2,type=all,bucket=output:2,4sh ovs-ofctl add-group -OOpenFlow13 s1 group_id=3,type=all,bucket=output:3## Table 0 - Classifier# Send ARP to ARP Respondersh ovs-ofctl add-flow -OOpenFlow13 s1 "table=0, priority=1000, dl_type=0x0806, actions=goto_table=105"# Send L3 traffic to L3 Rewrite Tablesh ovs-ofctl add-flow -OOpenFlow13 s1 "table=0, priority=100, dl_dst=00:00:5E:00:02:01, action=goto_table=5"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=0, priority=100, dl_dst=00:00:5E:00:02:02, action=goto_table=5"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=0, priority=100, dl_dst=00:00:5E:00:02:03, action=goto_table=5"# Send L3 to L2 Rewrite Tablesh ovs-ofctl add-flow -OOpenFlow13 s1 "table=0, priority=0, action=goto_table=20"## Table 5 - L3 Rewrites# Exclude connected subnetssh ovs-ofctl add-flow -OOpenFlow13 s1 "table=5, priority=65535, dl_type=0x0800, nw_dst=10.10.10.0/24 actions=goto_table=10"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=5, priority=65535, dl_type=0x0800, nw_dst=10.10.20.0/24 actions=goto_table=10"# DNATsh ovs-ofctl add-flow -OOpenFlow13 s1 "table=5, priority=100, dl_type=0x0800, nw_dst=172.16.1.10 actions=mod_nw_dst=10.10.10.2, goto_table=10"# SNATsh ovs-ofctl add-flow -OOpenFlow13 s1 "table=5, priority=100, dl_type=0x0800, nw_src=10.10.10.2, actions=mod_nw_src=172.16.1.10, goto_table=10"# If no rewrite needed, continue to table 10sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=5, priority=0, actions=goto_table=10"## Table 10 - IPv4 Routingsh ovs-ofctl add-flow -OOpenFlow13 s1 "table=10, dl_type=0x0800, nw_dst=10.10.10.0/24, actions=mod_dl_src=00:00:5E:00:02:01, dec_ttl, goto_table=15"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=10, dl_type=0x0800, nw_dst=10.10.20.0/24, actions=mod_dl_src=00:00:5E:00:02:02, dec_ttl, goto_table=15"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=10, dl_type=0x0800, nw_dst=172.16.1.0/24, actions=mod_dl_src=00:00:5E:00:02:03, dec_ttl, goto_table=15"# Explicit drop if cannot routesh ovs-ofctl add-flow -OOpenFlow13 s1 "table=10, priority=0, actions=output:0"## Table 15 - L3 Forwardingsh ovs-ofctl add-flow -OOpenFlow13 s1 "table=15, dl_type=0x0800, nw_dst=10.10.10.2, actions=mod_dl_dst:00:00:00:00:00:01, goto_table=20"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=15, dl_type=0x0800, nw_dst=10.10.20.2, actions=mod_dl_dst:00:00:00:00:00:02, goto_table=20"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=15, dl_type=0x0800, nw_dst=10.10.20.4, actions=mod_dl_dst:00:00:00:00:00:04, goto_table=20"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=15, dl_type=0x0800, nw_dst=172.16.1.2, actions=mod_dl_dst:00:00:00:00:00:03, goto_table=20"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=15, priority=0, actions=goto_table=20"## Table 20 - L2 Rewrite# Go to next tablesh ovs-ofctl add-flow -OOpenFlow13 s1 "table=20, priority=0, actions=goto_table=25"## Table 25 - L2 Forwarding# Use groups for BUM trafficsh ovs-ofctl add-flow -OOpenFlow13 s1 "table=25, in_port=1, dl_dst=01:00:00:00:00:00/01:00:00:00:00:00, actions=group=1"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=25, in_port=2, dl_dst=01:00:00:00:00:00/01:00:00:00:00:00, actions=group=2"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=25, in_port=3, dl_dst=01:00:00:00:00:00/01:00:00:00:00:00, actions=group=3"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=25, in_port=4, dl_dst=01:00:00:00:00:00/01:00:00:00:00:00, actions=group=2"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=25, dl_dst=00:00:00:00:00:01,actions=output=1"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=25, dl_dst=00:00:00:00:00:02,actions=output=2"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=25, dl_dst=00:00:00:00:00:03,actions=output=3"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=25, dl_dst=00:00:00:00:00:04,actions=output=4"## Table 105 - ARP Responder# Respond to ARP for Router Addressessh ovs-ofctl add-flow -OOpenFlow13 s1 "table=105, dl_type=0x0806, nw_dst=10.10.10.1, actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[], mod_dl_src:00:00:5E:00:02:01, load:0x2->NXM_OF_ARP_OP[], move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[], move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[], load:0x00005e000201->NXM_NX_ARP_SHA[], load:0x0a0a0a01->NXM_OF_ARP_SPA[], in_port"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=105, dl_type=0x0806, nw_dst=10.10.20.1, actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[], mod_dl_src:00:00:5E:00:02:02, load:0x2->NXM_OF_ARP_OP[], move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[], move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[], load:0x00005e000202->NXM_NX_ARP_SHA[], load:0xa0a1401->NXM_OF_ARP_SPA[], in_port"sh ovs-ofctl add-flow -OOpenFlow13 s1 "table=105, dl_type=0x0806, nw_dst=172.16.1.1, actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[], mod_dl_src:00:00:5E:00:02:03, load:0x2->NXM_OF_ARP_OP[], move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[], move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[], load:0x00005e000203->NXM_NX_ARP_SHA[], load:0xac100101->NXM_OF_ARP_SPA[], in_port"# Proxy ARP for all floating IPs go belowsh ovs-ofctl add-flow -OOpenFlow13 s1 "table=105, dl_type=0x0806, nw_dst=172.16.1.10, actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[], mod_dl_src:00:00:5E:00:02:03, load:0x2->NXM_OF_ARP_OP[], move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[], move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[], load:0x00005e000203->NXM_NX_ARP_SHA[], load:0xac10010a->NXM_OF_ARP_SPA[], in_port"# if we made it here, the arp packet is to be handled as any other regular L2 packetsh ovs-ofctl add-flow -OOpenFlow13 s1 "table=105, priority=0, action=resubmit(,20)"
If you'd like to paste these in without comments, you can use this Gist
Testing
We can run a pingall to test this out:
mininet> pingall*** Ping: testing ping reachabilityh1 -> h2 h3 h4h2 -> h1 X h4h3 -> X X Xh4 -> h1 h2 X*** Results: 41% dropped (7/12 received)
hosts 1,2 and 4 can speak to each other. Everything initiated by h3 is dropped (as expected) but h1 can speak to h3 (thanks to NAT). We can test our DNAT and Proxy ARP for Floating IP's using this command:
mininet> h3 ping 172.16.1.10PING 172.16.1.10 (172.16.1.10) 56(84) bytes of data.64 bytes from 172.16.1.10: icmp_seq=1 ttl=63 time=1.30 ms64 bytes from 172.16.1.10: icmp_seq=2 ttl=63 time=0.043 ms64 bytes from 172.16.1.10: icmp_seq=3 ttl=63 time=0.045 ms64 bytes from 172.16.1.10: icmp_seq=4 ttl=63 time=0.050 ms^C--- 172.16.1.10 ping statistics ---4 packets transmitted, 4 received, 0% packet loss, time 2999msrtt min/avg/max/mdev = 0.043/0.360/1.305/0.545 ms
Cool! It works!
Conclusion
So far we've implemented the nuts and bolts of routing, but we are missing one crucial piece - ICMP handling. Without this useful things like Path MTU Discovery won't work and neither will diagnostics tools like Ping and Traceroute. I think we can do this using a Open vSwitch but first we'll need to add some new NXM fields to ICMP Data and ICMP Checksum and to also make the existing OXM's writable through set_field
. I'm going to start talking to the OVS community about this to see if it's possible so watch this space!
@dave_tucker
Helpful Links and Further Reading
ovs-ofctl Man Page Neutron ARP Responder Write Up Address Resolution Protocol OpenFlow 1.3.1 Spec
本文转自http://dtucker.co.uk/hack/building-a-router-with-openvswitch.html
- DVR SNAT避免东西向流量的一种方案
- Openstack中的DVR Part1 -- 东西向流量处理
- 关于UDWorks和TI的DVR方案
- SSO的一种方案
- 一种流量成本节省60%以上的手机直播微信直播H5直播幼儿园直播方案
- 基于TI8168平台的16路D1分辨率的DVR方案
- hi3520d的DVR设计
- NVR、DVR的区别
- eclipse的一种配置方案
- win32 自动更新的一种方案
- 一种无线网络的搭建方案
- WebAPI的一种单元测试方案
- 一种场景的制作方案
- P2P流量的监控与管理方案
- 16路DVR智能视频监控方案
- DVR
- dvr
- 一种有效避免死锁的互斥锁设计
- MFC总结(7)--- 操作Ini文件 操作
- * daemon not running. starting it now * ADB server didn't ACK * failed to start daemon *
- ViewPager轮播效果
- 关于 PreparedStatement 返回值的问题
- Android开发UI之隐藏系统状态栏
- DVR SNAT避免东西向流量的一种方案
- mysql常用doc命令大全
- Xcode 6制作动态及静态Framework
- Android Appium Server从启动到case完成的活动分析
- 主activity报空指针异常java.lang.RuntimeException: Unable to resume activity
- linux 文本相关命令
- redis资料
- Tokumx 代替 Mongodb 群集部署
- 网易互联网2015年C++研发面经