How to run strongswan vpnaas driver (by quqi99)

作者：张华  发表于：2015-03-12
版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明

( http://blog.csdn.net/quqi99 )

A layer two VPN tunnel connects broadcast domains, not subnets. Also, tunneling the same subnet in different namespaces does not magically convert the layer three VPN to a layer two VPN.

1, Use devstack to install a all-in-one openstack env with strongswan driver in 5 minutes.
   NOTE:
   a, Only supports ubuntu 14.04 now, devstack will help us disable apparmor in 14.04 by the following commands:
      sudo ln -sf /etc/apparmor.d/usr.lib.ipsec.charon /etc/apparmor.d/disable/
      sudo ln -sf /etc/apparmor.d/usr.lib.ipsec.stroke /etc/apparmor.d/disable/
      # NOTE: Due to https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1387220
      # one must use 'sudo start apparmor ACTION=reload' for Ubuntu 14.10
      sudo service apparmor restart
   b, No any network related requirements for your node. devstack will create br-phy and br-ex automatically.
      Both virtual machine and physical machine are ok, any network configurations are ok.
      But need to be able to access to the Internet.
   c, Assume openstack work directory is: /bak/openstack, make sure you are using a non-root user like hua,
      then change the file permissions for /bak/openstack directory ( sudo chown -R hua:root /bak/openstack ).

   The next steps are below:
   a, Download devstack
      cd /bak/openstack && git clone https://github.com/openstack-dev/devstack.git
   b, Paste the following content to the file /bak/openstack/devstack/localrc ,
      then run devstack by the command:
      cd /bak/openstack/devstack && ./stack.sh

# Use strongswan driver code, https://review.openstack.org/#/c/144391/
DEST=/bak/openstack
IPSEC_PACKAGE=strongswan

sudo route del -net 10.0.1.0/24 gw 192.168.101.3
sudo apt-get install openvswitch-switch qemu-kvm libvirt-bin
sudo ovs-vsctl -- --may-exist add-br br-phy
sudo ifconfig br-phy 172.16.1.1/24
ENABLED_SERVICES=rabbit,mysql,key,g-api,g-reg,tempest
ENABLED_SERVICES+=,n-api,n-crt,n-obj,n-cpu,n-cond,n-sch
ENABLED_SERVICES+=,cinder,c-api,c-vol,c-sch
ENABLED_SERVICES+=,q-svc,q-agt,q-dhcp,q-l3,q-meta,neutron,q-lbaas,q-fwaas,q-vpn

HOST_IP=172.16.1.1
SERVICE_HOST=$HOST_IP
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
Q_HOST=$SERVICE_HOST

FIXED_RANGE=10.0.1.0/24
FLOATING_RANGE=192.168.101.0/24
Q_FLOATING_ALLOCATION_POOL=start=192.168.101.3,end=192.168.101.100
PUBLIC_NETWORK_GATEWAY=192.168.101.1
NETWORK_GATEWAY=10.0.1.1
PUBLIC_BRIDGE=br-ex
# sudo ovs-vsctl add-port br-ex eth0
OVS_PHYSICAL_BRIDGE=br-phy

DATABASE_USER=root
DATABASE_PASSWORD=password
ADMIN_PASSWORD=password
SERVICE_PASSWORD=password
RABBIT_PASSWORD=password
SERVICE_TOKEN=ADMIN
LOGFILE=$DEST/logs/stack.log
ENABLE_DEBUG_LOG_LEVEL=False
SYSLOG=False
SCREEN_LOGDIR=$DEST/logs
LOG_COLOR=False
Q_USE_DEBUG_COMMAND=False
APACHE_ENABLED_SERVICES+=keystone
KEYSTONE_TOKEN_FORMAT=uuid
USE_SSL=False
disable_service tls-proxy
IMAGE_URLS+=",http://cdn.download.cirros-cloud.net/0.3.2/cirros-0.3.2-x86_64-disk.img"


2, I often use vpnaas tempest scenario test (https://review.openstack.org/#/c/153292/) to speed up to set up vpn env.
   But this time I will demo how to use CLI commands to set up vpn tunnel between two routers with two namespaces in a all-in-one env.
     (10.0.2.0/24 - DevStack East)
              |
              |  10.0.2.1
     [Neutron Router]
              |  192.168.101.4
              |
              |  192.168.101.1
     [Internet GW]
              |  
              |
     [Internet GW]
              | 192.168.101.1
              |
              | 192.168.101.5
     [Nuetron Router]
              |  10.0.3.1
              |
     (10.0.3.0/24 DevStack West)

a, Create network elements On East and West
export OS_USERNAME=admin
export OS_PASSWORD=password
export OS_TENANT_NAME=demo
export OS_AUTH_URL=http://172.16.1.1:5000/v2.0
export OS_AUTH_STRATEGY=keystone

TENANT_ID=$(keystone tenant-list |grep ' admin ' |awk '{print $2}')
EXT_NET_ID=$(neutron net-list |grep ' public ' |awk '{print $2}')
neutron net-create vpn-net-east --provider:network_type vxlan --provider:segmentation_id 1012
neutron subnet-create --tenant_id $TENANT_ID --ip_version 4 --gateway 10.0.2.1 vpn-net-east 10.0.2.0/24
neutron router-create --tenant_id $TENANT_ID vpn-router-east
EAST_ROUTER_ID=$(neutron router-list |grep ' vpn-router-east ' |awk '{print $2}')
EAST_SUBNET_ID=$(neutron subnet-list |grep '10.0.2.0/24' |awk '{print $2}')
neutron router-interface-add $EAST_ROUTER_ID $EAST_SUBNET_ID
# devstack has helped us create public network
# neutron net-create public -- --router:external=True
# neutron subnet-create --allocation-pool start=192.168.101.3,end=192.168.101.100 --gateway 192.168.101.1 public 192.168.101.0/24 --enable_dhcp=False
neutron router-gateway-set $EAST_ROUTER_ID $EXT_NET_ID

TENANT_ID=$(keystone tenant-list |grep ' admin ' |awk '{print $2}')
EXT_NET_ID=$(neutron net-list |grep ' public ' |awk '{print $2}')
neutron net-create vpn-net-west --provider:network_type vxlan --provider:segmentation_id 1013
neutron subnet-create --tenant_id $TENANT_ID --ip_version 4 --gateway 10.0.3.1 vpn-net-west 10.0.3.0/24
neutron router-create --tenant_id $TENANT_ID vpn-router-west
WEST_ROUTER_ID=$(neutron router-list |grep ' vpn-router-west ' |awk '{print $2}')
WEST_SUBNET_ID=$(neutron subnet-list |grep '10.0.3.0/24' |awk '{print $2}')
neutron router-interface-add $WEST_ROUTER_ID $WEST_SUBNET_ID
neutron router-gateway-set $WEST_ROUTER_ID $EXT_NET_ID

b, Create two VMs On East and West
   nova keypair-add --pub_key ~/.ssh/id_rsa.pub mykey
   IMAGE_ID=$(nova image-list |grep 'cirros' |awk '{print $2}')
   nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
   nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0

   EAST_NET_ID=$(neutron net-list |grep ' vpn-net-east ' |awk '{print $2}')
   nova boot --key_name mykey --image $IMAGE_ID --flavor 1 --nic net-id=$EAST_NET_ID vpn-vm-east
   EAST_VM_ID=$(nova list |grep 'vpn-vm-east' |awk '{print $2}')
   FLOATING_IP_EAST=$(nova floating-ip-create |grep ' public ' |awk '{print $4}')
   nova add-floating-ip $EAST_VM_ID $FLOATING_IP_EAST
   ssh -i mykey.priv cirros@$FLOATING_IP_EAST

   WEST_NET_ID=$(neutron net-list |grep ' vpn-net-west ' |awk '{print $2}')
   nova boot --key_name mykey --image $IMAGE_ID --flavor 1 --nic net-id=$WEST_NET_ID vpn-vm-west
   WEST_VM_ID=$(nova list |grep 'vpn-vm-west' |awk '{print $2}')
   FLOATING_IP_WEST=$(nova floating-ip-create |grep 'public' |awk '{print $4}')
   nova add-floating-ip $WEST_VM_ID $FLOATING_IP_WEST
   ssh -i mykey.priv cirros@$FLOATING_IP_WEST

c, Create VPN elements On East and West
EAST_EXT_IP=$(neutron router-show vpn-router-east |grep 'external_gateway_info' |grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}')
WEST_EXT_IP=$(neutron router-show vpn-router-west |grep 'external_gateway_info' |grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}')
neutron vpn-ikepolicy-create ikepolicy1
neutron vpn-ipsecpolicy-create ipsecpolicy1

neutron vpn-service-create --name vpn-east --description "VPN EAST" $EAST_ROUTER_ID $EAST_SUBNET_ID
neutron ipsec-site-connection-create --name vpn_conn_east --vpnservice-id vpn-east --ikepolicy-id ikepolicy1 --ipsecpolicy-id ipsecpolicy1 --peer-address $WEST_EXT_IP --peer-id $WEST_EXT_IP --peer-cidr 10.0.3.0/24 --psk password

neutron vpn-service-create --name vpn-west --description "VPN WEST" $WEST_ROUTER_ID $WEST_SUBNET_ID
neutron ipsec-site-connection-create --name  vpn_conn_west --vpnservice-id vpn-west --ikepolicy-id ikepolicy1 --ipsecpolicy-id ipsecpolicy1 --peer-address $EAST_EXT_IP --peer-id $EAST_EXT_IP --peer-cidr 10.0.2.0/24 --psk password

neutron ipsec-site-connection-list
neutron vpn-service-list
neutron vpn-service-update --admin_state_up=True <vpnservice-id>

d, Check the status, ok, the status of ipsec-site-connection object should be DOWN before the data arrive in for 'auto=route' passive mode.

hua@hua-ThinkPad-T440p:/bak/openstack/devstack$ nova list
+--------------------------------------+-------------+--------+------------+-------------+---------------------------------------+
| ID                                   | Name        | Status | Task State | Power State | Networks                              |
+--------------------------------------+-------------+--------+------------+-------------+---------------------------------------+
| 9ce00d9d-8145-4859-aa43-539fc8efb10e | vpn-vm-east | ACTIVE | -          | Running     | vpn-net-east=10.0.2.2, 192.168.101.12 |
| f733855a-e768-4f59-9dad-731facdc2e4c | vpn-vm-west | ACTIVE | -          | Running     | vpn-net-west=10.0.3.2, 192.168.101.13 |
+--------------------------------------+-------------+--------+------------+-------------+---------------------------------------+
hua@hua-ThinkPad-T440p:/bak/openstack/devstack$ neutron vpn-service-list
+--------------------------------------+----------+--------------------------------------+--------+
| id                                   | name     | router_id                            | status |
+--------------------------------------+----------+--------------------------------------+--------+
| 1e9c2fa1-deb4-4d02-8907-4979ced048d0 | vpn-west | d993a249-3a4d-4c64-b33a-35363dcd83b7 | ACTIVE |
| e65a78fa-6bda-4aea-bd0f-ccb5010a4ed0 | vpn-east | efb33180-6ad2-44ce-b94a-708943d1150a | ACTIVE |
+--------------------------------------+----------+--------------------------------------+--------+
hua@hua-ThinkPad-T440p:/bak/openstack/devstack$ neutron ipsec-site-connection-list
+--------------------------------------+---------------+---------------+---------------+------------+-----------+--------+
| id                                   | name          | peer_address  | peer_cidrs    | route_mode | auth_mode | status |
+--------------------------------------+---------------+---------------+---------------+------------+-----------+--------+
| 81bbd115-d713-4135-8394-25aeeccc1a8c | vpn_conn_east | 192.168.101.5 | "10.0.3.0/24" | static     | psk       | DOWN   |
| fed4a953-36d8-47a9-b439-d785144e9a11 | vpn_conn_west | 192.168.101.5 | "10.0.2.0/24" | static     | psk       | DOWN   |
+--------------------------------------+---------------+---------------+---------------+------------+-----------+--------+

e, Check the status again after the ping data arrive in, ok, the status of ipsec-site-connection object should be ACTIVE after the data arrive in for 'auto=route' passive mode.

hua@hua-ThinkPad-T440p:~$ ssh cirros@192.168.101.13
$ ip addr show eth0 |grep eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    inet 10.0.3.2/24 brd 10.0.3.255 scope global eth0
$ ping -c 1 10.0.2.2
PING 10.0.2.2 (10.0.2.2): 56 data bytes
64 bytes from 10.0.2.2: seq=0 ttl=62 time=1.038 ms

--- 10.0.2.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 1.038/1.038/1.038 ms

hua@hua-ThinkPad-T440p:/bak/openstack/devstack$ neutron ipsec-site-connection-list
+--------------------------------------+---------------+---------------+---------------+------------+-----------+--------+
| id                                   | name          | peer_address  | peer_cidrs    | route_mode | auth_mode | status |
+--------------------------------------+---------------+---------------+---------------+------------+-----------+--------+
| 801509ee-8cff-4a2e-ae16-08aafb170e5b | vpn_conn_west | 192.168.101.4 | "10.0.2.0/24" | static     | psk       | ACTIVE |
| 81bbd115-d713-4135-8394-25aeeccc1a8c | vpn_conn_east | 192.168.101.5 | "10.0.3.0/24" | static     | psk       | ACTIVE |
+--------------------------------------+---------------+---------------+---------------+------------+-----------+--------+
hua@hua-ThinkPad-T440p:/bak/openstack/devstack$ neutron vpn-service-list
+--------------------------------------+----------+--------------------------------------+--------+
| id                                   | name     | router_id                            | status |
+--------------------------------------+----------+--------------------------------------+--------+
| 1e9c2fa1-deb4-4d02-8907-4979ced048d0 | vpn-west | d993a249-3a4d-4c64-b33a-35363dcd83b7 | ACTIVE |
| e65a78fa-6bda-4aea-bd0f-ccb5010a4ed0 | vpn-east | efb33180-6ad2-44ce-b94a-708943d1150a | ACTIVE |
+--------------------------------------+----------+--------------------------------------+--------+


3, The principle behind
  a, route info added by strongswan process On East and West
     hua@hua-ThinkPad-T440p:~$ sudo ip netns exec qrouter-efb33180-6ad2-44ce-b94a-708943d1150a ip route list table 220
       10.0.3.0/24 via 192.168.101.5 dev qg-1dd8df9a-bf  proto static  src 10.0.2.1
     hua@hua-ThinkPad-T440p:~$ sudo ip netns exec qrouter-d993a249-3a4d-4c64-b33a-35363dcd83b7 ip route list table 220
       10.0.2.0/24 via 192.168.101.4 dev qg-12ccbd3c-6d  proto static  src 10.0.3.1
  b, iptables rulles added by strongswan vpnaas driver On East and West
     hua@hua-ThinkPad-T440p:~$ sudo ip netns exec qrouter-efb33180-6ad2-44ce-b94a-708943d1150a iptables -nL -t nat |grep ipsec
     ACCEPT     all  --  10.0.2.0/24          10.0.3.0/24          policy match dir out pol ipsec
     hua@hua-ThinkPad-T440p:~$ sudo ip netns exec qrouter-d993a249-3a4d-4c64-b33a-35363dcd83b7 iptables -nL -t nat |grep ipsec
     ACCEPT     all  --  10.0.3.0/24          10.0.2.0/24          policy match dir out pol ipsec
  c, debug

hua@hua-ThinkPad-T440p:~$ sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf ip netns exec qrouter-efb33180-6ad2-44ce-b94a-708943d1150a neutron-vpn-netns-wrapper --mount_paths=/etc:/opt/stack/data/neutron/ipsec/efb33180-6ad2-44ce-b94a-708943d1150a/etc,/var/run:/opt/stack/data/neutron/ipsec/efb33180-6ad2-44ce-b94a-708943d1150a/var/run --cmd=ipsec,status
Command: ['mount', '--bind', '/opt/stack/data/neutron/ipsec/efb33180-6ad2-44ce-b94a-708943d1150a/etc', '/etc'] Exit code: 0 Stdout:  Stderr: Command: ['mount', '--bind', '/opt/stack/data/neutron/ipsec/efb33180-6ad2-44ce-b94a-708943d1150a/var/run', '/var/run'] Exit code: 0 Stdout:  Stderr: Command: ['ipsec', 'status'] Exit code: 0 Stdout: Routed Connections:
81bbd115-d713-4135-8394-25aeeccc1a8c{1}:  ROUTED, TUNNEL
81bbd115-d713-4135-8394-25aeeccc1a8c{1}:   10.0.2.0/24 === 10.0.3.0/24
Security Associations (1 up, 0 connecting):
81bbd115-d713-4135-8394-25aeeccc1a8c[2]: ESTABLISHED 11 minutes ago, 192.168.101.4[192.168.101.4]...192.168.101.5[192.168.101.5]
81bbd115-d713-4135-8394-25aeeccc1a8c{1}:  INSTALLED, TUNNEL, ESP SPIs: cbabdd51_i cfb67c36_o
81bbd115-d713-4135-8394-25aeeccc1a8c{1}:   10.0.2.0/24 === 10.0.3.0/24
 Stderr:
hua@hua-ThinkPad-T440p:~$ sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf ip netns exec qrouter-d993a249-3a4d-4c64-b33a-35363dcd83b7 neutron-vpn-netns-wrapper --mount_paths=/etc:/opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/etc,/var/run:/opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/var/run --cmd=ipsec,status
Command: ['mount', '--bind', '/opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/etc', '/etc'] Exit code: 0 Stdout:  Stderr: Command: ['mount', '--bind', '/opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/var/run', '/var/run'] Exit code: 0 Stdout:  Stderr: Command: ['ipsec', 'status'] Exit code: 0 Stdout: Routed Connections:
801509ee-8cff-4a2e-ae16-08aafb170e5b{1}:  ROUTED, TUNNEL
801509ee-8cff-4a2e-ae16-08aafb170e5b{1}:   10.0.3.0/24 === 10.0.2.0/24
Security Associations (1 up, 0 connecting):
801509ee-8cff-4a2e-ae16-08aafb170e5b[1]: ESTABLISHED 11 minutes ago, 192.168.101.5[192.168.101.5]...192.168.101.4[192.168.101.4]
801509ee-8cff-4a2e-ae16-08aafb170e5b{1}:  INSTALLED, TUNNEL, ESP SPIs: cfb67c36_i cbabdd51_o
801509ee-8cff-4a2e-ae16-08aafb170e5b{1}:   10.0.3.0/24 === 10.0.2.0/24
 Stderr:

4, configurations generated by strongswan vpnaas driver

hua@hua-ThinkPad-T440p:~$ cat /opt/stack/data/neutron/ipsec/efb33180-6ad2-44ce-b94a-708943d1150a/etc/ipsec.conf
# Configuration for vpn-east
config setup

conn %default
        ikelifetime=60m
        keylife=20m
        rekeymargin=3m
        keyingtries=1
        authby=psk
        keyexchange=ikev2
        mobike=no

conn 81bbd115-d713-4135-8394-25aeeccc1a8c
    left=192.168.101.4
    leftsubnet=10.0.2.0/24
    leftid=192.168.101.4
    leftfirewall=yes
    right=192.168.101.5
    rightsubnet=10.0.3.0/24
    rightid=192.168.101.5
    auto=route

hua@hua-ThinkPad-T440p:~$ cat /opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/etc/ipsec.conf
# Configuration for vpn-west
config setup

conn %default
        ikelifetime=60m
        keylife=20m
        rekeymargin=3m
        keyingtries=1
        authby=psk
        keyexchange=ikev2
        mobike=no

conn 801509ee-8cff-4a2e-ae16-08aafb170e5b
    left=192.168.101.5
    leftsubnet=10.0.3.0/24
    leftid=192.168.101.5
    leftfirewall=yes
    right=192.168.101.4
    rightsubnet=10.0.2.0/24
    rightid=192.168.101.4
    auto=route

hua@hua-ThinkPad-T440p:~$ cat /opt/stack/data/neutron/ipsec/efb33180-6ad2-44ce-b94a-708943d1150a/etc/ipsec.secrets
# Configuration for vpn-east
192.168.101.4 192.168.101.5 : PSK "password"

hua@hua-ThinkPad-T440p:~$ cat /opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/etc/ipsec.secrets
# Configuration for vpn-west
192.168.101.5 192.168.101.4 : PSK "password"

hua@hua-ThinkPad-T440p:~$ cat /opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/etc/strstrongswan.conf
charon {
        load_modular = yes
        plugins {
                include strongswan.d/charon/*.conf
        }
}

hua@hua-ThinkPad-T440p:~$ cat /opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/etc/strongswan.conf
charon {
        load_modular = yes
        plugins {
                include strongswan.d/charon/*.conf
        }
}

include strongswan.d/*.conf