OpenStack Folsom 双网卡快速安装指南

来源：互联网发布：listview.java 1594 编辑：程序博客网时间：2024/06/14 16:08

https://github.com/888888/OpenStack-Folsom-Install-guide/blob/GRE/2NICs/OpenStack_Folsom_Install_Guide_WebVersion.rst

http://wenku.baidu.com/view/ee82392e4b73f242326c5f00.html

产品研发与市场推广流程

OpenStack Folsom 双网卡快速安装指南

关键字: 多节点安装，双网卡，Multi node OpenStack, Folsom, Quantum, Nova, Keystone, Glance, Horizon, Cinder, OpenVSwitch, KVM, Ubuntu Server 12.10 (64 bits).

作者

梁小白 <11315889@qq.com>

0. 前言

Openstack Folsom 发布好久了，但由于新的组件Quantum的加入，以及知识的跨度，比如同时需要系统管理及网络工程方面的知识，所以Folsom的安装还是挺费事的。经过几天的测试，参考各种文档，终于完成了Folsom基于双网卡的安装，总结至此。

1. 需求说明

节点名称:NICs控制节点:eth0 (100.10.10.51), eth1 (192.168.100.51)网络节点:eth0 (100.10.10.52), eth2 (0.0.0.0)计算节点:eth0 (100.10.10.53)

**备注 1: ** 本文为双网卡安装Folsom设计,根据官方说明，网络节点最好采用三块网卡控制节点可以和计算节点合二为一.

备注 2： 本文安装指南环境为实现Folsom功能评估，力求简单方便，安全性差，不可用于生产环境。

**备注 3: ** 本文不适用于虚拟机环境.请使用物理计算机安装.

2. 控制节点

2.1. 准备系统

安装系统注意事项:

- ubuntu-12.10-server-amd64.iso- 为Cinder服务预留独立分区 例如: /dev/sda5- 提前定义好各服务器主机名及ＩＰ，尽量别改，一定要改，请修改/etc/hosts中的对应关系

以下所有命令均在root权限下完成，所以在装好ubuntu后，请切换到root:
```
sudo passwdsu
```
更新系统(依据笔者经验，安装完Folsom环境后最好别再使用dist-upgrade,以免产生些许小问题，如虚拟机获得不了ip等..):
```
apt-get updateapt-get upgradeapt-get dist-upgrade
```
因为要更新和下载的软件比较多，可以在空闲时间一次更新系统并提前安装所需要软件，以后只需要配置就行了:
apt-get update && apt-get dist-upgrade -y && apt-get update -y && apt-get dist-upgrade -y && apt-get install -y rabbitmq-server ntp vlan bridge-utils keystone curl openssl glance quantum-server quantum-plugin-openvswitch nova-api nova-cert novnc nova-consoleauth nova-scheduler nova-novncproxy cinder-api cinder-scheduler cinder-volume iscsitarget open-iscsi iscsitarget-dkms openstack-dashboard memcached python-mysqldb mysql-server

2.2.配置网卡

主控应该有一个外网网卡:

#访问Openstack ＡＰＩauto eth1iface eth1 inet staticaddress 192.168.100.51netmask 255.255.255.0gateway 192.168.100.1dns-nameservers 8.8.8.8#管理网络和虚拟机网络合二为一auto eth0iface eth0 inet staticaddress 100.10.10.51netmask 255.255.255.0

重启网络服务:
```
service networking restart
```

2.3. MySQL & RabbitMQ

安装 MySQL 和 RabbitMQ:

apt-get install mysql-server python-mysqldb rabbitmq-server

配置Mysql监听所有地址:

sed -i 's/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnfservice mysql restart

为了简化安装，以后所有连接mysql服务均使用 root:password登录,将root权限更改为所有主机可以访问(默认只能本机访问)
mysql -uroot -ppassword use mysql; update user set host='%' where user='root' and host='localhost'; flush privileges;

创建所有必须的数据库:

create database keystone;create database nova;create database glance;create database cinder;create database quantum;

2.4. 节点时间同步

安装时间服务器,其它节点时间同此服务器同步:

apt-get install ntpsed -i 's/server ntp.ubuntu.com/server ntp.ubuntu.com\nserver 127.127.1.0\nfudge 127.127.1.0 stratum 10/g' /etc/ntp.confservice ntp restart

2.5. Others

安装其它服务:
```
apt-get install vlan bridge-utils
```

允许IP转发:

vi /etc/sysctl.confnet.ipv4.conf.all.rp_filter = 0net.ipv4.conf.default.rp_filter = 0sysctl net.ipv4.ip_forward=1# 检查一下sysctl -p

2.6. Keystone

安装组件:
```
apt-get install keystone
```

编辑 /etc/keystone/keystone.conf 数据库连接:

connection = mysql://root:password@100.10.10.51/keystone

重启keystone并初始化数据库:

service keystone restartkeystone-manage db_sync

使用自动化脚本创建keystone用户、服务、服务端点。为了简化，这里只创建admin一个用户，请不要修改此用户密码。
bash keystone_basic.sh 执行一次，否则会创建多个service bash keystone_endpoints_basic.sh

创建/root/novarc文件，写入以下内容:

export OS_TENANT_NAME=adminexport OS_TENANT_ID=c7fb80d964a24ab1bc0fd370696c804eexport OS_USERNAME=adminexport OS_PASSWORD=passwordexport OS_AUTH_URL="http://127.0.0.1:35357/v2.0"export OS_REGION_NAME=RegionOneexport OS_IDENTITY_API_VERSION=2.0export SERVICE_TOKEN=ADMINexport SERVICE_ENDPOINT="http://127.0.0.1:35357/v2.0"各项值请根据实际情况替换keystone tenant-list将获取的tenant_id替换到novarcsource /root/novarcecho "source /root/novarc" >> ~/.bashrc

2.7. Glance

安装组件
apt-get install glance

分别修改 /etc/glance/glance-api-paste.ini 和/etc/glance/glance-registry-paste.ini:

[filter:authtoken]paste.filter_factory = keystone.middleware.auth_token:filter_factoryauth_host = 100.10.10.51auth_port = 35357auth_protocol = httpadmin_tenant_name = adminadmin_user = adminadmin_password = password

分别修改 /etc/glance/glance-api.conf 和/etc/glance/glance-registry.conf

sql_connection = mysql://root:password@100.10.10.51/glance[paste_deploy]flavor = keystone

重启glance服务并同步glance数据库:

service glance-api restart; service glance-registry restartglance-manage db_sync

测试glance 服务，不输出任何结果代表成功:
```
glance index
```

上传个镜像:

wget https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.imgglance image-create --name myFirstImage --is-public true --container-format bare \ --disk-format qcow2 < cirros-0.3.0-x86_64-disk.img

再查看一下:
```
glance image-list
```

2.8. Quantum

安装组件:

apt-get install quantum-server quantum-plugin-openvswitch

修改 /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini,移动后文件最后，有模板:

[DATABASE]sql_connection = mysql://root:password@100.10.10.51/quantum#Under the OVS section[OVS]tenant_network_type = gretunnel_id_ranges = 1:1000enable_tunneling = True

修改 /etc/quantum/api-paste.ini

[filter:authtoken]paste.filter_factory = keystone.middleware.auth_token:filter_factoryauth_host = 100.10.10.51auth_port = 35357auth_protocol = httpadmin_tenant_name = adminadmin_user = adminadmin_password = password

重启 quantum server:
```
service quantum-server restart
```

2.9. Nova

安装组件:

apt-get install -y nova-api nova-cert novnc nova-consoleauth nova-scheduler nova-novncproxy

修改 /etc/nova/api-paste.ini

[filter:authtoken]paste.filter_factory = keystone.middleware.auth_token:filter_factoryauth_host = 100.10.10.51auth_port = 35357auth_protocol = httpadmin_tenant_name = adminadmin_user = adminadmin_password = passwordsigning_dirname = /tmp/keystone-signing-nova

Modify the /etc/nova/nova.conf like this:

[DEFAULT]logdir=/var/log/novastate_path=/var/lib/novalock_path=/run/lock/novaverbose=Trueapi_paste_config=/etc/nova/api-paste.inischeduler_driver=nova.scheduler.simple.SimpleSchedulers3_host=100.10.10.51ec2_host=100.10.10.51ec2_dmz_host=100.10.10.51rabbit_host=100.10.10.51cc_host=100.10.10.51dmz_cidr=169.254.169.254/32metadata_host=100.10.10.51metadata_listen=0.0.0.0nova_url=http://100.10.10.51:8774/v1.1/sql_connection=mysql://novaUser:novaPass@100.10.10.51/novaec2_url=http://100.10.10.51:8773/services/Cloudroot_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf# Authuse_deprecated_auth=falseauth_strategy=keystonekeystone_ec2_url=http://100.10.10.51:5000/v2.0/ec2tokens# Imaging serviceglance_api_servers=100.10.10.51:9292image_service=nova.image.glance.GlanceImageService# Vnc configurationnovnc_enabled=truenovncproxy_base_url=http://192.168.100.51:6080/vnc_auto.htmlnovncproxy_port=6080vncserver_proxyclient_address=192.168.100.51vncserver_listen=0.0.0.0# Network settingsnetwork_api_class=nova.network.quantumv2.api.APIquantum_url=http://100.10.10.51:9696quantum_auth_strategy=keystonequantum_admin_tenant_name=adminquantum_admin_username=adminquantum_admin_password=passwordquantum_admin_auth_url=http://100.10.10.51:35357/v2.0libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriverlinuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriverfirewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver# Compute #compute_driver=libvirt.LibvirtDriver# Cinder #volume_api_class=nova.volume.cinder.APIosapi_volume_listen_port=5900

初始化nova数据库:
```
nova-manage db sync
```

重启所有nova服务:

cd /etc/init.d/; for i in $( ls nova-* ); do sudo service $i restart; done

检查nova服务，有笑脸图标，证明服务正常:

nova-manage service list Binary           Host             Zone             Status     State Updated_At nova-cert        sm1u07           nova             enabled    :-)   2013-03-15 12:08:31 nova-consoleauth sm1u07           nova             enabled    :-)   2013-03-15 12:08:30 nova-scheduler   sm1u07           nova             enabled    :-)   2013-03-15 12:08:30

2.10. Cinder

安装组件:

apt-get install cinder-api cinder-scheduler cinder-volume iscsitarget open-iscsi iscsitarget-dkms

打开iscsi服务:

sed -i 's/false/true/g' /etc/default/iscsitargetservice iscsitarget startservice open-iscsi start

修改 /etc/cinder/api-paste.ini 认证信息:

[filter:authtoken]paste.filter_factory = keystone.middleware.auth_token:filter_factoryservice_protocol = httpservice_host = 100.10.10.51service_port = 5000auth_host = 100.10.10.51auth_port = 35357auth_protocol = httpadmin_tenant_name = adminadmin_user = passwordadmin_password = password

修改the /etc/cinder/cinder.conf to:

[DEFAULT]rootwrap_config=/etc/cinder/rootwrap.confsql_connection = mysql://root:password@100.10.10.51/cinderapi_paste_confg = /etc/cinder/api-paste.iniiscsi_helper=ietadmvolume_name_template = volume-%svolume_group = cinder-volumesverbose = Trueauth_strategy = keystone

初始化cinder数据库:
```
cinder-manage db sync
```

创建cinder使用的物理卷及卷组:

pvcreate /dev/sda5vgcreate cinder-volumes /dev/sda5service cinder-volume restartservice cinder-api restart

2.11. 控制面板

安装组件

apt-get install openstack-dashboard memcached

dashboard依赖apache和memcache:

service apache2 restart; service memcached restart

现在可以访问Dashboard了 http://192.168.100.51/horizon 用户名密码 admin:password.

3. 网络节点

3.1. 准备系统

安装ubuntu12.01:

apt-get updateapt-get upgradeapt-get dist-upgrade 快速： apt-get update && apt-get dist-upgrade -y && apt-get install -y ntp vlan bridge-utils openvswitch-switch openvswitch-datapath-dkms quantum-plugin-openvswitch-agent quantum-dhcp-agent quantum-l3-agent

安装配置基本服务ntp,vlan,bridge-utils:

apt-get install ntp vlan bridge-utilssed -i 's/server ntp.ubuntu.com/server 100.10.10.51/g' /etc/ntp.confservice ntp restart

允许ip转发:

vi /etc/sysctl.confnet.ipv4.conf.all.rp_filter = 0net.ipv4.conf.default.rp_filter = 0sysctl net.ipv4.ip_forward=1

3.2.配置网卡

网络节点eth1网卡将做为虚拟机与互联网通讯端口，设置网卡为 promisc mode:

#虚拟机外网出口auto eth1iface eth1 inet manualup ifconfig $IFACE 0.0.0.0 upup ip link set $IFACE promisc ondown ip link set $IFACE promisc offdown ifconfig $IFACE down#管理网络及内部通信auto eth0iface eth0 inet staticaddress 100.10.10.52netmask 255.255.255.0

3.3. OpenVSwitch

安装虚拟交换机:

apt-get install -y openvswitch-switch openvswitch-datapath-dkms

创建网桥:

#br-int is used for VM integrationovs-vsctl add-br br-int#br-ex is used for accessing internet.ovs-vsctl add-br br-exovs-vsctl br-set-external-id br-ex bridge-id br-exovs-vsctl add-port br-ex eth1启动br-exip link set br-ex up

3.4. Quantum

安装quantum组件:

apt-get -y install quantum-dhcp-agent quantum-l3-agent quantum-plugin-openvswitch-agent

编辑/etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini

#Under the database section[DATABASE]sql_connection = mysql://root:password@100.10.10.51/quantum#Under the OVS section[OVS]tenant_network_type = gretunnel_id_ranges = 1:1000integration_bridge = br-inttunnel_bridge = br-tunlocal_ip = 100.10.10.52enable_tunneling = True

更新 /etc/quantum/l3_agent.ini:

auth_url = http://100.10.10.51:35357/v2.0auth_region = RegionOneadmin_tenant_name = adminadmin_user = adminadmin_password = passwordmetadata_ip = 100.10.10.51metadata_port = 8775use_namespaces = False

修改 /etc/quantum/dhcp_agent.ini:
```
use_namespaces = False
```
修改/etc/quantum/quantum.conf
```
rabbit_host = 100.10.10.51
```

重启所有服务:

service quantum-dhcp-agent restartservice quantum-l3-agent restartservice quantum-plugin-openvswitch-agent restart

4. 计算节点

4.1. 准备系统

更新升级:

apt-get updateapt-get upgradeapt-get dist-upgrade快速： apt-get update && apt-get dist-upgrade -y && apt-get install -y ntp vlan bridge-utils cpu-checker kvm libvirt-bin pm-utils openvswitch-switch openvswitch-datapath-dkms  quantum-plugin-openvswitch-agent nova-compute-kvm

安装 ntp vlan bridge-utils:

apt-get install ntp vlan bridge-utilssed -i 's/server ntp.ubuntu.com/server 100.10.10.51/g' /etc/ntp.confservice ntp restart

允许ＩＰ转发:

vi /etc/sysctl.confnet.ipv4.conf.all.rp_filter = 0net.ipv4.conf.default.rp_filter = 0sysctl net.ipv4.ip_forward=1

4.2.配置网卡

vi /etc/network/interfaces

# 管理网络和内部通讯网络auto eth0iface eth0 inet staticaddress 100.10.10.53netmask 255.255.255.0

4.3 KVM

确认硬件支持虚拟化:
```
apt-get install cpu-checkerkvm-ok
```

安装kvm组件:

apt-get install -y kvm libvirt-bin pm-utils

编辑libvirt设备列表支持tun /etc/libvirt/qemu.conf:

cgroup_device_acl = ["/dev/null", "/dev/full", "/dev/zero","/dev/random", "/dev/urandom","/dev/ptmx", "/dev/kvm", "/dev/kqemu","/dev/rtc", "/dev/hpet","/dev/net/tun"]

删除kvm默认网络配置

virsh net-destroy defaultvirsh net-undefine default

允许动态迁移

vi /etc/libvirt/libvirtd.conflisten_tls = 0listen_tcp = 1auth_tcp = "none"vi /etc/init/libvirt-bin.confenv libvirtd_opts="-d -l"vi /etc/default/libvirt-binlibvirtd_opts="-d -l"service libvirt-bin restart

4.4. OpenVSwitch

安装 openVSwitch:

apt-get install -y openvswitch-switch openvswitch-datapath-dkms

创建网桥 bridges:

#br-int will be used for VM integrationovs-vsctl add-br br-int

4.5. Quantum

安装 Quantum openvswitch agent:

apt-get -y install quantum-plugin-openvswitch-agent

编辑OVS配置 /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini:

#Under the database section[DATABASE]sql_connection = mysql://root:password@100.10.10.51/quantum#Under the OVS section[OVS]tenant_network_type = gretunnel_id_ranges = 1:1000integration_bridge = br-inttunnel_bridge = br-tunlocal_ip = 100.10.10.53enable_tunneling = True

修改rabbitMQ IP

vi /etc/quantum/quantum.confrabbit_host = 100.10.10.51

重启所有服务:

service quantum-plugin-openvswitch-agent restart

4.6. Nova

安装nova compute组件:
```
apt-get install nova-compute-kvm
```

修改 /etc/nova/api-paste.ini:

[filter:authtoken]paste.filter_factory = keystone.middleware.auth_token:filter_factoryauth_host = 100.10.10.51auth_port = 35357auth_protocol = httpadmin_tenant_name = adminadmin_user = adminadmin_password = passwordsigning_dirname = /tmp/keystone-signing-nova

编辑 /etc/nova/nova-compute.conf:

[DEFAULT]libvirt_type=kvmlibvirt_ovs_bridge=br-intlibvirt_vif_type=ethernetlibvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriverlibvirt_use_virtio_for_bridges=True

修改 /etc/nova/nova.conf

[DEFAULT]logdir=/var/log/novastate_path=/var/lib/novalock_path=/run/lock/novaverbose=Trueapi_paste_config=/etc/nova/api-paste.inischeduler_driver=nova.scheduler.simple.SimpleSchedulers3_host=100.10.10.51ec2_host=100.10.10.51ec2_dmz_host=100.10.10.51rabbit_host=100.10.10.51cc_host=100.10.10.51dmz_cidr=169.254.169.254/32metadata_host=100.10.10.51metadata_listen=0.0.0.0nova_url=http://100.10.10.51:8774/v1.1/sql_connection=mysql://novaUser:novaPass@100.10.10.51/novaec2_url=http://100.10.10.51:8773/services/Cloudroot_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf# Authuse_deprecated_auth=falseauth_strategy=keystonekeystone_ec2_url=http://100.10.10.51:5000/v2.0/ec2tokens# Imaging serviceglance_api_servers=100.10.10.51:9292image_service=nova.image.glance.GlanceImageService# Vnc configurationnovnc_enabled=truenovncproxy_base_url=http://192.168.100.51:6080/vnc_auto.htmlnovncproxy_port=6080vncserver_proxyclient_address=100.10.10.53vncserver_listen=0.0.0.0# Network settingsnetwork_api_class=nova.network.quantumv2.api.APIquantum_url=http://100.10.10.51:9696quantum_auth_strategy=keystonequantum_admin_tenant_name=adminquantum_admin_username=adminquantum_admin_password=passwordquantum_admin_auth_url=http://100.10.10.51:35357/v2.0libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriverlinuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriverfirewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver# Compute #compute_driver=libvirt.LibvirtDriver# Cinder #volume_api_class=nova.volume.cinder.APIosapi_volume_listen_port=5900

Restart nova-* services:

cd /etc/init.d/; for i in $( ls nova-* ); do sudo service $i restart; done

Check for the smiling faces on nova-* services to confirm your installation:
```
nova-manage service list
```

5. 创建虚拟机

使用 <http://192.168.10.51/horizon> 管理虚拟机
编辑安全组，允许所有协议,tcp,udp,icmp
root@sm1u07:~# nova secgroup-list-rules default Please enter password for encrypted keyring:

IP ProtocolFrom PortTo PortIP RangeSource Groupicmp tcp udp-1 1 1255 65535 655350.0.0.0/0 0.0.0.0/0 0.0.0.0/0

使用脚本 quantum.sh 为admin创建相关的网络，即虚拟机内网和外网
查看创建好的网络
root@hp4u:~# quantum net-list

idnamesubnets14dbb282-c74a-4784-bfc3-351f7ca3d034 d402e168-cbda-4345-8ffa-015e6a1c4aa1ext_net admin-net95bddb90-84dc-4579-99b8-798a393a3edf 8ef3c4dd-a265-421c-afa2-6cff28ae2c74

root@hp4u:~# quantum router-list

idnameexternal_gateway_info623b68f4-967a-4028-9a92-dc5a7d3e16e8provider-router{"network_id": "14dbb282-c74a-4784-bfc3-351f7ca3d034"}

修改 /etc/quantum/l3_agent.ini :
gateway_external_network_id = 14dbb282-c74a-4784-bfc3-351f7ca3d034 router_id = 623b68f4-967a-4028-9a92-dc5a7d3e16e8
service quantum-l3-agent restart
使用控制面板创建一个虚拟机，并记录vm-uuid，勇冠vm-uuid获取vm的端口id
quantum port-list -- --device_id <vm-uuid>
目前horizon不支持quantum的floatingip操作,通过quantum 命令行为vm 分配floatingip,
quantum floatingip-create --port_id <port_id> <ext_net_id>
大功告成，现在你可以去dashboard中用vnc登录vm，测试一下各个网络是否通畅

OpenStack Folsom 双网卡快速安装指南

产品研发与市场推广流程

OpenStack Folsom 双网卡快速安装指南

作者

目录

0. 前言

1. 需求说明

2. 控制节点

2.1. 准备系统

2.2.配置网卡

2.3. MySQL & RabbitMQ

2.4. 节点时间同步

2.5. Others

2.6. Keystone

2.7. Glance

2.8. Quantum

2.9. Nova

2.10. Cinder

2.11. 控制面板

3. 网络节点

3.1. 准备系统

3.2.配置网卡

3.3. OpenVSwitch

3.4. Quantum

4. 计算节点

4.1. 准备系统

4.2.配置网卡

4.3 KVM

4.4. OpenVSwitch

4.5. Quantum

4.6. Nova

5. 创建虚拟机