Openstack安装Openstack（Juno版本）

国内现在做Openstack的培训也开始多起来，不过应该都是采用虚拟机的方式来培训，我个人感觉还是很土的，不能吃自己的狗食。建议日后大家参加Openstack的培训的时候，都问一句，是不是在Openstack下培训Openstack。

上次写过一篇Icehouse版本的。这次Juno版本，基本是一样的过程。不过调整一下顺序。用用户更容易操作。

这次我是使用刻通云平台进行，希望可以做到更加流畅。

（1）基本情况

默认是有一个基础网络。我们还是需要创建一个自己的网络，来满足openstack需求。

整理一下

角色管理网络虚拟机通讯网络外部网络控制节点eth0（10.0.0.11）eth1（192.168.100.11）   网络节点eth0（10.0.0.21）eth1（10.0.1.21）eth2（192.168.100.21）计算节点eth0（10.0.0.31）eth1（10.0.1.21）

文档很清楚，

1. 网络节点，需要3块网卡。
2. 控制节点和网络节点，需要外部网络，就是需要所谓的公网的IP
3. 计算节点是不需要公网IP
4. 所有的虚拟机访问公网，都是需要经过网络节点。
5. 192.168.100.0,就相当于公网的IP地址段

根据上图，我们组建我们自己的网络

1. 创建router
2. 创建管理网络，公网，虚拟机网络（记得按顺序创建）
3. 公网连接router
4. 申请公网IP
5. 把IP绑定在router

（2）控制节点

1、网络搭建

创建一个ubuntu14.04的虚拟机，1core，2G内存，应该就够用了。网络，记得设置固定IP地址

为了登陆vnc，所以选择密码登陆

最后一步

虚拟机就创建完毕。

我们需要给控制节点添加一块网卡，连接到公网

看看拓扑图

vnc登陆

由于虚拟机获得两块网卡，默认网关就需要手工指定。

route add default gw 192.168.100.1

这个时候,你就可以访问外网.

从远程访问虚拟机，可以通过端口映射，或者vpn，这里直接在router上设置端口映射。这样你直接sshrouter的IP地址，就可以访问。

2、设置源

apt-get install ubuntu-cloud-keyringecho "deb http://ubuntu-cloud.archive.canonical.com/ubuntu" \  "trusty-updates/juno main" > /etc/apt/sources.list.d/cloudarchive-juno.list

更新

apt-get update && apt-get dist-upgrade

3、NTP服务器

apt-get install -y ntp

4、数据库

apt-get install mariadb-server python-mysqldb

修改/etc/mysql/my.cnf

bind-address = 10.0.0.11[mysqld]default-storage-engine = innodbinnodb_file_per_tablecollation-server = utf8_general_ciinit-connect = 'SET NAMES utf8'character-set-server = utf8

重启数据库

service mysql restart

5、消息队列RabbitMQ

apt-get install -y rabbitmq-server

6、keystone

安装

apt-get install -y keystone

设置

创建keystone数据库，都是通过 mysql –u root –p 进入

CREATE DATABASE keystone;GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' IDENTIFIED BY 'KEYSTONE_DBPASS';GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%' IDENTIFIED BY 'KEYSTONE_DBPASS';exit;

删除sqllite数据库

rm /var/lib/keystone/keystone.db

编辑 /etc/keystone/keystone.conf

connection = mysql://keystone:KEYSTONE_DBPASS@10.0.0.11/keystone[DEFAULT]admin_token=ADMINlog_dir=/var/log/keystone

初始化keystone数据库

service keystone restartkeystone-manage db_sync

设置环境变量

export OS_SERVICE_TOKEN=ADMINexport OS_SERVICE_ENDPOINT=http://10.0.0.11:35357/v2.0

创建管理员权力的用户

keystone user-create --name=admin --pass=admin_pass --email=[e-mail] role-create --name=adminkeystone role-create --name=_member_keystone tenant-create --name=admin --description="Admin Tenant"keystone user-role-add --user=admin --tenant=admin --role=adminkeystone user-role-add --user=admin --role=_member_ --tenant=admin

创建普通用户

keystone user-create --name=demo --pass=demo_pass --email=[e-mail] tenant-create --name=demo --description="Demo Tenant"keystone user-role-add --user=demo --role=_member_ --tenant=demo

创建 service 租户

keystone tenant-create --name=service --description="Service Tenant"

定义服务的API的endpoint

keystone service-create --name=keystone --type=identity --description="OpenStack Identity"

创建endpoint

keystone endpoint-create \--service-id=$(keystone service-list | awk '/ identity / {print $2}') \--publicurl=http://192.168.100.11:5000/v2.0 \--internalurl=http://10.0.0.11:5000/v2.0 \--adminurl=http://10.0.0.11:35357/v2.0

检测keystone

通过下面命令检查keystone的初始化是否正常

设置环境变量，创建creds 和 admin_creds 两个文件

cat <<EOF >>/root/credsexport OS_TENANT_NAME=adminexport OS_USERNAME=adminexport OS_PASSWORD=admin_passexport OS_AUTH_URL="http://192.168.100.11:5000/v2.0/"EOFcat <<EOF >>/root/admin_credsexport OS_USERNAME=adminexport OS_PASSWORD=admin_passexport OS_TENANT_NAME=adminexport OS_AUTH_URL=http://10.0.0.11:35357/v2.0EOF

检测

先退出ssh，让以前设置的环境变量失效。再登陆。

设置环境变量才能进行下面操作

source creds

这样就可以

root@controller:~# keystone user-list+----------------------------------+-------+---------+------------------+|                id                |  name | enabled |      email       |+----------------------------------+-------+---------+------------------+| 6f8bcafd62ec4e23ab2be28016829f91 | admin |   True  | [e-mail] || 66713a75b7c14f73a1c5a015241f5826 |  demo |   True  | [e-mail]  |+----------------------------------+-------+---------+------------------+root@controller:~# keystone role-list+----------------------------------+----------+|                id                |   name   |+----------------------------------+----------+| 9fe2ff9ee4384b1894a90878d3e92bab | _member_ || cd8dec7752d24a028f95657556f7573d |  admin   |+----------------------------------+----------+root@controller:~# keystone tenant-list+----------------------------------+---------+---------+|                id                |   name  | enabled |+----------------------------------+---------+---------+| efc81990ab4c433f94573e2e0fcf08c3 |  admin  |   True  || be10dc11d4034b389bef8bbcec657f6f |   demo  |   True  || cb45c886bc094f65940ba29d79eab8aa | service |   True  |+----------------------------------+---------+---------+

查看日志

日志在/var/log/keystone/ 下，先清空日志,看看日志是否还有错误信息.

echo "" > /var/log/keystone/keystone-all.logecho "" > /var/log/keystone/keystone-manage.logtail  /var/log/keystone/*

7、Glance

Openstack组件安装，都比较类似。

apt-get install -y glance python-glanceclient

创建数据库 mysql –uroot –p

CREATE DATABASE glance;GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'localhost' IDENTIFIED BY 'GLANCE_DBPASS';GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'%' IDENTIFIED BY 'GLANCE_DBPASS';exit;

keystone创建glance用户和服务

keystone user-create --name=glance --pass=service_pass --email=[e-mail] user-role-add --user=glance --tenant=service --role=admin

设置endpoint

keystone service-create --name=glance --type=image --description="OpenStack Image Service"keystone endpoint-create \--service-id=$(keystone service-list | awk '/ image / {print $2}') \--publicurl=http://192.168.100.11:9292 \--internalurl=http://10.0.0.11:9292 \--adminurl=http://10.0.0.11:9292

编辑 /etc/glance/glance-api.conf

[database]connection = mysql://glance:GLANCE_DBPASS@10.0.0.11/glance[DEFAULT]rpc_backend = rabbitrabbit_host = 10.0.0.11[keystone_authtoken]auth_uri = http://10.0.0.11:5000identity_uri = http://10.0.0.11:35357admin_tenant_name = serviceadmin_user = glanceadmin_password = service_pass[paste_deploy]flavor = keystone

编辑 /etc/glance/glance-registry.conf

[database]# The file name to use with SQLite (string value)#sqlite_db = /var/lib/glance/glance.sqliteconnection = mysql://glance:GLANCE_DBPASS@10.0.0.11/glance[keystone_authtoken]auth_uri = http://10.0.0.11:5000auth_host = 10.0.0.11auth_port = 35357auth_protocol = httpadmin_tenant_name = serviceadmin_user = glanceadmin_password = service_pass[paste_deploy]flavor = keystone

重启服务

service glance-api restart; service glance-registry restart

初始化glance数据库

glance-manage db_sync

上传镜像

source credsglance image-create --name "cirros-0.3.2-x86_64" --is-public true \--container-format bare --disk-format qcow2 \--location http://cdn.download.cirros-cloud.net/0.3.2/cirros-0.3.2-x86_64-disk.img

查看镜像

# glance image-list+--------------------------------------+---------------------+-------------+------------------+----------+--------+| ID                                   | Name                | Disk Format | Container Format | Size     | Status |+--------------------------------------+---------------------+-------------+------------------+----------+--------+| d7a6d71d-4222-44f4-82d0-49c14ba19676 | cirros-0.3.2-x86_64 | qcow2       | bare             | 13167616 | active |+--------------------------------------+---------------------+-------------+------------------+----------+--------+

检测log

root@controller:~# tail /var/log/glance/*==> /var/log/glance/api.log <==-09-02 07:07:12.315 2946 WARNING glance.store.base [-] Failed to configure store correctly: Store sheepdog could not be configured correctly. Reason: Error in store configuration: [Errno 2] No such file or directory Disabling add method.-09-02 07:07:12.316 2946 WARNING glance.store [-] Deprecated: glance.store.sheepdog.Store not found in `known_store`. Stores need to be explicitly enabled in the configuration file.

你会发现log里有类似的所谓错误，这个不是问题。希望glance改进一下这个地方的log。不然让很多新手很郁闷。

8、Nova

安装软件

apt-get install -y nova-api nova-cert nova-conductor nova-consoleauth \nova-novncproxy nova-scheduler python-novaclient

创建nova 数据库 mysql –u root –p

CREATE DATABASE nova;GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' IDENTIFIED BY 'NOVA_DBPASS';GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%' IDENTIFIED BY 'NOVA_DBPASS';exit;

配置keystone

keystone user-create --name=nova --pass=service_pass --email=[e-mail] user-role-add --user=nova --tenant=service --role=admin

设置endpoint

keystone service-create --name=nova --type=compute --description="OpenStack Compute"keystone endpoint-create \--service-id=$(keystone service-list | awk '/ compute / {print $2}') \--publicurl=http://192.168.100.11:8774/v2/%\(tenant_id\)s \--internalurl=http://10.0.0.11:8774/v2/%\(tenant_id\)s \--adminurl=http://10.0.0.11:8774/v2/%\(tenant_id\)s

编辑 /etc/nova/nova.conf

下面是我的nova.conf 文件的全部内容

[DEFAULT]dhcpbridge_flagfile=/etc/nova/nova.confdhcpbridge=/usr/bin/nova-dhcpbridgelogdir=/var/log/novastate_path=/var/lib/novalock_path=/var/lock/novaforce_dhcp_release=Trueiscsi_helper=tgtadmlibvirt_use_virtio_for_bridges=Trueconnection_type=libvirtroot_helper=sudo nova-rootwrap /etc/nova/rootwrap.confverbose=Trueec2_private_dns_show_ip=Trueapi_paste_config=/etc/nova/api-paste.inivolumes_path=/var/lib/nova/volumesenabled_apis=ec2,osapi_compute,metadatarpc_backend = rabbitrabbit_host = 10.0.0.11my_ip = 10.0.0.11vncserver_listen = 10.0.0.11vncserver_proxyclient_address = 10.0.0.11auth_strategy = keystone[keystone_authtoken]auth_uri = http://10.0.0.11:5000auth_host = 10.0.0.11auth_port = 35357auth_protocol = httpadmin_tenant_name = serviceadmin_user = novaadmin_password = service_pass[database]connection = mysql://nova:NOVA_DBPASS@10.0.0.11/nova

删除sqlite数据库

rm /var/lib/nova/nova.sqlite

初始化nova数据库

nova-manage db sync

重启nova相关服务

service nova-api restartservice nova-cert restartservice nova-conductor restartservice nova-consoleauth restartservice nova-novncproxy restartservice nova-scheduler restart

检查

# nova-manage service listBinary           Host                                 Zone             Status     State Updated_Atnova-cert        controller                           internal         enabled       2014-08-26 14:13:08nova-consoleauth controller                           internal         enabled       2014-08-26 14:13:08nova-conductor   controller                           internal         enabled       2014-08-26 14:13:08nova-scheduler   controller                           internal         enabled       2014-08-26 14:13:08

9、Neutron

控制节点，也是需要安装Neutron server

apt-get install -y neutron-server neutron-plugin-ml2

创建Neutron数据库 mysql –u root –p

CREATE DATABASE neutron;GRANT ALL PRIVILEGES ON neutron.* TO neutron@'localhost' IDENTIFIED BY 'NEUTRON_DBPASS';GRANT ALL PRIVILEGES ON neutron.* TO neutron@'%' IDENTIFIED BY 'NEUTRON_DBPASS';exit;

keystone创建neutron用户和角色

keystone user-create --name=neutron --pass=service_pass --email=[e-mail] user-role-add --user=neutron --tenant=service --role=admin

注册服务和endpoint

keystone service-create --name=neutron --type=network --description="OpenStack Networking"keystone endpoint-create \--service-id=$(keystone service-list | awk '/ network / {print $2}') \--publicurl=http://192.168.100.11:9696 \--internalurl=http://10.0.0.11:9696 \--adminurl=http://10.0.0.11:9696

编辑 /etc/neutron/neutron.conf，关键的是nova_admin_tenant_id 需要你手工用命令获得，再填写

keystone tenant-list | awk '/ service / { print $2 }'

#core_plugin = neutron.plugins.ml2.plugin.Ml2Plugincore_plugin = ml2# service_plugins =# Example: service_plugins = router,firewall,lbaas,vpnaas,meteringservice_plugins = router# auth_strategy = keystoneauth_strategy = keystone# allow_overlapping_ips = Falseallow_overlapping_ips = Truerpc_backend = rabbitrabbit_host = 10.0.0.11notification_driver = neutron.openstack.common.notifier.rpc_notifier# ======== neutron nova interactions ==========# Send notification to nova when port status is active.notify_nova_on_port_status_changes = True# Send notifications to nova when port data (fixed_ips/floatingips) change# so nova can update it's cache.notify_nova_on_port_data_changes = True# URL for connection to nova (Only supports one nova region currently).nova_url = http://10.0.0.11:8774/v2# Name of nova region to use. Useful if keystone manages more than one region# nova_region_name =# Username for connection to nova in admin contextnova_admin_username = nova# The uuid of the admin nova tenantnova_admin_tenant_id = cb45c886bc094f65940ba29d79eab8aa# Password for connection to nova in admin context.nova_admin_password = service_pass# Authorization URL for connection to nova in admin context.nova_admin_auth_url = http://10.0.0.11:35357/v2.0[keystone_authtoken]#auth_host = 127.0.0.1#auth_port = 35357#auth_protocol = http#admin_tenant_name = %SERVICE_TENANT_NAME%#admin_user = %SERVICE_USER%#admin_password = %SERVICE_PASSWORD%#signing_dir = $state_path/keystone-signingauth_uri = http://10.0.0.11:5000auth_host = 10.0.0.11auth_port = 35357auth_protocol = httpadmin_tenant_name = serviceadmin_user = neutronadmin_password = service_pass[database]# This line MUST be changed to actually run the plugin.# Example:# connection = mysql://root:pass@127.0.0.1:3306/neutron# Replace 127.0.0.1 above with the IP address of the database used by the# main neutron server. (Leave it as is if the database runs on this host.)#connection = sqlite:////var/lib/neutron/neutron.sqliteconnection = mysql://neutron:NEUTRON_DBPASS@10.0.0.11/neutron

编辑 /etc/neutron/plugins/ml2/ml2_conf.ini

[ml2]type_drivers = gretenant_network_types = gremechanism_drivers = openvswitch[ml2_type_gre]tunnel_id_ranges = 1:1000[securitygroup]firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriverenable_security_group = True

编辑/etc/nova/nova.conf, 让nova支持neutron，在[DEFAULT] 添加

network_api_class=nova.network.neutronv2.api.APIneutron_url=http://10.0.0.11:9696neutron_auth_strategy=keystoneneutron_admin_tenant_name=serviceneutron_admin_username=neutronneutron_admin_password=service_passneutron_admin_auth_url=http://10.0.0.11:35357/v2.0libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriverlinuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriverfirewall_driver=nova.virt.firewall.NoopFirewallDriversecurity_group_api=neutron

重启nova服务

service nova-api restartservice nova-scheduler restartservice nova-conductor restart

这里面有一个bug，需要修复 http://www.tuicool.com/articles/vmaiiua

neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade juno

重启neutron服务

service neutron-server restart

查看log

root@controller:~# tail -f /var/log/neutron/*-09-02 07:27:53.950 5373 WARNING neutron.api.extensions [-] Extension fwaas not supported by any of loaded plugins-09-02 07:27:53.952 5373 WARNING neutron.api.extensions [-] Extension flavor not supported by any of loaded plugins-09-02 07:27:53.962 5373 WARNING neutron.api.extensions [-] Extension lbaas_agent_scheduler not supported by any of loaded plugins-09-02 07:27:53.967 5373 WARNING neutron.api.extensions [-] Extension lbaas not supported by any of loaded plugins-09-02 07:27:53.969 5373 WARNING neutron.api.extensions [-] Extension metering not supported by any of loaded plugins-09-02 07:27:53.973 5373 WARNING neutron.api.extensions [-] Extension port-security not supported by any of loaded plugins-09-02 07:27:53.977 5373 WARNING neutron.api.extensions [-] Extension routed-service-insertion not supported by any of loaded plugins

日志里显示找不到插件，这都是正常的。

10、Horizon

Dashboard的安装，倒是比较简单，不需要创建数据库。

apt-get install -y apache2 memcached libapache2-mod-wsgi openstack-dashboard

编辑 /etc/openstack-dashboard/local_settings.py

#ALLOWED_HOSTS = ['horizon.example.com', ]ALLOWED_HOSTS = ['localhost','192.168.100.11']#OPENSTACK_HOST = "127.0.0.1"OPENSTACK_HOST = "10.0.0.11"

重启apache服务

service apache2 restart; service memcached restart

这个时候，你可以通过 http://192.168.100.11/horizon

看到登录界面，应该是无法登录。

安装Openstack client端

在控制节点装上Openstack的client端，这样会方便很多，很多Neutron的操作，你都可以进行

apt-get -y install python-openstackclient

（3）网络节点

看图理解的更好，这图来自redhat的官方文档。

网络节点需要3块网卡。经常有朋友问，1块网卡是否可以。其实1块网卡肯定也是可以的，不过不利于大家理解。不过大家都很难找到3块网卡的机器，所以在IaaS下来测试，就方便很多。

创建一个虚拟机，名字为：network, 删除网卡，并且添加3块网卡。ssh到虚拟机上，默认是无法访问外网的，原因也很简单，没有默认路由，手工添加默认路由就可以。

由于网络节点,比较特殊,我们需要把网卡的Ip设置成固定 /etc/netwrok/interface

# This file describes the network interfaces available on your system# and how to activate them. For more information, see interfaces(5).# The loopback network interfaceauto loiface lo inet loopback# Source interfaces# Please check /etc/network/interfaces.d before changing this file# as interfaces may have been defined in /etc/network/interfaces.d# NOTE: the primary ethernet device is defined in# /etc/network/interfaces.d/eth0# See LP: #1262951#source /etc/network/interfaces.d/*.cfg# The management network interface  auto eth0  iface eth0 inet static  address 10.0.0.21  netmask 255.255.255.0# VM traffic interface  auto eth1  iface eth1 inet static  address 10.0.1.21  netmask 255.255.255.0# The public network interface auto eth2 iface eth2 inet static address 192.168.100.21 netmask 255.255.255.0 gateway 192.168.100.1 dns-nameservers 114.114.114.114

设置完毕，重启虚拟机。

这个时候，你就可以访问外网，安装包。

apt-get update -y && apt-get upgrade -y && apt-get dist-upgrade

同步时间

apt-get install -y ntp

编辑 /etc/ntp.conf

server 10.0.0.11

重启NTP服务

service ntp restart

安装基础组件

apt-get install -y vlan bridge-utils

编辑 /etc/sysctl.conf

net.ipv4.ip_forward=1net.ipv4.conf.all.rp_filter=0net.ipv4.conf.default.rp_filter=0

检测

sysctl -p

安装Neutron组件

apt-get install -y neutron-plugin-ml2 neutron-plugin-openvswitch-agent \dnsmasq neutron-l3-agent neutron-dhcp-agent

编辑 /etc/neutron/neutron.conf , 这里修改的内容，比控制节点少很多。

#core_plugin = neutron.plugins.ml2.plugin.Ml2Plugincore_plugin = ml2# service_plugins =# Example: service_plugins = router,firewall,lbaas,vpnaas,meteringservice_plugins = router# The strategy to be used for auth.# Supported values are 'keystone'(default), 'noauth'.auth_strategy = keystoneallow_overlapping_ips = Truerpc_backend = neutron.openstack.common.rpc.impl_komburabbit_host = 10.0.0.11[keystone_authtoken]#auth_host = 127.0.0.1#auth_port = 35357#auth_protocol = http#admin_tenant_name = %SERVICE_TENANT_NAME%#admin_user = %SERVICE_USER%#admin_password = %SERVICE_PASSWORD%#signing_dir = $state_path/keystone-signingauth_uri = http://10.0.0.11:5000auth_host = 10.0.0.11auth_port = 35357auth_protocol = httpadmin_tenant_name = serviceadmin_user = neutronadmin_password = service_pass

编辑 /etc/neutron/l3_agent.ini

interface_driver = neutron.agent.linux.interface.OVSInterfaceDriveruse_namespaces = True

编辑 /etc/neutron/dhcp_agent.ini

interface_driver = neutron.agent.linux.interface.OVSInterfaceDriverdhcp_driver = neutron.agent.linux.dhcp.Dnsmasquse_namespaces = True

编辑 /etc/neutron/metadata_agent.ini

auth_url = http://10.0.0.11:5000/v2.0auth_region = regionOneadmin_tenant_name = serviceadmin_user = neutronadmin_password = service_passnova_metadata_ip = 10.0.0.11metadata_proxy_shared_secret = helloOpenStack

登录控制节点，修改 /etc/nova.conf 在[DEFAULT] 加入下面内容

service_neutron_metadata_proxy = trueneutron_metadata_proxy_shared_secret = helloOpenStack

重启nova api服务

service nova-api restart

编辑 /etc/neutron/plugins/ml2/ml2_conf.ini

[ml2]type_drivers = gretenant_network_types = gremechanism_drivers = openvswitch[ml2_type_gre]tunnel_id_ranges = 1:1000[ovs]local_ip = 10.0.1.21tunnel_type = greenable_tunneling = True[securitygroup]firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriverenable_security_group = True

重启openvswitch

service openvswitch-switch restart

创建br-ex

创建br-ex连接外网，这个不太好理解，看图

大概意思是：我们创建一个bridge br-ex，把br-ex绑定在eth2下，eth2是连接到公网的路由器上的。

ovs-vsctl add-br br-exovs-vsctl add-port br-ex eth2

下面内容是我操作的结果,大家慢慢理解.

编辑 /etc/network/interfaces

# This file describes the network interfaces available on your system# and how to activate them. For more information, see interfaces(5).# The loopback network interfaceauto loiface lo inet loopback# Source interfaces# Please check /etc/network/interfaces.d before changing this file# as interfaces may have been defined in /etc/network/interfaces.d# NOTE: the primary ethernet device is defined in# /etc/network/interfaces.d/eth0# See LP: #1262951#source /etc/network/interfaces.d/*.cfg# The management network interface  auto eth0  iface eth0 inet static  address 10.0.0.21  netmask 255.255.255.0# VM traffic interface  auto eth1  iface eth1 inet static  address 10.0.1.21  netmask 255.255.255.0# The public network interface# auto eth2# iface eth2 inet static# address 192.168.100.21# netmask 255.255.255.0# gateway 192.168.100.1# dns-nameservers 114.114.114.114auto eth2iface eth2 inet manualup ifconfig $IFACE 0.0.0.0 upup ip link set $IFACE promisc ondown ip link set $IFACE promisc offdown ifconfig $IFACE downauto br-exiface br-ex inet staticaddress 192.168.100.21netmask 255.255.255.0gateway 192.168.100.1dns-nameservers 114.114.114.114

重启虚拟机

替换br-ex和eth2的mac地址

由于网络的限制，目前192.168.100.21和192.168.100.11是无法通讯的，原因是因为出于安全的考虑，对网络访问的mac地址和ip地址做了绑定和限制。

通过ifconfig 查看网卡的mac地址，通过命令，把mac地址互换。

• br-ex mac 地址 c2:32:7d:cf:9d:4
• eth2 mac地址 fa:16:3e:80:5d:e6

ip link set eth2 addr c2:32:7d:cf:9d:43ip link set br-ex addr fa:16:3e:80:5d:e6

这个时候，外部网络的IP就可以互相访问。这些修改是临时性的，如果重启neutron服务，mac地址就会恢复。不过我们实验不需要重启服务。这里提供的是临时的方法，后面有彻底解决问题的办法。

设置环境变量

cat <<EOF >>/root/credsexport OS_TENANT_NAME=adminexport OS_USERNAME=adminexport OS_PASSWORD=admin_passexport OS_AUTH_URL="http://192.168.100.11:5000/v2.0/"EOF

这样你就可以看到安装的agent

source credsneutron agent-list

# neutron agent-list+--------------------------------------+--------------------+---------+-------+----------------+| id                                   | agent_type         | host    | alive | admin_state_up |+--------------------------------------+--------------------+---------+-------+----------------+| 3a80d2ea-bcf6-4835-b125-55144948024c | Open vSwitch agent | network |    | True           || 4219dd20-c4fd-4586-b2fc-c81bec0015d6 | L3 agent           | network |    | True           || e956687f-a658-4226-a34f-368da61e9e44 | Metadata agent     | network |    | True           || f3e841f8-b803-4134-9ba6-3152c3db5592 | DHCP agent         | network |    | True           |+--------------------------------------+--------------------+---------+-------+----------------+

（4）计算节点

创建一个虚拟机，名字为：compute1, 删除网卡，并且添加2块网卡。ssh到虚拟机上.

计算节点默认是不需要接公网，不过由于我需要安装包，必须联网，所以你可以创建完虚拟机后，给虚拟机连接到外部网络，装完系统后，再断开就可以。

route add default gw 192.168.100.1

这个时候，你就可以访问外网，安装包。

apt-get update -y && apt-get upgrade -y && apt-get dist-upgrade

同步时间

apt-get install -y ntp

编辑 /etc/ntp.conf

server 10.0.0.11

重启NTP服务

service ntp restart

安装kvm套件

apt-get install -y kvm libvirt-bin pm-utils

安装计算节点组件

apt-get install -y nova-compute-kvm python-guestfs

让内核只读

dpkg-statoverride  --update --add root root 0644 /boot/vmlinuz-$(uname -r)

创建脚本 /etc/kernel/postinst.d/statoverride

#!/bin/shversion="$1"# passing the kernel version is required[ -z "${version}" ] && exit 0dpkg-statoverride --update --add root root 0644 /boot/vmlinuz-${version}

允许运行

chmod +x /etc/kernel/postinst.d/statoverride

编辑 /etc/nova/nova.conf 文件，添加下面内容

[DEFAULT]dhcpbridge_flagfile=/etc/nova/nova.confdhcpbridge=/usr/bin/nova-dhcpbridgelogdir=/var/log/novastate_path=/var/lib/novalock_path=/var/lock/novaforce_dhcp_release=Trueiscsi_helper=tgtadmlibvirt_use_virtio_for_bridges=Trueconnection_type=libvirtroot_helper=sudo nova-rootwrap /etc/nova/rootwrap.confverbose=Trueec2_private_dns_show_ip=Trueapi_paste_config=/etc/nova/api-paste.inivolumes_path=/var/lib/nova/volumesenabled_apis=ec2,osapi_compute,metadataauth_strategy = keystonerpc_backend = rabbitrabbit_host = 10.0.0.11my_ip = 10.0.0.31vnc_enabled = Truevncserver_listen = 0.0.0.0vncserver_proxyclient_address = 10.0.0.31novncproxy_base_url = http://192.168.100.11:6080/vnc_auto.htmlglance_host = 10.0.0.11vif_plugging_is_fatal=falsevif_plugging_timeout=0[database]connection = mysql://nova:NOVA_DBPASS@10.0.0.11/nova[keystone_authtoken]auth_uri = http://10.0.0.11:5000auth_host = 10.0.0.11auth_port = 35357auth_protocol = httpadmin_tenant_name = serviceadmin_user = novaadmin_password = service_pass

删除sqlite

rm /var/lib/nova/nova.sqlite

重启compute服务

service nova-compute restart

编辑 /etc/sysctl.conf

net.ipv4.ip_forward=1net.ipv4.conf.all.rp_filter=0net.ipv4.conf.default.rp_filter=0

马上生效

sysctl -p

安装网络组件

apt-get install -y neutron-common neutron-plugin-ml2 neutron-plugin-openvswitch-agent

编辑 /etc/neutron/neutron.conf

#core_plugin = neutron.plugins.ml2.plugin.Ml2Plugincore_plugin = ml2# service_plugins =# Example: service_plugins = router,firewall,lbaas,vpnaas,meteringservice_plugins = routerauth_strategy = keystoneallow_overlapping_ips = Truerpc_backend = neutron.openstack.common.rpc.impl_komburabbit_host = 10.0.0.11[keystone_authtoken]#auth_host = 127.0.0.1#auth_port = 35357#auth_protocol = http#admin_tenant_name = %SERVICE_TENANT_NAME%#admin_user = %SERVICE_USER%#admin_password = %SERVICE_PASSWORD%#signing_dir = $state_path/keystone-signingauth_uri = http://10.0.0.11:5000auth_host = 10.0.0.11auth_port = 35357auth_protocol = httpadmin_tenant_name = serviceadmin_user = neutronadmin_password = service_pass

编辑  /etc/neutron/plugins/ml2/ml2_conf.ini

[ml2]type_drivers = gretenant_network_types = gremechanism_drivers = openvswitch[ml2_type_gre]tunnel_id_ranges = 1:1000[ovs]local_ip = 10.0.1.31tunnel_type = greenable_tunneling = True[securitygroup]firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriverenable_security_group = True

重启OVS

service openvswitch-switch restart

再编辑 /etc/nova/nova.conf ，在[DEFAULT]里添加下面

network_api_class = nova.network.neutronv2.api.APIneutron_url = http://10.0.0.11:9696neutron_auth_strategy = keystoneneutron_admin_tenant_name = serviceneutron_admin_username = neutronneutron_admin_password = service_passneutron_admin_auth_url = http://10.0.0.11:35357/v2.0linuxnet_interface_driver = nova.network.linux_net.LinuxOVSInterfaceDriverfirewall_driver = nova.virt.firewall.NoopFirewallDriversecurity_group_api = neutron

编辑 /etc/nova/nova-compute.conf ,修改为使用qemu

[DEFAULT]compute_driver=libvirt.LibvirtDriver[libvirt]virt_type=qemu

重启相关服务

service nova-compute restartservice neutron-plugin-openvswitch-agent restart

安装就全部完成。

登录控制节点

root@controller:~# source creds root@controller:~# nova-manage service listBinary           Host                                 Zone             Status     State Updated_Atnova-cert        controller                           internal         enabled       2014-09-02 10:31:03nova-conductor   controller                           internal         enabled       2014-09-02 10:31:04nova-scheduler   controller                           internal         enabled       2014-09-02 10:30:58nova-consoleauth controller                           internal         enabled       2014-09-02 10:31:00nova-compute     compute1                             nova             enabled       2014-09-02 10:30:57root@controller:~#

1、命令行创建虚拟机

在控制节点上，运行下面的命令就可以。镜像我上面已经上传。下面的操作，你完全可以在Dashboard里进行操作，这里命令行下，了解更加深入。

下面的操作，在控制节点完成。

创建外部网络

source creds#Create the external network:neutron net-create ext-net --shared --router:external=True#Create the subnet for the external network:neutron subnet-create ext-net --name ext-subnet \--allocation-pool start=192.168.100.101,end=192.168.100.200 \--disable-dhcp --gateway 192.168.100.1 192.168.100.0/24

给租户创建内部网络

#Create the internal network:neutron net-create int-net#Create the subnet for the internal network:neutron subnet-create int-net --name int-subnet \--dns-nameserver 114.114.114.114 --gateway 172.16.1.1 172.16.1.0/24

创建路由，并且连接到外部网络

#Create the router:neutron router-create router1#Attach the router to the internal subnet:neutron router-interface-add router1 int-subnet#Attach the router to the external network by setting it as the gateway:neutron router-gateway-set router1 ext-net

创建密钥

ssh-keygen

添加公钥

nova keypair-add --pub-key ~/.ssh/id_rsa.pub key1

设置安全组

# Permit ICMP (ping):nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0# Permit secure shell (SSH) access:nova secgroup-add-rule default tcp 22 22 0.0.0.0/0

创建虚拟机

NET_ID=$(neutron net-list | awk '/ int-net / { print $2 }')nova boot --flavor m1.tiny --image cirros-0.3.2-x86_64 --nic net-id=$NET_ID \--security-group default --key-name key1 instance1

查看虚拟机

nova list

申请公网IP

neutron floatingip-create ext-net

关联floating IP

nova floating-ip-associate instance1 192.168.100.102

这个时候，你会发现你在控制节点上，根本是无法访问 router 192.168.100.101和floating ip 192.168.100.102。

访问虚拟机，你需要登录网络节点上，你可以用下面命令访问虚拟机

# ip netnsqdhcp-bf7f3043-d696-4735-9bc7-8c2e4d95c8d5qrouter-7e8bbb53-1ea6-4763-a69c-a0c875b5224b

第一个的虚拟机，第二个是路由器

# ip netns exec qdhcp-bf7f3043-d696-4735-9bc7-8c2e4d95c8d5 ifconfiglo        Link encap:Local Loopback            inet addr:127.0.0.1  Mask:255.0.0.0          inet6 addr: ::1/128 Scope:Host          UP LOOPBACK RUNNING  MTU:65536  Metric:1          RX packets:16 errors:0 dropped:0 overruns:0 frame:0          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0          collisions:0 txqueuelen:0           RX bytes:1216 (1.2 KB)  TX bytes:1216 (1.2 KB)tap1a85db16-da Link encap:Ethernet  HWaddr fa:16:3e:ce:e0:e2            inet addr:172.16.1.3  Bcast:172.16.1.255  Mask:255.255.255.0          inet6 addr: fe80::f816:3eff:fece:e0e2/64 Scope:Link          UP BROADCAST RUNNING  MTU:1500  Metric:1          RX packets:415 errors:0 dropped:0 overruns:0 frame:0          TX packets:105 errors:0 dropped:0 overruns:0 carrier:0          collisions:0 txqueuelen:0           RX bytes:64724 (64.7 KB)  TX bytes:10228 (10.2 KB)

2、访问公网

你可能发现一个很明显的问题，你在网络节点是可以ping 通虚拟机的floating IP，router的IP，不过你在控制节点是无法访问的。

如果希望比较完美，实现虚拟机可以ping通公网，那么需要我们多了解一下内容才行。可以发现全部的流量都是通过192.168.100.21这个端口出去，我们需要设置一下这个端口，运行所有的IP和mac地址通过。

登录网络节点，通过ping 192.168.100.101 和192.168.100.102 ，获得他们的mac地址。

# arp -a? (10.0.0.11) at fa:16:3e:34:d0:7a [ether] on eth0? (192.168.100.102) at fa:16:3e:0c:be:cd [ether] on br-ex? (10.0.1.31) at fa:16:3e:eb:96:1c [ether] on eth1? (192.168.100.101) at fa:16:3e:0c:be:cd [ether] on br-ex? (192.168.100.1) at fa:16:3e:c2:a8:a8 [ether] on br-ex

下面的操作，你可以在控制节点完成

通过curl获取token

使用token，修改192.168.100.21 port 的allow_address_pairs ，可以顺便把eth2和br-ex也修改，这样就不担心重启服务。

详细的操作，就参考这篇文档就可以。

http://www.chenshake.com/use-the-uos-api/

3、vnc访问

如果你登录Horizon，访问虚拟机，vnc可能无法访问，你需要登录uos，修改安全组规则。默认第一个虚拟机使用vnc的端口是6080。或者你全部打开端口。

参考资料

http://oddbit.com/rdo-hangout-multinode-packstack-slides/#/

https://github.com/ChaimaGhribi/OpenStack-Icehouse-Installation/blob/master/OpenStack-Icehouse-Installation.rst

参考文档 http://blog.oddbit.com/2014/05/23/open-vswitch-and-persistent-ma/

ovs-vsctl操作

root@network:~# ovs-vsctl show05dd-bd0d-4af1-a331-c9394fbcb775    Bridge br-int        fail_mode: secure        Port br-int            Interface br-int                type: internal    ovs_version: "2.0.2"root@network:~# ovs-vsctl add-br br-exroot@network:~# ovs-vsctl show        05dd-bd0d-4af1-a331-c9394fbcb775    Bridge br-ex        Port br-ex            Interface br-ex                type: internal    Bridge br-int        fail_mode: secure        Port br-int            Interface br-int                type: internal    ovs_version: "2.0.2"root@network:~# ovs-vsctl add-port br-ex eth2root@network:~# ovs-vsctl show05dd-bd0d-4af1-a331-c9394fbcb775    Bridge br-ex        Port "eth2"            Interface "eth2"        Port br-ex            Interface br-ex                type: internal    Bridge br-int        fail_mode: secure        Port br-int            Interface br-int                type: internal    ovs_version: "2.0.2"

网络节点重启服务

service neutron-plugin-openvswitch-agent restartservice neutron-dhcp-agent restartservice neutron-l3-agent restartservice neutron-metadata-agent restartservice dnsmasq restart

附录

建议采用vpn的方式，我就简单，采用pptp

PPTP，默认拨号连接上去，你本地就无法上网了。你需要设置一下。

去掉这个勾就可以了。

这个时候，你拨号连接上去，访问虚拟机的时候，走vpn通道，访问外网，还是走你以前的网络。

你还需要在你的增加一条路由才行，不过在win7，win8，增加路由，需要管理员权限。

按 Windows key + X, 再按 A

这时候，你就调出管理员权限的cmd

添加一条路由

route add 192.168.100.0 mask 255.255.255.0 10.100.100.1

这时候，你就可以ping通那台控制节点的虚拟机。不容易吧。

其他参考：http://www.tuicool.com/articles/jIVR3eM
http://www.aboutyun.com/thread-9670-1-1.html