[openstack swift]SAIO

来源：互联网发布：用友软件t6价格编辑：程序博客网时间：2024/05/22 00:26

原文地址：http://swift.openstack.org/development_saio.html

Openstack swift 安装说明，安装官方网站操作

SAIO - Swift All In One¶

Instructions for setting up a development VM安装开发用虚拟机

This section documents setting up a virtual machine for doing Swift development. The virtual machine will emulate running a four node Swift cluster.

Get the Ubuntu 10.04 LTS (Lucid Lynx) server image:
- Ubuntu Server ISO: http://releases.ubuntu.com/lucid/ubuntu-10.04.4-server-amd64.iso (717 MB)
- Ubuntu Live/Install: http://cdimage.ubuntu.com/releases/lucid/release/ubuntu-10.04.4-dvd-amd64.iso (4.2 GB)
- Ubuntu Mirrors: https://launchpad.net/ubuntu/+cdmirrors
Create guest virtual machine from the Ubuntu image.

Additional information about setting up a Swift development snapshot on other distributions is available on the wiki athttp://wiki.openstack.org/SAIOInstructions.

Installing dependencies and the core code安装依赖和核心代码

As root on guest (you’ll have to log in as you, then sudo su -):需要root权限运行以下
1. apt-get install python-software-properties
2. add-apt-repository ppa:swift-core/release
3. apt-get update
4. apt-get install curl gcc git-core memcached python-configobj python-coverage python-dev python-nose python-setuptools python-simplejson python-xattr sqlite3 xfsprogs python-webob python-eventlet python-greenlet python-pastedeploy python-netifaces
5. 安装其它你所需要的工具，例如： screen, ssh, vim, etc.
On Fedora, log in as root and do:
1. yum install openstack-swift openstack-swift-proxy openstack-swift-account openstack-swift-container openstack-swift-object
2. yum install xinetd rsync
3. yum install memcached
4. yum install python-netifaces python-nose
This installs all necessary dependencies, and also creates user swiftand groupswift. So,swift:swift ought to be used in every place where this manual calls for<your-user-name>:<your-group-name>.
Ensure that you are installing the version of Swift that corresponds to this document. If not, enable the correct update repositories.

Next, choose either Using a partition for storageorUsing a loopback device for storage.接下来选择分区创建的方式，选其一即可使用一个新的partition(逻辑卷)作为存储分区 或者 在已有的分区上建立存储分区

Using a partition for storage使用一个新的partition(逻辑卷)作为存储分区¶

If you are going to use a separate partition for Swift data, be sure to add another device when creating the VM, and follow these instructions.

fdisk /dev/sdb (set up a single partition)
mkfs.xfs -i size=1024 /dev/sdb1
Edit /etc/fstab and add
/dev/sdb1 /mnt/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0
mkdir /mnt/sdb1
mount /mnt/sdb1
mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
chown <your-user-name>:<your-group-name> /mnt/sdb1/*
mkdir /srv
for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done
mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift
chown -R <your-user-name>:<your-group-name> /etc/swift /srv/[1-4]/ /var/run/swift –Make sure to include the trailing slash after /srv/[1-4]/
Add to /etc/rc.local (before the exit 0):
mkdir -p /var/cache/swift /var/cache/swift2 /var/cache/swift3 /var/cache/swift4chown <your-user-name>:<your-group-name> /var/cache/swift*mkdir -p /var/run/swiftchown <your-user-name>:<your-group-name> /var/run/swift
Next, skip to Setting up rsync.

Using a loopback device for storage在已有的分区上建立存储分区¶

If you want to use a loopback device instead of another partition, follow these instructions.

mkdir /srv
dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000
(modify seek to make a larger or smaller partition)
mkfs.xfs -i size=1024 /srv/swift-disk
Edit /etc/fstab and add
/srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0
mkdir /mnt/sdb1
mount /mnt/sdb1
mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
chown <your-user-name>:<your-group-name> /mnt/sdb1/*
for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done
mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift
chown -R <your-user-name>:<your-group-name> /etc/swift /srv/[1-4]/ /var/run/swift –Make sure to include the trailing slash after /srv/[1-4]/
Add to /etc/rc.local (before the exit 0):
mkdir -p /var/cache/swift /var/cache/swift2 /var/cache/swift3 /var/cache/swift4chown <your-user-name>:<your-group-name> /var/cache/swift*mkdir -p /var/run/swiftchown <your-user-name>:<your-group-name> /var/run/swift

Setting up rsync设置rsync守护进程¶

Create /etc/rsyncd.conf:

uid = <Your user name>gid = <Your group name>log file = /var/log/rsyncd.logpid file = /var/run/rsyncd.pidaddress = 127.0.0.1[account6012]max connections = 25path = /srv/1/node/read only = falselock file = /var/lock/account6012.lock[account6022]max connections = 25path = /srv/2/node/read only = falselock file = /var/lock/account6022.lock[account6032]max connections = 25path = /srv/3/node/read only = falselock file = /var/lock/account6032.lock[account6042]max connections = 25path = /srv/4/node/read only = falselock file = /var/lock/account6042.lock[container6011]max connections = 25path = /srv/1/node/read only = falselock file = /var/lock/container6011.lock[container6021]max connections = 25path = /srv/2/node/read only = falselock file = /var/lock/container6021.lock[container6031]max connections = 25path = /srv/3/node/read only = falselock file = /var/lock/container6031.lock[container6041]max connections = 25path = /srv/4/node/read only = falselock file = /var/lock/container6041.lock[object6010]max connections = 25path = /srv/1/node/read only = falselock file = /var/lock/object6010.lock[object6020]max connections = 25path = /srv/2/node/read only = falselock file = /var/lock/object6020.lock[object6030]max connections = 25path = /srv/3/node/read only = falselock file = /var/lock/object6030.lock[object6040]max connections = 25path = /srv/4/node/read only = falselock file = /var/lock/object6040.lock

On Ubuntu, edit the following line in /etc/default/rsync:
```
RSYNC_ENABLE=true
```
On Fedora, edit the following line in /etc/xinetd.d/rsync:
```
disable = no
```
On Ubuntu service rsync restart

Starting memcached¶

On Fedora, make sure that memcached runs, running this if necessary:

systemctl enable memcached.service
systemctl start memcached.service

If this is not done, tokens of tempauth expire immediately and accessing Swift with curl becomes impossible.

Optional(可选): Setting up rsyslog for individual loggin生成日志文件¶

Create /etc/rsyslog.d/10-swift.conf:

# Uncomment the following to have a log containing all logs together#local1,local2,local3,local4,local5.*   /var/log/swift/all.log# Uncomment the following to have hourly proxy logs for stats processing#$template HourlyProxyLog,"/var/log/swift/hourly/%$YEAR%%$MONTH%%$DAY%%$HOUR%"#local1.*;local1.!notice ?HourlyProxyLoglocal1.*;local1.!notice /var/log/swift/proxy.loglocal1.notice           /var/log/swift/proxy.errorlocal1.*                ~local2.*;local2.!notice /var/log/swift/storage1.loglocal2.notice           /var/log/swift/storage1.errorlocal2.*                ~local3.*;local3.!notice /var/log/swift/storage2.loglocal3.notice           /var/log/swift/storage2.errorlocal3.*                ~local4.*;local4.!notice /var/log/swift/storage3.loglocal4.notice           /var/log/swift/storage3.errorlocal4.*                ~local5.*;local5.!notice /var/log/swift/storage4.loglocal5.notice           /var/log/swift/storage4.errorlocal5.*                ~

Edit /etc/rsyslog.conf and make the following change:
```
$PrivDropToGroup adm
```
mkdir -p /var/log/swift/hourly
chown -R syslog.adm /var/log/swift
chmod -R g+w /var/log/swift
service rsyslog restart

Getting the code and setting up test environment获得swift源代码并配置测试环境¶

Sample configuration files are provided with all defaults in line-by-line comments.配置文件的示例如下，并含有注释

Do these commands as you on guest.

mkdir ~/bin
Check out the swift repo with git clone https://github.com/openstack/swift.git
Build a development installation of swift, for example:cd ~/swift; sudo python setup.py develop
Check out the python-swiftclient repo with git clone https://github.com/openstack/python-swiftclient.git
Build a development installation of python-swiftclient, for example:cd ~/python-swiftclient; sudo python setup.py develop
Edit ~/.bashrc and add to the end:
export SWIFT_TEST_CONFIG_FILE=/etc/swift/test.confexport PATH=${PATH}:~/bin
. ~/.bashrc

Configuring each node配置各个结点(node)¶

Sample configuration files are provided with all defaults in line-by-line comments.

Create /etc/swift/proxy-server.conf:

[DEFAULT]bind_port = 8080user = <your-user-name>log_facility = LOG_LOCAL1[pipeline:main]pipeline = healthcheck cache tempauth proxy-logging proxy-server[app:proxy-server]use = egg:swift#proxyallow_account_management = trueaccount_autocreate = true[filter:tempauth]use = egg:swift#tempauthuser_admin_admin = admin .admin .reseller_adminuser_test_tester = testing .adminuser_test2_tester2 = testing2 .adminuser_test_tester3 = testing3[filter:healthcheck]use = egg:swift#healthcheck[filter:cache]use = egg:swift#memcache[filter:proxy-logging]use = egg:swift#proxy_logging

Create /etc/swift/swift.conf:

[swift-hash]# random unique string that can never change (DO NOT LOSE)swift_hash_path_suffix = changeme

Create /etc/swift/account-server/1.conf:

[DEFAULT]devices = /srv/1/nodemount_check = falsedisable_fallocate = truebind_port = 6012user = <your-user-name>log_facility = LOG_LOCAL2recon_cache_path = /var/cache/swift[pipeline:main]pipeline = recon account-server[app:account-server]use = egg:swift#account[filter:recon]use = egg:swift#recon[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]

Create /etc/swift/account-server/2.conf:

[DEFAULT]devices = /srv/2/nodemount_check = falsedisable_fallocate = truebind_port = 6022user = <your-user-name>log_facility = LOG_LOCAL3recon_cache_path = /var/cache/swift2[pipeline:main]pipeline = recon account-server[app:account-server]use = egg:swift#account[filter:recon]use = egg:swift#recon[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]

Create /etc/swift/account-server/3.conf:

[DEFAULT]devices = /srv/3/nodemount_check = falsedisable_fallocate = truebind_port = 6032user = <your-user-name>log_facility = LOG_LOCAL4recon_cache_path = /var/cache/swift3[pipeline:main]pipeline = recon account-server[app:account-server]use = egg:swift#account[filter:recon]use = egg:swift#recon[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]

Create /etc/swift/account-server/4.conf:

[DEFAULT]devices = /srv/4/nodemount_check = falsedisable_fallocate = truebind_port = 6042user = <your-user-name>log_facility = LOG_LOCAL5recon_cache_path = /var/cache/swift4[pipeline:main]pipeline = recon account-server[app:account-server]use = egg:swift#account[filter:recon]use = egg:swift#recon[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]

Create /etc/swift/container-server/1.conf:

[DEFAULT]devices = /srv/1/nodemount_check = falsedisable_fallocate = truebind_port = 6011user = <your-user-name>log_facility = LOG_LOCAL2recon_cache_path = /var/cache/swift[pipeline:main]pipeline = recon container-server[app:container-server]use = egg:swift#container[filter:recon]use = egg:swift#recon[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]

Create /etc/swift/container-server/2.conf:

[DEFAULT]devices = /srv/2/nodemount_check = falsedisable_fallocate = truebind_port = 6021user = <your-user-name>log_facility = LOG_LOCAL3recon_cache_path = /var/cache/swift2[pipeline:main]pipeline = recon container-server[app:container-server]use = egg:swift#container[filter:recon]use = egg:swift#recon[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]

Create /etc/swift/container-server/3.conf:

[DEFAULT]devices = /srv/3/nodemount_check = falsedisable_fallocate = truebind_port = 6031user = <your-user-name>log_facility = LOG_LOCAL4recon_cache_path = /var/cache/swift3[pipeline:main]pipeline = recon container-server[app:container-server]use = egg:swift#container[filter:recon]use = egg:swift#recon[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]

Create /etc/swift/container-server/4.conf:

[DEFAULT]devices = /srv/4/nodemount_check = falsedisable_fallocate = truebind_port = 6041user = <your-user-name>log_facility = LOG_LOCAL5recon_cache_path = /var/cache/swift4[pipeline:main]pipeline = recon container-server[app:container-server]use = egg:swift#container[filter:recon]use = egg:swift#recon[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]

Create /etc/swift/object-server/1.conf:

[DEFAULT]devices = /srv/1/nodemount_check = falsedisable_fallocate = truebind_port = 6010user = <your-user-name>log_facility = LOG_LOCAL2recon_cache_path = /var/cache/swift[pipeline:main]pipeline = recon object-server[app:object-server]use = egg:swift#object[filter:recon]use = egg:swift#recon[object-replicator]vm_test_mode = yes[object-updater][object-auditor]

Create /etc/swift/object-server/2.conf:

[DEFAULT]devices = /srv/2/nodemount_check = falsedisable_fallocate = truebind_port = 6020user = <your-user-name>log_facility = LOG_LOCAL3recon_cache_path = /var/cache/swift2[pipeline:main]pipeline = recon object-server[app:object-server]use = egg:swift#object[filter:recon]use = egg:swift#recon[object-replicator]vm_test_mode = yes[object-updater][object-auditor]

Create /etc/swift/object-server/3.conf:

[DEFAULT]devices = /srv/3/nodemount_check = falsedisable_fallocate = truebind_port = 6030user = <your-user-name>log_facility = LOG_LOCAL4recon_cache_path = /var/cache/swift3[pipeline:main]pipeline = recon object-server[app:object-server]use = egg:swift#object[filter:recon]use = egg:swift#recon[object-replicator]vm_test_mode = yes[object-updater][object-auditor]

Create /etc/swift/object-server/4.conf:

[DEFAULT]devices = /srv/4/nodemount_check = falsedisable_fallocate = truebind_port = 6040user = <your-user-name>log_facility = LOG_LOCAL5recon_cache_path = /var/cache/swift4[pipeline:main]pipeline = recon object-server[app:object-server]use = egg:swift#object[filter:recon]use = egg:swift#recon[object-replicator]vm_test_mode = yes[object-updater][object-auditor]

Setting up scripts for running Swift配置脚本运行 Swift¶

Create ~/bin/resetswift.
If you are using a loopback device substitute /dev/sdb1 with /srv/swift-disk.
If you did not set up rsyslog for individual logging, remove the find /var/log/swift... line:
#!/bin/bashswift-init all stopfind /var/log/swift -type f -exec rm -f {} \;sudo umount /mnt/sdb1sudo mkfs.xfs -f -i size=1024 /dev/sdb1sudo mount /mnt/sdb1sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4sudo chown <your-user-name>:<your-group-name> /mnt/sdb1/*mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log /var/log/syslogfind /var/cache/swift* -type f -name *.recon -exec rm -f {} \;sudo service rsyslog restartsudo service memcached restart
Create ~/bin/remakerings:
#!/bin/bashcd /etc/swiftrm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gzswift-ring-builder object.builder create 18 3 1swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1swift-ring-builder object.builder rebalanceswift-ring-builder container.builder create 18 3 1swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1swift-ring-builder container.builder rebalanceswift-ring-builder account.builder create 18 3 1swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1swift-ring-builder account.builder rebalance
Create ~/bin/startmain:
#!/bin/bashswift-init main start
Create ~/bin/startrest:
#!/bin/bashswift-init rest start
chmod +x ~/bin/*
remakerings
cd ~/swift; ./.unittests
startmain (The Unabletoincreasefiledescriptorlimit. Runningasnon-root? warnings are expected and ok.)
Get an X-Storage-Url and X-Auth-Token:curl-v-H'X-Storage-User:test:tester'-H'X-Storage-Pass:testing'http://127.0.0.1:8080/auth/v1.0
Check that you can GET account: curl-v-H'X-Auth-Token:<token-from-x-auth-token-above>'<url-from-x-storage-url-above>
Check that swift works: swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
cp ~/swift/test/sample.conf /etc/swift/test.conf
cd ~/swift; ./.functests (Note: functional tests will first delete everything in the configured accounts.)
cd ~/swift; ./.probetests (Note: probe tests will reset your environment as they callresetswift for each test.)

If you plan to work on documentation (and who doesn’t?!) you must install Sphinx and then you can build the documentation:

On Ubuntu:

sudo apt-get install python-sphinx
python setup.py build_sphinx

On MacOS:

sudo easy_install -U sphinx
python setup.py build_sphinx

Install tox so you find Py26 and PEP8 problems before Jenkins does: sudo apt-get install python2.6-dev python-pip sudo pip install tox

调试相关

如果按照如上配置后出现了问题，swift仍然未按预期执行，或者测试失败、auth无法认证，之类总总，以下可能会帮助你发现和解决问题:

所有事情都记录在 /var/log/syslog, 因此出现问题首先应该查看这里，看看是否出现什么错误的提示(多数情况会是python的一些反馈）。
确保所有server 的服务都是开启的。基本上任何正常情况下 Proxy、 Account、 Container、Object server都应该是正常运行在后台的。
如果上述server没有开启，syslog中也没有任何错误被记录。那么最好手动启动一下服务，例如：swift-object-server /etc/swift/object-server/1.conf 将启动object server。如果在syslog中没有记录任何事情，那么在手动启动后应该会看到相应的反馈