swift mul node install

配置ubuntu12.04

一台代理节点

一台存储节点上面运行4个contain，4个object，4个account

1.在每一个节点创建一个swift用户  

sudo useradd -mk /home/swift/ -s /bin/bash swift    sudo passwd swift    #为swift用户添加密码，在此我将其设为了swift    编辑/etc/sudoer文件，在文件末尾添加如下代码    swift  ALL=(ALL) NOPASSWD:ALL  用着命令可以进行远程的sudo su

2.下载并却安装swift
    下载一些必要的软件  

apt-get update    apt-get install curl gcc memcached rsync sqlite3 xfsprogs git-core libffi-dev python-setuptools    apt-get install python-coverage python-dev python-nose python-simplejson python-xattr python-eventlet python-greenlet python-pastedeploy python-netifaces python-pip python-dnspython python-mock

    (我自己已经下载好了，直接用dpkg -i *.deb)

git  clone https://github.com/openstack/swift.git    cd swift/    sudo python setup.py install --record in.txt #in.txt 可以看安装了什么东西便于删除    sudo python setup.py develop 

3.代理节点上分别要配置proxy-server memcached 并且要生成3个ring.gz
  存储节点上要配置objiect-server contain-server account-server
 3.1
    配置 /etc/memcached.conf 将-l 127.0.0.1 换成代理机器的ip

  perl -pi -e "s/-l 127.0.0.1/-l $PROXY_LOCAL_NET_IP/" /etc/memcached.conf

    然后重启服务 

  sudo service memcached restart

 3.2
   创建Create /etc/swift/proxy-server.conf:
 

 [DEFAULT]bind_port = 8080workers = 1user = <your-user-name> #替换成你的用户名（swift）log_facility = LOG_LOCAL1eventlet_debug = true[pipeline:main]# Yes, proxy-logging appears twice. This is not a mistake.pipeline = healthcheck proxy-logging cache tempauth proxy-logging proxy-server[app:proxy-server]use = egg:swift#proxyallow_account_management = trueaccount_autocreate = true[filter:tempauth]use = egg:swift#tempauthuser_admin_admin = admin .admin .reseller_adminuser_test_tester = testing .adminuser_test2_tester2 = testing2 .adminuser_test_tester3 = testing3[filter:healthcheck]use = egg:swift#healthcheck[filter:cache]use = egg:swift#memcache[filter:proxy-logging]use = egg:swift#proxy_logging

3.3

    生成 swift.conf的hash标识符

    Create /etc/swift/swift.conf:

 [swift-hash]        # random unique strings that can never change (DO NOT LOSE)        swift_hash_path_prefix = changeme        swift_hash_path_suffix = changeme

    生成环(存储节点下的4个存储区域)
   

rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz    swift-ring-builder object.builder create 10 3 1    swift-ring-builder object.builder add r1z1-192.168.1.4:6010/sdb1 1    swift-ring-builder object.builder add r1z2-192.168.1.4:6020/sdb2 1    swift-ring-builder object.builder add r1z3-192.168.1.4:6030/sdb3 1    swift-ring-builder object.builder add r1z4-192.168.1.4:6040/sdb4 1    swift-ring-builder object.builder rebalance    swift-ring-builder container.builder create 10 3 1    swift-ring-builder container.builder add r1z1-192.168.1.4:6011/sdb1 1    swift-ring-builder container.builder add r1z2-192.168.1.4:6021/sdb2 1        swift-ring-builder container.builder add r1z3-192.168.1.4:6031/sdb3 1    swift-ring-builder container.builder add r1z4-192.168.1.4:6041/sdb4 1    swift-ring-builder container.builder rebalance    swift-ring-builder account.builder create 10 3 1    swift-ring-builder account.builder add r1z1-192.168.1.4:6012/sdb1 1    swift-ring-builder account.builder add r1z2-192.168.1.4:6022/sdb2 1    swift-ring-builder account.builder add r1z3-192.168.1.4:6032/sdb3 1    swift-ring-builder account.builder add r1z4-192.168.1.4:6042/sdb4 1    swift-ring-builder account.builder rebalance

    更改权限 /etc/swift 为swift用户

    chown -R swift:swift /etc/swift

  将生成的*.ring.gz 复制到存储节点的/etc/swift/下

 3.4（存储节点）
   存储节点首先要进行swift的软件安装，用户的建立。
   将代理节点上的/etc/swift/*.ring.gz 下载到当前节点的/etc/swift/下
 3.5配置代理节点的rsyncd

uid = <your-user-name>gid = <your-group-name>log file = /var/log/rsyncd.logpid file = /var/run/rsyncd.pidaddress = 127.0.0.1[account6012]max connections = 25path = /srv/1/node/read only = falselock file = /var/lock/account6012.lock[account6022]max connections = 25path = /srv/2/node/read only = falselock file = /var/lock/account6022.lock[account6032]max connections = 25path = /srv/3/node/read only = falselock file = /var/lock/account6032.lock[account6042]max connections = 25path = /srv/4/node/read only = falselock file = /var/lock/account6042.lock[container6011]max connections = 25path = /srv/1/node/read only = falselock file = /var/lock/container6011.lock[container6021]max connections = 25path = /srv/2/node/read only = falselock file = /var/lock/container6021.lock[container6031]max connections = 25path = /srv/3/node/read only = falselock file = /var/lock/container6031.lock[container6041]max connections = 25path = /srv/4/node/read only = falselock file = /var/lock/container6041.lock[object6010]max connections = 25path = /srv/1/node/read only = falselock file = /var/lock/object6010.lock[object6020]max connections = 25path = /srv/2/node/read only = falselock file = /var/lock/object6020.lock[object6030]max connections = 25path = /srv/3/node/read only = falselock file = /var/lock/object6030.lock[object6040]max connections = 25path = /srv/4/node/read only = falselock file = /var/lock/object6040.lock

重新启动 sudo service rsync restart
进行测试
rsync rsync://pub@localhost/

   3.6配置轮回设备

   Using a loopback device for storage
   If you want to use a loopback device instead of another partition, follow these instructions.
        mkdir /srv
        truncate -s 1GB /srv/swift-disk

            (modify size to make a larger or smaller partition)

        mkfs.xfs /srv/swift-disk

        Edit /etc/fstab and add

            /srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0

        mkdir /mnt/sdb1

        mount /mnt/sdb1

        mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4

        chown <your-user-name>:<your-group-name> /mnt/sdb1/*

        for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done

        mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift

        chown -R <your-user-name>:<your-group-name> /etc/swift /srv/[1-4]/ /var/run/swift – Make sure to include the trailing slash after /srv/[1-4]/

        Add to /etc/rc.local (before the exit 0):

        mkdir -p /var/cache/swift /var/cache/swift2 /var/cache/swift3 /var/cache/swift4
        chown <your-user-name>:<your-group-name> /var/cache/swift*
        mkdir -p /var/run/swift
        chown <your-user-name>:<your-group-name> /var/run/swift
        Note that on some systems you might have to create /etc/rc.local.
        On Fedora 19 or later, you need to place these in /etc/rc.d/rc.local.
   3.7更改/srv/1，2，3，4 的拥有者都为swift
   3.8进行contain-server account-server object-server 的配置
     http://docs.openstack.org/developer/swift/development_saio.html 与这里的配置一模一样

  Create /etc/swift/swift.conf:

 [swift-hash]        # random unique strings that can never change (DO NOT LOSE)        swift_hash_path_prefix = changeme        swift_hash_path_suffix = changeme

    

 Create /etc/swift/account-server/1.conf:[DEFAULT]devices = /srv/1/nodemount_check = falsedisable_fallocate = truebind_port = 6012workers = 1user = <your-user-name>log_facility = LOG_LOCAL2recon_cache_path = /var/cache/swifteventlet_debug = true[pipeline:main]pipeline = recon account-server[app:account-server]use = egg:swift#account[filter:recon]use = egg:swift#recon[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]Create /etc/swift/account-server/2.conf:[DEFAULT]devices = /srv/2/nodemount_check = falsedisable_fallocate = truebind_port = 6022workers = 1user = <your-user-name>log_facility = LOG_LOCAL3recon_cache_path = /var/cache/swift2eventlet_debug = true[pipeline:main]pipeline = recon account-server[app:account-server]use = egg:swift#account[filter:recon]use = egg:swift#recon[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]Create /etc/swift/account-server/3.conf:[DEFAULT]devices = /srv/3/nodemount_check = falsedisable_fallocate = truebind_port = 6032workers = 1user = <your-user-name>log_facility = LOG_LOCAL4recon_cache_path = /var/cache/swift3eventlet_debug = true[pipeline:main]pipeline = recon account-server[app:account-server]use = egg:swift#account[filter:recon]use = egg:swift#recon[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]Create /etc/swift/account-server/4.conf:[DEFAULT]devices = /srv/4/nodemount_check = falsedisable_fallocate = truebind_port = 6042workers = 1user = <your-user-name>log_facility = LOG_LOCAL5recon_cache_path = /var/cache/swift4eventlet_debug = true[pipeline:main]pipeline = recon account-server[app:account-server]use = egg:swift#account[filter:recon]use = egg:swift#recon[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]Create /etc/swift/container-server/1.conf:[DEFAULT]devices = /srv/1/nodemount_check = falsedisable_fallocate = truebind_port = 6011workers = 1user = <your-user-name>log_facility = LOG_LOCAL2recon_cache_path = /var/cache/swifteventlet_debug = true[pipeline:main]pipeline = recon container-server[app:container-server]use = egg:swift#container[filter:recon]use = egg:swift#recon[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]Create /etc/swift/container-server/2.conf:[DEFAULT]devices = /srv/2/nodemount_check = falsedisable_fallocate = truebind_port = 6021workers = 1user = <your-user-name>log_facility = LOG_LOCAL3recon_cache_path = /var/cache/swift2eventlet_debug = true[pipeline:main]pipeline = recon container-server[app:container-server]use = egg:swift#container[filter:recon]use = egg:swift#recon[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]Create /etc/swift/container-server/3.conf:[DEFAULT]devices = /srv/3/nodemount_check = falsedisable_fallocate = truebind_port = 6031workers = 1user = <your-user-name>log_facility = LOG_LOCAL4recon_cache_path = /var/cache/swift3eventlet_debug = true[pipeline:main]pipeline = recon container-server[app:container-server]use = egg:swift#container[filter:recon]use = egg:swift#recon[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]Create /etc/swift/container-server/4.conf:[DEFAULT]devices = /srv/4/nodemount_check = falsedisable_fallocate = truebind_port = 6041workers = 1user = <your-user-name>log_facility = LOG_LOCAL5recon_cache_path = /var/cache/swift4eventlet_debug = true[pipeline:main]pipeline = recon container-server[app:container-server]use = egg:swift#container[filter:recon]use = egg:swift#recon[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]Create /etc/swift/object-server/1.conf:[DEFAULT]devices = /srv/1/nodemount_check = falsedisable_fallocate = truebind_port = 6010workers = 1user = <your-user-name>log_facility = LOG_LOCAL2recon_cache_path = /var/cache/swifteventlet_debug = true[pipeline:main]pipeline = recon object-server[app:object-server]use = egg:swift#object[filter:recon]use = egg:swift#recon[object-replicator]vm_test_mode = yes[object-updater][object-auditor]Create /etc/swift/object-server/2.conf:[DEFAULT]devices = /srv/2/nodemount_check = falsedisable_fallocate = truebind_port = 6020workers = 1user = <your-user-name>log_facility = LOG_LOCAL3recon_cache_path = /var/cache/swift2eventlet_debug = true[pipeline:main]pipeline = recon object-server[app:object-server]use = egg:swift#object[filter:recon]use = egg:swift#recon[object-replicator]vm_test_mode = yes[object-updater][object-auditor]Create /etc/swift/object-server/3.conf:[DEFAULT]devices = /srv/3/nodemount_check = falsedisable_fallocate = truebind_port = 6030workers = 1user = <your-user-name>log_facility = LOG_LOCAL4recon_cache_path = /var/cache/swift3eventlet_debug = true[pipeline:main]pipeline = recon object-server[app:object-server]use = egg:swift#object[filter:recon]use = egg:swift#recon[object-replicator]vm_test_mode = yes[object-updater][object-auditor]Create /etc/swift/object-server/4.conf:[DEFAULT]devices = /srv/4/nodemount_check = falsedisable_fallocate = truebind_port = 6040workers = 1user = <your-user-name>log_facility = LOG_LOCAL5recon_cache_path = /var/cache/swift4eventlet_debug = true[pipeline:main]pipeline = recon object-server[app:object-server]use = egg:swift#object[filter:recon]use = egg:swift#recon[object-replicator]vm_test_mode = yes[object-updater][object-auditor]

        Update <your-user-name>:

       

find /etc/swift/ -name \*.conf | xargs sed -i "s/<your-user-name>/${USER}/"

  
测试:
   1.确保rsync和memchached启动
   

 sudo service rsync restart     sudo service memcached start

   2.复制文件（没懂干啥用的）
   

cp ~/swift/test/sample.conf /etc/swift/test.conf    ~/swift/.unittests 

（进行测试）

   2.进行注册
   

curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0

    curl -v -H 'X-Storage-User: 用户名:用户组' -H 'X-Storage-Pass: 密码' ip
    备注 ：
    swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
    错误 ：account HEAD failed: http://127.0.0.1:8080/v1/AUTH_test 401 Unauthorized
    有可能是memchached没有启动
   3.进行状态检测

 wift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat

回复
   Account: AUTH_test
Containers: 0
   Objects: 0
     Bytes: 0
Content-Type: text/plain; charset=utf-8
X-Timestamp: 1384246637.44463
X-Put-Timestamp: 1384246637.44463
   (如何这个可以正常显示，那么基本上就是配置成功了)
  4.上传
 

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing upload container_test /etc/swift/*.builder

  swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing upload 容器名 文件
显示（会以路径名未文件名进行上传）
etc/swift/container.builder
etc/swift/account.builder
etc/swift/object.builder
  5.下载
 

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing download  container_test

  swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing download  容器
 etc/swift/object.builder [headers 0.019s, total 0.036s, 0.255 MB/s]
 etc/swift/container.builder [headers 0.021s, total 0.037s, 0.249 MB/s]
 etc/swift/account.builder [headers 0.018s, total 0.037s, 0.252 MB/s]
 6.列出容器

 swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list

回复 ：container_test
 7.列出容器中的内容

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list container_test

    回复：
        etc/swift/account.builder
    etc/swift/container.builder
    etc/swift/object.builder

参考http://docs.openstack.org/developer/swift/howto_installmultinode.html

      http://docs.openstack.org/developer/swift/development_saio.html