ceph error question

Question1

2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
osd的状态为down，无法激活up
[root@compute-192-168-2-233 ~]# ceph osd tree
id weight type name up/down reweight
-1 0.9 root default
-2 0.45 host compute-192-168-2-231
0 0.45 osd.0 down 1
-3 0.45 host compute-192-168-2-233
1 0.45 osd.1 down 1
2014-12-09 17:31:36.241343 7f5860dfa700 0 – :/1028616 >> 192.168.2.230:6789/0 pipe(0x7f5864021390 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7f5864021620).fault
2014-12-09 17:31:39.235376 7f5860cf9700 0 – :/1028616 >> 192.168.2.230:6789/0 pipe(0x7f5858000c00 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7f5858000e90).fault
2014-12-09 17:31:42.235596 7f5860dfa700 0 – :/1028616 >> 192.168.2.230:6789/0 pipe(0x7f5858003010 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f58580032a0).fault
2014-12-09 17:31:46.236063 7f5860cf9700 0 – :/1028616 >> 192.168.2.230:6789/0 pipe(0x7f5858003830 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f5858003ac0).fault
2014-12-09 17:31:49.235913 7f5860dfa700 0 – :/1028616 >> 192.168.2.230:6789/0 pipe(0x7f58580025c0 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f5858002850).fault
2014-12-09 17:31:52.235875 7f5860cf9700 0 – :/1028616 >> 192.168.2.230:6789/0 pipe(0x7f5858002d10 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f5858002fa0).fault
2014-12-09 17:31:55.236904 7f5860dfa700 0 – :/1028616 >> 192.168.2.230:6789/0 pipe(0x7f58580041f0 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f5858004480).fault
2014-12-09 17:31:58.236879 7f5860cf9700 0 – :/1028616 >> 192.168.2.230:6789/0 pipe(0x7f5858002cb0 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f5858002f40).fault
2014-12-09 17:32:01.237865 7f5860dfa700 0 – :/1028616 >> 192.168.2.230:6789/0 pipe(0x7f58580062f0 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f5858006580).fault
2014-12-09 17:32:04.237890 7f5860cf9700 0 – :/1028616 >> 192.168.2.230:6789/0 pipe(0x7f5858005240 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f58580054d0).fault
failed: ‘timeout 30 /usr/bin/ceph -c /etc/ceph/ceph.conf –name=osd.0 –keyring=/var/lib/ceph/osd/ceph-0/keyring osd crush create-or-move – 0 0.45 host=comte-192-168-2-231 root=default’

answer1：

主要是因为osd进程无法连接到minitor守护进程（192.168.2.230:6789），端口6789连接不上，关闭minitor所在主机的iptables服务。

Question2 ：

2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Ceph health

Answer2：

关闭各节点的防火墙
Service iptables stop

Question3：

[root@ceph-client ~]# sudo rbd map foo –pool rbd –name client.admin
ERROR: modinfo: could not find module rbd
FATAL: Module rbd not found.
rbd: modprobe rbd failed! (256)
在不到rbd模块
os：centos 6.5 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
安装了ceph client。

Answer3：

需要加载一个rbd内核模块，Linux2.6.34之前的版本都没有该模块，所以要把系统内核升级到2.6.34或者更新。
rpm –import http://elrepo.org/RPM-GPG-KEY-elrepo.org

导入key

rpm -Uvh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm#
安装源
yum –enablerepo=elrepo-kernel install kernel-ml#使elrepo-kernel源生效并安装内核模块
然后编辑/etc/grub.conf把Default=1改成Default=0，（如果不修改默认还是加载老的内核模块）使系统默认从新的内核启动

Question4：

ceph-deploy install admin-node node1 node2 node3
在安装firefly找giant源。
ceph-deploy looking for ceph-release-1-0.el6.noarch.rpm in wrong location
Message from ceph-deploy:
…. [INFO ] Running command: sudo rpm -Uvh –replacepkgs http://ceph.com/rpm-giant/el6/noarch/ceph-release-1-0.el6.noarch.rpm
…. [WARNIN] curl: (22) The requested URL returned error: 404 Not Found

Answer4：

安装ceph源，（rpm -Uvh –replacepkgs http://ceph.com/rpm-giant/el6/noarch/ceph-release-1-0.el6.noarch.rpm）在指定的路径下找不到giant版本的源；在ceph-deploy install 指定版本：
ceph-deploy install –release firefly admin-node node1 node2 node3

question5

ceph-deploy osd activate node:/osd5
 [10.10.200.165][WARNIN] Error EINVAL: entity osd.5 exists but key does not match
 [10.10.200.165][WARNIN] Traceback (most recent call last):
 [10.10.200.165][WARNIN] File “/usr/sbin/ceph-disk”, line 2591, in
 [10.10.200.165][WARNIN] main()
 [10.10.200.165][WARNIN] File “/usr/sbin/ceph-disk”, line 2569, in main
 [10.10.200.165][WARNIN] args.func(args)
 [10.10.200.165][WARNIN] File “/usr/sbin/ceph-disk”, line 1929, in main_activate
 [10.10.200.165][WARNIN] init=args.mark_init,
 [10.10.200.165][WARNIN] File “/usr/sbin/ceph-disk”, line 1761, in activate_dir
 [10.10.200.165][WARNIN] (osd_id, cluster) = activate(path, activate_key_template, init)
 [10.10.200.165][WARNIN] File “/usr/sbin/ceph-disk”, line 1897, in activate
 [10.10.200.165][WARNIN] keyring=keyring,
 [10.10.200.165][WARNIN] File “/usr/sbin/ceph-disk”, line 1520, in auth_key
 [10.10.200.165][WARNIN] ‘mon’, ‘allow profile osd’,
 [10.10.200.165][WARNIN] File “/usr/sbin/ceph-disk”, line 304, in command_check_call
 [10.10.200.165][WARNIN] return subprocess.check_call(arguments)
 [10.10.200.165][WARNIN] File “/usr/lib64/python2.6/subprocess.py”, line 505, in check_call
 [10.10.200.165][WARNIN] raise CalledProcessError(retcode, cmd)
 [10.10.200.165][WARNIN] subprocess.CalledProcessError: Command ‘[‘/usr/bin/ceph’, ‘–cluster’, ‘ceph’, ‘–name’, ‘client.bootstrap-osd’, ‘–keyring’, ‘/var/lib/ceph/bootstrap
osd/ceph.keyring’, ‘auth’, ‘add’, ‘osd.5’, ‘-i’, ‘/osd5/keyring’, ‘osd’, ‘allow *’, ‘mon’, ‘allow profile osd’]’ returned non-zero exit status 22
 [10.10.200.165][ERROR ] RuntimeError: command returned non-zero exit status: 1
 [ceph_deploy][ERROR ] RuntimeError: Failed to execute command: ceph-disk -v activate –mark-init sysvinit –moun
在有一个osd的磁盘坏掉之后，你要删除其osd然后在添加一个新的osd会出现这样的情况.

Answer5：

这是因为在删除osd的时候没有删除原先的key导致的。所以如果要删除osd要先删除其osd所用的ceph auth del osd.5,然后再删除osd， ceph osd rm 5,这样在添加osd.5就不会出错了。

Question6

[root@compute-192-168-2-144 init.d]# yum -y install yum-plugin-priorities
Loaded plugins: fastestmirror, priorities, security
Determining fastest mirrors
Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again

answer6

处理很简单，修改文件“/etc/yum.repos.d/epel.repo”， 将baseurl的注释取消， mirrorlist注释掉。即可。

Qusetion7

req 23:0.008213:swift:HEAD /swift/v1/volume-02:stat_bucket:reading permissions
2015-01-24 04:01:53.442282 7f1b2b1c4700 20 get_obj_state: rctx=0x7f1b2b1c3640 obj=.rgw:volume-02 state=0x7f1b60046a88 s->prefetch_data=0
2015-01-24 04:01:53.442287 7f1b2b1c4700 10 cache get: name=.rgw+volume-02 : miss
2015-01-24 04:01:55.178199 7f1b2b1c4700 10 cache put: name=.rgw+volume-02
2015-01-24 04:01:55.178209 7f1b2b1c4700 10 adding .rgw+volume-02 to cache LRU end
2015-01-24 04:01:55.178222 7f1b2b1c4700 10 read_permissions on volume-02(@[]): only_bucket=0 ret=-2002
2015-01-24 04:01:55.178246 7f1b2b1c4700 2 req 23:1.744196:swift:HEAD /swift/v1/volume-02:stat_bucket:http status=404

2015-01-24 02:45:57.846420 7f2b1e5f6700 20 token_id=7ba2bf61d6c883ff5576fc02fce1270e
2015-01-24 02:45:57.846785 7f2b1e5f6700 20 successfully decoded pki token
2015-01-24 02:45:57.846792 7f2b1e5f6700 20 received response: {“access”: {“token”: {“issued_at”: “2015-01-24T15:41:06.433712”, “expires”: “2015-01-24T16:41:06Z”, “id”: “placeholder”, “tenant”: {“description”: “admin tenant”, “enabled”: true, “id”: “41541bd904e4403889819e34931c439e”, “name”: “admin”}}, “serviceCatalog”: [{“endpoints”: [{“adminURL”: “http://192.168.2.141:8774/v2/41541bd904e4403889819e34931c439e“, “region”: “RegionOne”, “internalURL”: “http://192.168.2.141:8774/v2/41541bd904e4403889819e34931c439e“, “id”: “12e85a1b41214153947ed701416b9777”, “publicURL”: “http://192.168.2.141:8774/v2/41541bd904e4403889819e34931c439e“}], “endpoints_links”: [], “type”: “compute”, “name”: “nova”}, {“endpoints”: [{“adminURL”: “http://192.168.2.141:9696/“, “region”: “RegionOne”, “internalURL”: “http://192.168.2.141:9696/“, “id”: “2866bb7c6a104c9f85eaec7188608674”, “publicURL”: “http://192.168.2.141:9696/“}], “endpoints_links”: [], “type”: “network”, “name”: “neutron”}, {“endpoints”: [{“adminURL”: “http://192.168.2.141:8776/v2/41541bd904e4403889819e34931c439e“, “region”: “RegionOne”, “internalURL”: “http://192.168.2.141:8776/v2/41541bd904e4403889819e34931c439e“, “id”: “3f0e2fc3aa2c4055ab03930edbb030b4”, “publicURL”: “http://192.168.2.141:8776/v2/41541bd904e4403889819e34931c439e“}], “endpoints_links”: [], “type”: “volumev2”, “name”: “cinderv2”}, {“endpoints”: [{“adminURL”: “http://192.168.2.141:9292“, “region”: “RegionOne”, “internalURL”: “http://192.168.2.141:9292“, “id”: “0565aab3e9b1480aaf894dcf7bb96ac6”, “publicURL”: “http://192.168.2.141:9292“}], “endpoints_links”: [], “type”: “image”, “name”: “glance”}, {“endpoints”: [{“adminURL”: “http://192.168.2.141:8776/v1/41541bd904e4403889819e34931c439e“, “region”: “RegionOne”, “internalURL”: “http://192.168.2.141:8776/v1/41541bd904e4403889819e34931c439e“, “id”: “0656d40822b347dda9f85914c21c9811”, “publicURL”: “http://192.168.2.141:8776/v1/41541bd904e4403889819e34931c439e“}], “endpoints_links”: [], “type”: “volume”, “name”: “cinder”}, {“endpoints”: [{“adminURL”: “http://192.168.2.141:8773/services/Admin“, “region”: “RegionOne”, “internalURL”: “http://192.168.2.141:8773/services/Cloud“, “id”: “6aa20f0e9cc040348604b2d497d1858d”, “publicURL”: “http://192.168.2.141:8773/services/Cloud“}], “endpoints_links”: [], “type”: “ec2”, “name”: “nova_ec2”}, {“endpoints”: [{“adminURL”: “http://192.168.2.232/swift/v1“, “region”: “RegionOne”, “internalURL”: “http://192.168.2.232/swift/v1“, “id”: “2547ee33f06a4e09885b237be093029b”, “publicURL”: “http://192.168.2.232/swift/v1“}], “endpoints_links”: [], “type”: “object-store”, “name”: “swift”}, {“endpoints”: [{“adminURL”: “http://192.168.2.141:35357/v2.0“, “region”: “RegionOne”, “internalURL”: “http://192.168.2.141:5000/v2.0“, “id”: “9d3c9ce32afa4a5cbddf300e4b2d6a63”, “publicURL”: “http://192.168.2.141:5000/v2.0“}], “endpoints_links”: [], “type”: “identity”, “name”: “keystone”}], “user”: {“username”: “admin”, “roles_links”: [], “id”: “ad2cdbfd6cc04eefb5f4ca6cea081888”, “roles”: [{“name”: “admin”}], “name”: “admin”}, “metadata”: {“is_admin”: 0, “roles”: [“fb2bc5f029e34dbb9f11897c3a0409be”]}}}
erWlfFdJM7HgjhU-pLrbuFyXZ+ydGvfAwo05Y=”, “tenant”: {“enabled”: true, “id”: “41541bd904e4403889819e34931c439e”, “name”: “admin”, “description”: “admin tenant”}}, “serviceCatalog”: [{“endpoints_links”: [], “endpoints”: [{“adminURL”: “http://192.168.2.141:8774/v2/41541bd904e4403889819e34931c439e“, “region”: “RegionOne”, “publicURL”: “http://192.168.2.141:8774/v2/41541bd904e4403889819e34931c439e“, “id”: “12e85a1b41214153947ed701416b9777”, “internalURL”: “http://192.168.2.141:8774/v2/41541bd904e4403889819e34931c439e“}], “type”: “compute”, “name”: “nova”}, {“endpoints_links”: [], “endpoints”: [{“adminURL”: “http://192.168.2.141:9696/“, “region”: “RegionOne”, “publicURL”: “http://192.168.2.141:9696/“, “id”: “2866bb7c6a104c9f85eaec7188608674”, “internalURL”: “http://192.168.2.141:9696/“}], “type”: “network”, “name”: “neutron”}, {“endpoints_links”: [], “endpoints”: [{“adminURL”: “http://192.168.2.141:8776/v2/41541bd904e4403889819e34931c439e“, “region”: “RegionOne”, “publicURL”: “http://192.168.2.141:8776/v2/41541bd904e4403889819e34931c439e“, “id”: “3f0e2fc3aa2c4055ab03930edbb030b4”, “internalURL”: “http://192.168.2.141:8776/v2/41541bd904e4403889819e34931c439e“}], “type”: “volumev2”, “name”: “cinderv2”}, {“endpoints_links”: [], “endpoints”: [{“adminURL”: “http://192.168.2.141:9292“, “region”: “RegionOne”, “publicURL”: “http://192.168.2.141:9292“, “id”: “0565aab3e9b1480aaf894dcf7bb96ac6”, “internalURL”: “http://192.168.2.141:9292“}], “type”: “image”, “name”: “glance”}, {“endpoints_links”: [], “endpoints”: [{“adminURL”: “http://192.168.2.141:8776/v1/41541bd904e4403889819e34931c439e“, “region”: “RegionOne”, “publicURL”: “http://192.168.2.141:8776/v1/41541bd904e4403889819e34931c439e“, “id”: “0656d40822b347dda9f85914c21c9811”, “internalURL”: “http://192.168.2.141:8776/v1/41541bd904e4403889819e34931c439e“}], “type”: “volume”, “name”: “cinder”}, {“endpoints_links”: [], “endpoints”: [{“adminURL”: “http://192.168.2.141:8773/services/Admin“, “region”: “RegionOne”, “publicURL”: “http://192.168.2.141:8773/services/Cloud“, “id”: “6aa20f0e9cc040348604b2d497d1858d”, “internalURL”: “http://192.168.2.141:8773/services/Cloud“}], “type”: “ec2”, “name”: “nova_ec2”}, {“endpoints_links”: [], “endpoints”: [{“adminURL”: “http://192.168.2.232/swift/v1“, “region”: “RegionOne”, “publicURL”: “http://192.168.2.232/swift/v1“, “id”: “0acbdff1c73747e5a23aac365d68309b”, “internalURL”: “http://192.168.2.232/swift/v1“}], “type”: “object-store”, “name”: “swift”}, {“endpoints_links”: [], “endpoints”: [{“adminURL”: “http://192.168.2.141:35357/v2.0“, “region”: “RegionOne”, “publicURL”: “http://192.168.2.141:5000/v2.0“, “id”: “9d3c9ce32afa4a5cbddf300e4b2d6a63”, “internalURL”: “http://192.168.2.141:5000/v2.0“}], “type”: “identity”, “name”: “keystone”}], “user”: {“username”: “admin”, “roles_links”: [], “id”: “ad2cdbfd6cc04eefb5f4ca6cea081888”, “roles”: [{“name”: “admin”}], “name”: “admin”}, “metadata”: {“is_admin”: 0, “roles”: [“fb2bc5f029e34dbb9f11897c3a0409be”]}}}
2015-01-24 03:05:25.973534 7f2b0c3d9700 0 validated token: admin:admin expires: 1422118703
2015-01-24 03:05:25.973566 7f2b0c3d9700 20 get_obj_state: rctx=0x7f2b6c0142b0 obj=.users.uid:41541bd904e4403889819e34931c439e state=0x7f2b6c029ea8 s->prefetch_data=0
2015-01-24 03:05:25.973571 7f2b0c3d9700 10 cache get: name=.users.uid+41541bd904e4403889819e34931c439e : hit
2015-01-24 03:05:25.973575 7f2b0c3d9700 20 get_obj_state: s->obj_tag was set empty
2015-01-24 03:05:25.973578 7f2b0c3d9700 10 cache get: name=.users.uid+41541bd904e4403889819e34931c439e : hit
2015-01-24 03:05:25.973592 7f2b0c3d9700 2 req 27:0.008316:swift:HEAD /swift/v1/shyouyun:stat_bucket:reading permissions
2015-01-24 03:05:25.973605 7f2b0c3d9700 20 get_obj_state: rctx=0x7f2b0c3d8640 obj=.rgw:shyouyun state=0x7f2b6c028f28 s->prefetch_data=0
2015-01-24 03:05:25.973608 7f2b0c3d9700 10 cache get: name=.rgw+shyouyun : miss
2015-01-24 03:05:25.974558 7f2b0c3d9700 10 cache put: name=.rgw+shyouyun
2015-01-24 03:05:25.974567 7f2b0c3d9700 10 adding .rgw+shyouyun to cache LRU end
2015-01-24 03:05:25.974571 7f2b0c3d9700 20 get_obj_state: s->obj_tag was set empty
2015-01-24 03:05:25.974573 7f2b0c3d9700 20 Read xattr: user.rgw.idtag
2015-01-24 03:05:25.974574 7f2b0c3d9700 20 Read xattr: user.rgw.manifest
2015-01-24 03:05:25.974578 7f2b0c3d9700 10 cache get: name=.rgw+shyouyun : type miss (requested=17, cached=22)
2015-01-24 03:05:25.974581 7f2b0c3d9700 20 get_obj_state: rctx=0x7f2b0c3d8640 obj=.rgw:shyouyun state=0x7f2b6c028f28 s->prefetch_data=0
2015-01-24 03:05:25.974587 7f2b0c3d9700 20 get_obj_state: rctx=0x7f2b0c3d8640 obj=.rgw:shyouyun state=0x7f2b6c028f28 s->prefetch_data=0
2015-01-24 03:05:25.974588 7f2b0c3d9700 20 state for obj=.rgw:shyouyun is not atomic, not appending atomic test
2015-01-24 03:05:25.974601 7f2b0c3d9700 20 rados->read obj-ofs=0 read_ofs=0 read_len=524288
2015-01-24 03:05:25.975144 7f2b0c3d9700 20 rados->read r=0 bl.length=143
2015-01-24 03:05:25.975154 7f2b0c3d9700 10 cache put: name=.rgw+shyouyun
2015-01-24 03:05:25.975155 7f2b0c3d9700 10 moving .rgw+shyouyun to cache LRU end
2015-01-24 03:05:25.975160 7f2b0c3d9700 20 rgw_get_bucket_info: bucket instance: shyouyun(@{i=.rgw.buckets.index,e=.rgw.buckets.extra}.rgw.buckets[default.24201.2])
2015-01-24 03:05:25.975165 7f2b0c3d9700 20 reading from .rgw:.bucket.meta.shyouyun:default.24201.2
2015-01-24 03:05:25.975182 7f2b0c3d9700 20 get_obj_state: rctx=0x7f2b0c3d8640 obj=.rgw:.bucket.meta.shyouyun:default.24201.2 state=0x7f2b6c013688 s->prefetch_data=0
2015-01-24 03:05:25.975185 7f2b0c3d9700 10 cache get: name=.rgw+.bucket.meta.shyouyun:default.24201.2 : miss
2015-01-24 03:05:26.475446 7f2b0c3d9700 10 cache put: name=.rgw+.bucket.meta.shyouyun:default.24201.2
2015-01-24 03:05:26.475456 7f2b0c3d9700 10 adding .rgw+.bucket.meta.shyouyun:default.24201.2 to cache LRU end
2015-01-24 03:05:26.475459 7f2b0c3d9700 10 updating xattr: name=user.rgw.acl bl.length()=159
2015-01-24 03:05:26.475464 7f2b0c3d9700 20 get_obj_state: s->obj_tag was set empty
2015-01-24 03:05:26.475466 7f2b0c3d9700 20 Read xattr: user.rgw.acl
2015-01-24 03:05:26.475466 7f2b0c3d9700 20 Read xattr: user.rgw.idtag
2015-01-24 03:05:26.475467 7f2b0c3d9700 20 Read xattr: user.rgw.manifest
2015-01-24 03:05:26.475472 7f2b0c3d9700 10 cache get: name=.rgw+.bucket.meta.shyouyun:default.24201.2 : type miss (requested=17, cached=22)
2015-01-24 03:05:26.475478 7f2b0c3d9700 20 get_obj_state: rctx=0x7f2b0c3d8640 obj=.rgw:.bucket.meta.shyouyun:default.24201.2 state=0x7f2b6c013688 s->prefetch_data=0
2015-01-24 03:05:26.475488 7f2b0c3d9700 20 get_obj_state: rctx=0x7f2b0c3d8640 obj=.rgw:.bucket.meta.shyouyun:default.24201.2 state=0x7f2b6c013688 s->prefetch_data=0
2015-01-24 03:05:26.475490 7f2b0c3d9700 20 state for obj=.rgw:.bucket.meta.shyouyun:default.24201.2 is not atomic, not appending atomic test
2015-01-24 03:05:26.475507 7f2b0c3d9700 20 rados->read obj-ofs=0 read_ofs=0 read_len=524288
2015-01-24 03:05:26.476350 7f2b0c3d9700 20 rados->read r=0 bl.length=202
2015-01-24 03:05:26.476360 7f2b0c3d9700 10 cache put: name=.rgw+.bucket.meta.shyouyun:default.24201.2
2015-01-24 03:05:26.476363 7f2b0c3d9700 10 moving .rgw+.bucket.meta.shyouyun:default.24201.2 to cache LRU end
2015-01-24 03:05:26.476387 7f2b0c3d9700 15 Read AccessControlPolicyshyouyunshanghai youyunshyouyunshanghai youyunFULL_CONTROL
2015-01-24 03:05:26.476414 7f2b0c3d9700 2 req 27:0.511137:swift:HEAD /swift/v1/shyouyun:stat_bucket:init op
2015-01-24 03:05:26.476419 7f2b0c3d9700 2 req 27:0.511143:swift:HEAD /swift/v1/shyouyun:stat_bucket:verifying op mask
2015-01-24 03:05:26.476422 7f2b0c3d9700 20 required_mask= 1 user.op_mask=7
2015-01-24 03:05:26.476423 7f2b0c3d9700 2 req 27:0.511147:swift:HEAD /swift/v1/shyouyun:stat_bucket:verifying op permissions
2015-01-24 03:05:26.476426 7f2b0c3d9700 5 Searching permissions for uid=41541bd904e4403889819e34931c439e mask=49
2015-01-24 03:05:26.476427 7f2b0c3d9700 5 Permissions for user not found
2015-01-24 03:05:26.476428 7f2b0c3d9700 5 Searching permissions for group=1 mask=49
2015-01-24 03:05:26.476429 7f2b0c3d9700 5 Permissions for group not found
2015-01-24 03:05:26.476430 7f2b0c3d9700 5 Searching permissions for group=2 mask=49
2015-01-24 03:05:26.476431 7f2b0c3d9700 5 Permissions for group not found
2015-01-24 03:05:26.476432 7f2b0c3d9700 5 Getting permissions id=41541bd904e4403889819e34931c439e owner=shyouyun perm=0
2015-01-24 03:05:26.476436 7f2b0c3d9700 10 uid=41541bd904e4403889819e34931c439e requested perm (type)=1, policy perm=0, user_perm_mask=1, acl perm=0
2015-01-24 03:05:26.476455 7f2b0c3d9700 2 req 27:0.511179:swift:HEAD /swift/v1/shyouyun:stat_bucket:http status=401
2015-01-24 03:05:26.476458 7f2b0c3d9700 1 ====== req done req=0x7f2b2400e580 http_status=401 ======

2015-01-24 02:04:42.202352 7fbafa1fc700 20 enqueued request req=0x7fbae000e720
2015-01-24 02:04:42.202362 7fbafa1fc700 20 RGWWQ:
2015-01-24 02:04:42.202364 7fbafa1fc700 20 req: 0x7fbae000e720
2015-01-24 02:04:42.202368 7fbafa1fc700 10 allocated request req=0x7fbae000ea10
2015-01-24 02:04:42.202373 7fbadaff7700 20 dequeued request req=0x7fbae000e720
2015-01-24 02:04:42.202378 7fbadaff7700 20 RGWWQ: empty
2015-01-24 02:04:42.202423 7fbadaff7700 20 CONTENT_LENGTH=0
2015-01-24 02:04:42.202425 7fbadaff7700 20 DOCUMENT_ROOT=/var/www/html
2015-01-24 02:04:42.202426 7fbadaff7700 20 FCGI_ROLE=RESPONDER
2015-01-24 02:04:42.202427 7fbadaff7700 20 GATEWAY_INTERFACE=CGI/1.1
2015-01-24 02:04:42.202428 7fbadaff7700 20 HTTP_ACCEPT=/
2015-01-24 02:04:42.202428 7fbadaff7700 20 HTTP_ACCEPT_ENCODING=gzip, deflate, compress
2015-01-24 02:04:42.202429 7fbadaff7700 20 HTTP_AUTHORIZATION=
2015-01-24 02:04:42.202430 7fbadaff7700 20 HTTP_HOST=192.168.2.232
2015-01-24 02:04:42.202431 7fbadaff7700 20 HTTP_USER_AGENT=python-swiftclient-2.1.0
2015-01-24 02:04:42.202431 7fbadaff7700 20 HTTP_X_AUTH_KEY=QJQagHNBE3O8tQCtxtzfREoJZeBZJ08+UqH6c2Nl
2015-01-24 02:04:42.202432 7fbadaff7700 20 HTTP_X_AUTH_USER=shyouyun:swift
2015-01-24 02:04:42.202433 7fbadaff7700 20 PATH=/sbin:/usr/sbin:/bin:/usr/bin
2015-01-24 02:04:42.202433 7fbadaff7700 20 QUERY_STRING=
2015-01-24 02:04:42.202434 7fbadaff7700 20 REMOTE_ADDR=192.168.2.141
2015-01-24 02:04:42.202435 7fbadaff7700 20 REMOTE_PORT=54621
2015-01-24 02:04:42.202435 7fbadaff7700 20 REQUEST_METHOD=GET
2015-01-24 02:04:42.202436 7fbadaff7700 20 REQUEST_URI=/auth
2015-01-24 02:04:42.202439 7fbadaff7700 20 SCRIPT_FILENAME=/var/www/html/s3gw.fcgi
2015-01-24 02:04:42.202439 7fbadaff7700 20 SCRIPT_NAME=/auth
2015-01-24 02:04:42.202440 7fbadaff7700 20 SCRIPT_URI=http://192.168.2.232/auth
2015-01-24 02:04:42.202441 7fbadaff7700 20 SCRIPT_URL=/auth
2015-01-24 02:04:42.202441 7fbadaff7700 20 SERVER_ADDR=192.168.2.232
2015-01-24 02:04:42.202442 7fbadaff7700 20 SERVER_ADMIN=[e-mail]
2015-01-24 02:04:42.202443 7fbadaff7700 20 SERVER_NAME=192.168.2.232
2015-01-24 02:04:42.202444 7fbadaff7700 20 SERVER_PORT=80
2015-01-24 02:04:42.202444 7fbadaff7700 20 SERVER_PROTOCOL=HTTP/1.1
2015-01-24 02:04:42.202445 7fbadaff7700 20 SERVER_SIGNATURE=
2015-01-24 02:04:42.202454 7fbadaff7700 20 SERVER_SOFTWARE=Apache/2.2.22 (Fedora)
2015-01-24 02:04:42.202455 7fbadaff7700 1 ====== starting new request req=0x7fbae000e720 =====
2015-01-24 02:04:42.202465 7fbadaff7700 2 req 6:0.000011::GET /auth::initializing
2015-01-24 02:04:42.202469 7fbadaff7700 10 host=192.168.2.232 rgw_dns_name=cluster
2015-01-24 02:04:42.202485 7fbadaff7700 2 req 6:0.000031:swift-auth:GET /auth::getting op
2015-01-24 02:04:42.202490 7fbadaff7700 2 req 6:0.000036:swift-auth:GET /auth:swift_auth_get:authorizing
2015-01-24 02:04:42.202492 7fbadaff7700 2 req 6:0.000038:swift-auth:GET /auth:swift_auth_get:reading permissions
2015-01-24 02:04:42.202493 7fbadaff7700 2 req 6:0.000039:swift-auth:GET /auth:swift_auth_get:init op
2015-01-24 02:04:42.202495 7fbadaff7700 2 req 6:0.000041:swift-auth:GET /auth:swift_auth_get:verifying op mask
2015-01-24 02:04:42.202497 7fbadaff7700 20 required_mask= 0 user.op_mask=7
2015-01-24 02:04:42.202498 7fbadaff7700 2 req 6:0.000044:swift-auth:GET /auth:swift_auth_get:verifying op permissions
2015-01-24 02:04:42.202499 7fbadaff7700 2 req 6:0.000045:swift-auth:GET /auth:swift_auth_get:verifying op params
2015-01-24 02:04:42.202501 7fbadaff7700 2 req 6:0.000047:swift-auth:GET /auth:swift_auth_get:executing
2015-01-24 02:04:42.202533 7fbadaff7700 20 get_obj_state: rctx=0x2710760 obj=.users.swift:shyouyun:swift state=0x2713d38 s->prefetch_data=0
2015-01-24 02:04:42.202540 7fbadaff7700 10 cache get: name=.users.swift+shyouyun:swift : type miss (requested=6, cached=0)
2015-01-24 02:04:42.510522 7fbb225f6700 2 RGWDataChangesLog::ChangesRenewThread: start
2015-01-24 02:04:43.047449 7fbadaff7700 10 cache put: name=.users.swift+shyouyun:swift
2015-01-24 02:04:43.047457 7fbadaff7700 10 moving .users.swift+shyouyun:swift to cache LRU end
2015-01-24 02:04:43.047482 7fbadaff7700 2 req 6:0.845027:swift-auth:GET /auth:swift_auth_get:http status=403
2015-01-24 02:04:43.047486 7fbadaff7700 1 ====== req done req=0x7fbae000e720 http_status=403 ======
2015-01-24 02:04:43.047490 7fbadaff7700 20 process_request() returned -1

Answer：

以上出错信息，均是使用swiftclient 2.1.0版本中出现的错误，据网上介绍2.3.x也有该问题，是swiftclient在对http请做中间转换时出了错误，解决该问题可以使用s3api提交请求是没问题的，或者使用2.0.3版本的swiftclient可以解决该问题，也可以使用用curl直接调用ceph gw的api。
curl -i http://192.168.2.232/swift/v1/youyun -X PUT -H “X-Auth-Token: AUTH_rgwtk0e0000007368796f7579756e3a7377696674eea0da98a7c02bff4ff4c554e06bb01c51d9a5deb37b5295d93d4dad2e3f4827d3e75a99”