iscsi Initiator 登录target时报错iscsiadm: Login I/O error, failed to receive a PDU

iscsi Initiator 登录target时报错iscsiadm: Login I/O error, failed to receive a PDU

[root@rac2 ~]# iscsiadm -m discovery -t st -p 10.10.10.111:3260 -I iseriscsiadm: recv's end state machine bug?iscsiadm: Login I/O error, failed to receive a PDUiscsiadm: retrying discovery login to 10.10.10.111iscsiadm: recv's end state machine bug?iscsiadm: Login I/O error, failed to receive a PDUiscsiadm: retrying discovery login to 10.10.10.111iscsiadm: recv's end state machine bug?iscsiadm: Login I/O error, failed to receive a PDUiscsiadm: retrying discovery login to 10.10.10.111iscsiadm: recv's end state machine bug?iscsiadm: Login I/O error, failed to receive a PDUiscsiadm: retrying discovery login to 10.10.10.111iscsiadm: recv's end state machine bug?iscsiadm: Login I/O error, failed to receive a PDUiscsiadm: retrying discovery login to 10.10.10.111iscsiadm: recv's end state machine bug?iscsiadm: Login I/O error, failed to receive a PDUiscsiadm: retrying discovery login to 10.10.10.111iscsiadm: connection login retries (reopen_max) 5 exceedediscsiadm: Could not perform SendTargets discovery: encountered iSCSI login failure[root@rac2 ~]#

但是ping 该ip地址是ok的。

[root@rac2 ~]# ping 10.10.10.111PING 10.10.10.111 (10.10.10.111) 56(84) bytes of data.64 bytes from 10.10.10.111: icmp_seq=1 ttl=64 time=0.156 ms64 bytes from 10.10.10.111: icmp_seq=2 ttl=64 time=0.147 ms64 bytes from 10.10.10.111: icmp_seq=3 ttl=64 time=0.109 ms^C--- 10.10.10.111 ping statistics ---3 packets transmitted, 3 received, 0% packet loss, time 1999msrtt min/avg/max/mdev = 0.109/0.137/0.156/0.022 ms[root@rac2 ~]#

根据redhat.com文章：
https://access.redhat.com/solutions/2158021
明确说明是存储端的问题。

建议使用 debug模式来排错

iscsiadm -m discovery -t st -d8 -p X.X.X.X

于是

iscsiadm -m discovery -t st -d8 -p 10.10.10.111:3260 -I iser

不过功力尚浅，看不出什么门道，先记录下来输出：

[root@rac2 ~]# iscsiadm -m discovery -t st -d8 -p 10.10.10.111:3260 -I iseriscsiadm: ip 10.10.10.111, port 3260, tgpt -1iscsiadm: Max file limits 1024 4096iscsiadm: updating defaults from '/etc/iscsi/iscsid.conf'iscsiadm: updated 'discovery.sendtargets.iscsi.MaxRecvDataSegmentLength', '32768' => '32768'iscsiadm: updated 'node.startup', 'manual' => 'automatic'iscsiadm: updated 'node.leading_login', 'No' => 'No'iscsiadm: updated 'node.session.timeo.replacement_timeout', '120' => '120'iscsiadm: updated 'node.conn[0].timeo.login_timeout', '30' => '15'iscsiadm: updated 'node.conn[0].timeo.logout_timeout', '15' => '15'iscsiadm: updated 'node.conn[0].timeo.noop_out_interval', '5' => '5'iscsiadm: updated 'node.conn[0].timeo.noop_out_timeout', '5' => '5'iscsiadm: updated 'node.session.err_timeo.abort_timeout', '15' => '15'iscsiadm: updated 'node.session.err_timeo.lu_reset_timeout', '30' => '30'iscsiadm: updated 'node.session.err_timeo.tgt_reset_timeout', '30' => '30'iscsiadm: updated 'node.session.initial_login_retry_max', '4' => '8'iscsiadm: updated 'node.session.cmds_max', '128' => '128'iscsiadm: updated 'node.session.queue_depth', '32' => '32'iscsiadm: updated 'node.session.xmit_thread_priority', '-20' => '-20'iscsiadm: updated 'node.session.iscsi.InitialR2T', 'No' => 'No'iscsiadm: updated 'node.session.iscsi.ImmediateData', 'Yes' => 'Yes'iscsiadm: updated 'node.session.iscsi.FirstBurstLength', '262144' => '262144'iscsiadm: updated 'node.session.iscsi.MaxBurstLength', '16776192' => '16776192'iscsiadm: updated 'node.conn[0].iscsi.MaxRecvDataSegmentLength', '262144' => '262144'iscsiadm: updated 'node.conn[0].iscsi.MaxXmitDataSegmentLength', '0' => '0'iscsiadm: updated 'node.conn[0].iscsi.HeaderDigest', 'None' => 'None'iscsiadm: updated 'node.session.nr_sessions', '1' => '1'iscsiadm: updated 'node.session.iscsi.FastAbort', 'Yes' => 'Yes'iscsiadm: in read_transportsiscsiadm: Adding new transport iseriscsiadm: Matched transport iseriscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/iser'/'handle'iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/iscsi_transport/iser/handle'iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/iscsi_transport/iser/handle'iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/iser/handle' with attribute value '18446744072106209312'iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/iser'/'caps'iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/iscsi_transport/iser/caps'iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/iscsi_transport/iser/caps'iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/iser/caps' with attribute value '0x89'iscsiadm: Could not match iface[hw=,ip=,net_if=,iscsi_if=iser] to host.iscsiadm: Looking for config file /var/lib/iscsi/send_targets/10.10.10.111,3260iscsiadm: Looking for config file /var/lib/iscsi/send_targets/10.10.10.111,3260 config st_config.iscsiadm: updated 'discovery.startup', 'manual' => 'manual'iscsiadm: updated 'discovery.type', 'sendtargets' => 'sendtargets'iscsiadm: updated 'discovery.sendtargets.address', '' => '10.10.10.111'iscsiadm: updated 'discovery.sendtargets.port', '0' => '3260'iscsiadm: updated 'discovery.sendtargets.auth.authmethod', 'None' => 'None'iscsiadm: updated 'discovery.sendtargets.timeo.login_timeout', '15' => '15'iscsiadm: updated 'discovery.sendtargets.use_discoveryd', 'No' => 'No'iscsiadm: updated 'discovery.sendtargets.discoveryd_poll_inval', '30' => '30'iscsiadm: updated 'discovery.sendtargets.reopen_max', '5' => '5'iscsiadm: updated 'discovery.sendtargets.timeo.auth_timeout', '45' => '45'iscsiadm: updated 'discovery.sendtargets.timeo.active_timeout', '30' => '30'iscsiadm: updated 'discovery.sendtargets.iscsi.MaxRecvDataSegmentLength', '32768' => '32768'iscsiadm: disc rec already existsiscsiadm: Looking for config file /var/lib/iscsi/send_targets/10.10.10.111,3260iscsiadm: starting sendtargets discovery, address 10.10.10.111:3260,iscsiadm: in read_transportsiscsiadm: Updating transport iseriscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/iser'/'handle'iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/iser/handle'iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/iser'/'caps'iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/iser/caps'iscsiadm: no authentication configured...iscsiadm: sendtargets discovery to 10.10.10.111:3260 using isid 0x00023d000000iscsiadm: resolved 10.10.10.111 to 10.10.10.111iscsiadm: discovery timeouts: login 15, reopen_cnt 6, auth 45.iscsiadm: in ctldev_openiscsiadm: created NETLINK_ISCSI socket...iscsiadm: setting iface iser, dev , set ip , hw , transport iser.iscsiadm: iscsi_create_leading_conn discovery ep connectiscsiadm: in ktransport_ep_connectiscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: ktransport_ep_connect got handle 4iscsiadm: in ktransport_ep_polliscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: in ktransport_ep_polliscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: iscsi_create_leading_conn discovery create sessioniscsiadm: in kcreate_sessioniscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: expecting event 28, got 106, handling...iscsiadm: in ctldev_handleiscsiadm: in nl_readiscsiadm: ctldev_handle got event type 106iscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: iscsi_create_leading_conn discovery created session 18iscsiadm: iscsi_create_leading_conn discovery create conniscsiadm: in kcreate_conniscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: iscsi_create_leading_conn discovery bind conniscsiadm: in kbind_conniscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: connected to discovery address 10.10.10.111iscsiadm: discovery session to 10.10.10.111:3260 starting iSCSI loginiscsiadm: iscsi_create_session discovery set paramsiscsiadm: sysfs_attr_get_value: open '/class/iscsi_connection/connection18:0'/'ping_tmo'iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/iscsi_connection/connection18:0/ping_tmo'iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/iscsi_connection/connection18:0/ping_tmo'iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_connection/connection18:0/ping_tmo' with attribute value '0'iscsiadm: in kset_paramiscsiadm: set operational parameter 15 to:iscsiadm:iscsiadm: in kset_paramiscsiadm: set operational parameter 17 to:iscsiadm:iscsiadm: in kset_paramiscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: set operational parameter 18 to:iscsiadm: 0iscsiadm: in kset_paramiscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: set operational parameter 19 to:iscsiadm: 0iscsiadm: in kset_paramiscsiadm: set operational parameter 22 to:iscsiadm:iscsiadm: in kset_paramiscsiadm: set operational parameter 23 to:iscsiadm:iscsiadm: in kset_paramiscsiadm: set operational parameter 24 to:iscsiadm:iscsiadm: in kset_paramiscsiadm: set operational parameter 25 to:iscsiadm:iscsiadm: in kset_paramiscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: set operational parameter 26 to:iscsiadm: 0iscsiadm: in kset_paramiscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: set operational parameter 27 to:iscsiadm: 0iscsiadm: in kset_paramiscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: set operational parameter 28 to:iscsiadm: 0iscsiadm: in kset_paramiscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: set operational parameter 35 to:iscsiadm: 0iscsiadm: in kset_paramiscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: set operational parameter 30 to:iscsiadm: 0iscsiadm: in kset_paramiscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: set operational parameter 31 to:iscsiadm: 0iscsiadm: in kset_paramiscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: set operational parameter 32 to:iscsiadm: iseriscsiadm: in kset_paramiscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: set operational parameter 34 to:iscsiadm: iqn.1994-05.com.redhat:3fea8217eb5aiscsiadm: in kset_paramiscsiadm: set operational parameter 39 to:iscsiadm:iscsiadm: in kset_paramiscsiadm: set operational parameter 40 to:iscsiadm:iscsiadm: in kset_paramiscsiadm: set operational parameter 41 to:iscsiadm:iscsiadm: in kset_paramiscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: set operational parameter 43 to:iscsiadm: 1iscsiadm: sending login PDU with current stage 1, next stage 3, transit 0x80, isid 0x00023d120000 exp_statsn 0iscsiadm: >    InitiatorName=iqn.1994-05.com.redhat:3fea8217eb5aiscsiadm: >    InitiatorAlias=rac2iscsiadm: >    SessionType=Discoveryiscsiadm: >    HeaderDigest=Noneiscsiadm: >    DataDigest=Noneiscsiadm: >    DefaultTime2Wait=2iscsiadm: >    DefaultTime2Retain=0iscsiadm: >    IFMarker=Noiscsiadm: >    OFMarker=Noiscsiadm: >    ErrorRecoveryLevel=0iscsiadm: >    MaxRecvDataSegmentLength=8192iscsiadm: in ksend_pdu_beginiscsiadm: send PDU began for hdr 48 bytes and data 244 bytesiscsiadm: in kwriteviscsiadm: wrote 48 bytes of PDU headeriscsiadm: in kwriteviscsiadm: wrote 244 bytes of PDU dataiscsiadm: in ksend_pdu_endiscsiadm: in __kipc_calliscsiadm: in kwriteviscsiadm: in nlpayload_readiscsiadm: in nlpayload_readiscsiadm: send PDU finished for conn 18:0^Ciscsiadm: caught SIGINT, exiting... --------->>>>这是我执行ctrl+c的结果，否则会一直输出。[root@rac2 ~]#

突然想到我曾经写过测试ib网络的文章：
http://blog.csdn.net/msdnchina/article/details/71133494

于是在服务器端（有10.10.10.111 这个ip地址的这一端）敲入：

ibping -S -C mlx4_1 -P 2

在客户端执行：ibping -c 10000 -f -C mlx4_1 -P 2 -L 0

[root@rac2 ~]# ibping -c 10000 -f -C mlx4_1 -P 2 -L 0ibping: iberror: failed: can't resolve destination port 0[root@rac2 ~]# 

其实，有经验的人，一下就看出来了，链路正常的话，-L后边的值不能为0，这正说明ib链路不通，虽然ping 10.10.10.111能通。