iscsi远程块存储

1.iscsi概念

1.iSCSI技术是一种由IBM公司研究开发的，是一个供硬件设备使用的可以在IP协议的上层运行SCSI指令集，这种指令集合可以实现在IP网络上运行SCSI协议，使其能够在诸如高速千兆以太网上进行路由选择。iSCSI技术是一种新储存技术，该技术是将现有SCSI接口与以太网络(Ethernet)技术结合，使服务器可与使用IP网络的储存装置互相交换资料。

2.iSCSI：Internet 小型计算机系统接口 （iSCSI：Internet Small Computer System Interface）。

3.Internet 小型计算机系统接口（iSCSI）是一种基于 TCP/IP的协议，用来建立和管理 IP 存储设备、主机和客户机等之间的相互连接，并创建存储区域网络（SAN）。SAN 使得 SCSI 协议应用于高速数据传输网络成为可能，这种传输以数据块级别（block-level）在多个数据存储网络间进行。

4.SCSI 结构基于客户/服务器模式，其通常应用环境是：设备互相靠近，并且这些设备由 SCSI 总线连接。iSCSI 的主要功能是在 TCP/IP 网络上的主机系统（启动器 initiator）和存储设备（目标器 target）之间进行大量数据的封装和可靠传输过程。此外，iSCSI 提供了在 IP 网络封装 SCSI 命令，且运行在 TCP 上。

5.iSCSI(Internet SCSI)支持从客户端(发起端)通过IP向远程服务器上的SCSI存储设备(目标)发送SCSI命令。iSCSI限定名称用于确定发起端和目标,并采用iqn.yyyy-mm.{reverse domain}:label的格式。默认情况下,网络通信是至iSCSI目标上的端口3260/tcp的明文。

iSCSI发起端:需要访问原始SAN存储的客户端。
iSCSI目标:从iSCSI服务器提供的远程硬盘磁盘,或“目标门户”
iSCSI目标门户:通过网络向发起端提供目标的服务器。
IQN:“iSCSI限定名称”。每个发起端和目标需要唯一名称进行标识,最好的做法是使用一个在Internet上可能独一无二的名称。

2.图解

3.iscsi配置

在server端，
我们需要先建立一个逻辑卷，
安装iSCSI目标软件包:
yum install -y targetcli
启动服务:
systemctl start target
进入iSCSI目标交互式配置模式:
targetcli
然后进行设置
具体如下：

1.建立逻辑卷[root@iscsi-server ~]# fdisk /dev/vdb[root@iscsi-server ~]# partprobe[root@iscsi-server ~]# fdisk -l[root@iscsi-server ~]# pvcreate /dev/vdb1[root@iscsi-server ~]# vgcreate vg0 /dev/vdb1[root@iscsi-server ~]# vgdisplay [root@iscsi-server ~]# lvcreate -l 249 -n iscsi_storage vg0[root@iscsi-server ~]# lvs2.安装iSCSI目标软件包:[root@iscsi-server ~]# yum install targetcli -y3.启动服务[root@iscsi-server ~]# systemctl start target[root@iscsi-server ~]# systemctl enable target4.进入iSCSI目标交互式配置模式并进行设置[root@iscsi-server ~]# targetcliWarning: Could not load preferences file /root/.targetcli/prefs.bin.targetcli shell version 2.1.fb34Copyright 2011-2013 by Datera, Inc and others.For help on commands, type 'help'./> lso- / ......................................................................................................................... [...]  o- backstores .............................................................................................................. [...]  | o- block .................................................................................................. [Storage Objects: 0]  | o- fileio ................................................................................................. [Storage Objects: 0]  | o- pscsi .................................................................................................. [Storage Objects: 0]  | o- ramdisk ................................................................................................ [Storage Objects: 0]  o- iscsi ............................................................................................................ [Targets: 0]  o- loopback ......................................................................................................... [Targets: 0]/> /backstores/block create serverlstorage /dev/vg0/iscsi_storage Created block storage object serverlstorage using /dev/vg0/iscsi_storage./> /iscsi create iqn.2017-06.com.example:strage1Created target iqn.2017-06.com.example:strage1.Created TPG 1./> /iscsi/iqn.2017-06.com.example:strage1/tpg1/luns create /backstores/block/serverlstorage   ##关联Created LUN 0./> /iscsi/iqn.2017-06.com.example:strage1/tpg1/acls create iqn.2017-06.com.example:serverlstorage #加keyCreated Node ACL for iqn.2017-06.com.example:serverlstorageCreated mapped LUN 0./> /iscsi/iqn.2017-06.com.example:strage1/tpg1/portals create 172.25.254.10 #开放端口Using default IP port 3260Created network portal 172.25.254.10:3260./> exitGlobal pref auto_save_on_exit=trueLast 10 configs saved in /etc/target/backup.Configuration saved to /etc/target/saveconfig.json5.关闭防火墙[root@iscsi-server ~]# systemctl stop firewalld.service 

在客户端上：

1.安装iSCSI发起端软件包[root@iscsi-client ~]# yum install iscsi-initiator-utils.i686 -y2在/etc/iscsi/initiatorname.iscsi中设置发起端的IQN[root@iscsi-client ~]# vim /etc/iscsi/initiatorname.iscsi InitiatorName=iqn.2017-06.com.example:serverlstorage3.查找iSCSI服务器所提供的iSCSI目标(目标门户)[root@iscsi-client ~]# iscsiadm -m discovery -t st -p 172.25.254.10  172.25.254.10:3260,1 iqn.2017-06.com.example:strage14.登录服务器上的一个或多个iscsi目标[root@iscsi-client ~]# iscsiadm -m node -T iqn.2017-06.com.example:strage1 -p 172.25.254.10 -lLogging in to [iface: default, target: iqn.2017-06.com.example:strage1, portal: 172.25.254.10,3260] (multiple)Login to [iface: default, target: iqn.2017-06.com.example:strage1, portal: 172.25.254.10,3260] successful.此时,可以使用iSCSI磁盘,就好像它是本地连接硬盘驱动器。可以挂载现有文件系统。如果磁盘未格式化,可以通过fdisk进行分区,例如,通过文件系统格式化分区或作为LVM物理卷。[root@iscsi-client ~]# yum install tree -y  ##tree服务可以查看谁访问[root@iscsi-client ~]# tree -C /var/lib/iscsi/  ##进行查看/var/lib/iscsi/├── ifaces├── isns├── nodes│   └── iqn.2017-06.com.example:strage1│       └── 172.25.254.10,3260,1│           └── default├── send_targets│   └── 172.25.254.10,3260│       ├── iqn.2017-06.com.example:strage1,172.25.254.10,3260,1,default -> /var/lib/iscsi/nodes/iqn.2017-06.com.example:strage1/172.25.254.10,3260,1│       └── st_config├── slp└── static10 directories, 2 files[root@iscsi-client ~]# fdisk -l  ##列出可以使用的分区，发现/dev/sdb是远程iscsi的Disk /dev/sda: 8589 MB, 8589934592 bytes, 16777216 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk label type: dosDisk identifier: 0x000954db   Device Boot      Start         End      Blocks   Id  System/dev/sda1   *        2048      411647      204800   83  Linux/dev/sda2          411648    16777215     8182784   8e  Linux LVMDisk /dev/mapper/rhel-root: 7851 MB, 7851737088 bytes, 15335424 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk /dev/mapper/rhel-swap: 524 MB, 524288000 bytes, 1024000 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk /dev/sdb: 1044 MB, 1044381696 bytes, 2039808 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 4194304 bytes[root@iscsi-client ~]# fdisk /dev/sdbWelcome to fdisk (util-linux 2.23.2).Changes will remain in memory only, until you decide to write them.Be careful before using the write command.Device does not contain a recognized partition tableBuilding a new DOS disklabel with disk identifier 0x58c3c577.Command (m for help): nPartition type:   p   primary (0 primary, 0 extended, 4 free)   e   extendedSelect (default p): Using default response pPartition number (1-4, default 1): First sector (8192-2039807, default 8192): Using default value 8192Last sector, +sectors or +size{K,M,G} (8192-2039807, default 2039807): Using default value 2039807Partition 1 of type Linux and of size 992 MiB is setCommand (m for help): wqThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.[root@iscsi-client ~]# partprobe[root@iscsi-client ~]# mkfs.xfs /dev/sdb1meta-data=/dev/sdb1              isize=256    agcount=8, agsize=31744 blks         =                       sectsz=512   attr=2, projid32bit=1         =                       crc=0data     =                       bsize=4096   blocks=253952, imaxpct=25         =                       sunit=0      swidth=0 blksnaming   =version 2              bsize=4096   ascii-ci=0 ftype=0log      =internal log           bsize=4096   blocks=853, version=2         =                       sectsz=512   sunit=0 blks, lazy-count=1realtime =none                   extsz=4096   blocks=0, rtextents=0[root@iscsi-client ~]# dfFilesystem            1K-blocks    Used Available Use% Mounted on/dev/mapper/rhel-root   7657472 3134616   4522856  41% /devtmpfs                 499700       0    499700   0% /devtmpfs                    508996     140    508856   1% /dev/shmtmpfs                    508996    7060    501936   2% /runtmpfs                    508996       0    508996   0% /sys/fs/cgroup/dev/sda1                201388  107392     93996  54% /boot[root@iscsi-client ~]# mount /dev/sdb1 /mnt[root@iscsi-client ~]# cd /mnt[root@iscsi-client mnt]# ls

3.在/etc/fstab中永久挂载文件系统

1. 使用blkid确定文件系统UUID并使用UUID挂载,而不是/dev/sd*设备名称。(每次引导时显示的设备名称都不同,具体取决于iSCSI设备通过网络进行响应的顺序。如果按设备名称挂载,这会导致使用错误的设备。)
2. 在/etc/fstab中使用_netdev作为挂载选项。(这将确保客户端不会尝试挂载文件系统,直至启用联网。否则,在引导时系统将出错。)
3. 确保iscsi服务在引导时启动。
   [root@iscsi-client ~]# blkid
   /dev/sda1: UUID=”406fa427-bc71-49eb-a840-422189f08917” TYPE=”xfs”
   /dev/sda2: UUID=”KtTqpC-XtVz-4Ftl-XxBh-EHKU-oayK-EhxTpV” TYPE=”LVM2_member”
   /dev/mapper/rhel-root: UUID=”6e1e47dd-f268-4572-90eb-d157e04e969e” TYPE=”xfs”
   /dev/mapper/rhel-swap: UUID=”3fd05b1b-50d3-4432-a709-696f62811c66” TYPE=”swap”
   /dev/sdb1: UUID=”2f04d8e5-102e-40e7-8ed9-a37f463cecab” TYPE=”xfs”
   [root@iscsi-client ~]# vim /etc/fstab
   UUID=2f04d8e5-102e-40e7-8ed9-a37f463cecab /mnt xfs defaults,_netdev 0 0
   可以reboot查看一下是否正确。

4.中断使用iSCSI目标

确保没有使用目标所提供的任何设备。
确保从/etc/fstab等位置中删除使用目标的所有永久挂载。
登出iSCSI目标,以暂时断开连接。

 iscsiadm -m node -T iqn.2010-09.com.example:rdisks.demo -p 192.168.0.254 -u

删除iSCSI目标的本地记录,以永久断开连接。

 iscsiadm -m node -T iqn.2010-09.com.example:rdisks.demo -p 192.168.0.254 -o delete

具体如下：
在客户端

[root@iscsi-client ~]# umount /mnt[root@iscsi-client ~]# vim /etc/fstab [root@iscsi-client ~]# iscsiadm -m node -T iqn.2017-06.com.example:strage1 -p 172.25.254.10 -uLogging out of session [sid: 1, target: iqn.2017-06.com.example:strage1, portal: 172.25.254.10,3260]Logout of [sid: 1, target: iqn.2017-06.com.example:strage1, portal: 172.25.254.10,3260] successful.[root@iscsi-client ~]# iscsiadm -m node -T iqn.2017-06.com.example:strage1 -p 172.25.254.10 -o delete

在server端

[root@iscsi-server ~]# targetcli targetcli shell version 2.1.fb34Copyright 2011-2013 by Datera, Inc and others.For help on commands, type 'help'./> helpGENERALITIES============This is a shell in which you can create, delete and configureconfiguration objects.The available commands depend on the current path or targetpath you want to run a command in: different path havedifferent sets of available commands, i.e. a path pointing atan iscsi target will not have the same availaible commands as,say, a path pointing at a storage object.The prompt that starts each command line indicates yourcurrent path. Alternatively (useful if the prompt displaysan abbreviated path to save space), you can run thepwd command to display the complete current path.Navigating the tree is done using the cd command. Withoutany argument, cd will present you wil the full objectstree. Just use arrows to select the destination path, andenter will get you there. Please try help cd for navigationtips.COMMAND SYNTAX==============Commands are built using the following syntax:[TARGET_PATH] COMMAND_NAME [OPTIONS]The TARGET_PATH indicates the path to run the command from.If ommited, the command will be run from your current path.The OPTIONS depend on the command. Please use helpCOMMAND to get more information.AVAILABLE COMMANDS==================The following commands are available in thecurrent path:  - bookmarks action [bookmark]   - cd [path]   - clearconfig [confirm]   - exit   - get [group] [parameter...]   - help [topic]   - ls [path] [depth]   - pwd   - refresh   - restoreconfig [savefile] [clear_existing]   - saveconfig [savefile]   - sessions [action] [sid]   - set [group] [parameter=value...]   - status   - version /> clearconfigAs a precaution, confirm=True needs to be set/> clearconfig confirm=True  ##清空之前所有设置All configuration cleared/> ls   ##列出o- / ......................................................................................................................... [...]  o- backstores .............................................................................................................. [...]  | o- block .................................................................................................. [Storage Objects: 0]  | o- fileio ................................................................................................. [Storage Objects: 0]  | o- pscsi .................................................................................................. [Storage Objects: 0]  | o- ramdisk ................................................................................................ [Storage Objects: 0]  o- iscsi ............................................................................................................ [Targets: 0]  o- loopback ......................................................................................................... [Targets: 0]/>