SCSI热拔插

转自：http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Online_Storage_Reconfiguration_Guide/removing_devices.html

5. Removing a Storage Device

Before removing access to the storage device itself, it is advisable to back up data from the device first. Afterwards, flush I/O and remove all operating system references to the device (as described below). If the device uses multipathing, then do this for the multipath "pseudo device" (Section 4.1, “WWID”) and each of the identifiers that represent a path to the device. If you are only removing a path to a multipath device, and other paths will remain, then the procedure is simpler, as described inSection 7, “Adding a Storage Device or Path”.

Removal of a storage device is not recommended when the system is under memory pressure, since the I/O flush will add to the load. To determine the level of memory pressure, run the commandvmstat 1 100; device removal is not recommended if:

• Free memory is less than 5% of the total memory in more than 10 samples per 100 (the commandfree can also be used to display the total memory).
• Swapping is active (non-zero si andso columns in the vmstat output).

The general procedure for removing all access to a device is as follows:

Procedure 1. Ensuring a Clean Device Removal

1. Close all users of the device and backup device data as needed.
2. Use umount to unmount any file systems that mounted the device.
3. Remove the device from any md and LVM volume using it. If the device is a member of an LVM Volume group, then it may be necessary to move data off the device using thepvmove command, then use the vgreduce command to remove the physical volume, and (optionally)pvremove to remove the LVM metadata from the disk.
4. If the device uses multipathing, run multipath -l and note all the paths to the device. Afterwards, remove the multipathed device usingmultipath -f device.
5. Run blockdev –flushbufs device to flush any outstanding I/O to all paths to the device. This is particularly important for raw devices, where there is noumount or vgreduce operation to cause an I/O flush.
6. Remove any reference to the device's path-based name, like /dev/sd, /dev/disk/by-path or the major:minor number, in applications, scripts, or utilities on the system. This is important in ensuring that different devices added in the future will not be mistaken for the current device.
7. Finally, remove each path to the device from the SCSI subsystem. To do so, use the commandecho 1 > /sys/block/device-name/device/delete wheredevice-name may besde, for example.
   Another variation of this operation is echo 1 > /sys/class/scsi_device/h:c:t:l/device/delete, where h is the HBA number,c is the channel on the HBA,t is the SCSI target ID, andl is the LUN.

   Note

   The older form of these commands, echo "scsi remove-single-device 0 0 0 0" > /proc/scsi/scsi, is deprecated.

You can determine the device-name, HBA number, HBA channel, SCSI target ID and LUN for a device from various commands, such aslsscsi, scsi_id, multipath -l, and ls -l /dev/disk/by-*.

After performing Procedure 1, “Ensuring a Clean Device Removal”, a device can be physically removed safely from a running system. It is not necessary to stop I/O to other devices while doing so.

Other procedures, such as the physical removal of the device, followed by a rescan of the SCSI bus (as described inSection 9, “Scanning Storage Interconnects”) to cause the operating system state to be updated to reflect the change, are not recommended. This will cause delays due to I/O timeouts, and devices may be removed unexpectedly. If it is necessary to perform a rescan of an interconnect, it must be done while I/O is paused, as described inSection 9, “Scanning Storage Interconnects”.

转自：http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Online_Storage_Reconfiguration_Guide/removing_path-to-storage-device.html

6. Removing a Path to a Storage Device

If you are removing a path to a device that uses multipathing (without affecting other paths to the device), then the general procedure is as follows:

Procedure 2. Removing a Path to a Storage Device

1. Remove any reference to the device's path-based name, like /dev/sd or /dev/disk/by-path or the major:minor number, in applications, scripts, or utilities on the system. This is important in ensuring that different devices added in the future will not be mistaken for the current device.
2. Take the path offline using echo offline > /sys/block/sda/device/state.
   This will cause any subsequent IO sent to the device on this path to be failed immediately.Device-mapper-multipath will continue to use the remaining paths to the device.
3. Remove the path from the SCSI subsystem. To do so, use the commandecho 1 > /sys/block/device-name/device/delete wheredevice-name may besde, for example (as described in Procedure 1, “Ensuring a Clean Device Removal”).

After performing Procedure 2, “Removing a Path to a Storage Device”, the path can be safely removed from the running system. It is not necessary to stop I/O while this is done, asdevice-mapper-multipath will re-route I/O to remaining paths according to the configured path grouping and failover policies.

Other procedures, such as the physical removal of the cable, followed by a rescan of the SCSI bus to cause the operating system state to be updated to reflect the change, are not recommended. This will cause delays due to I/O timeouts, and devices may be removed unexpectedly. If it is necessary to perform a rescan of an interconnect, it must be done while I/O is paused, as described inSection 9, “Scanning Storage Interconnects”.

转自：http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Online_Storage_Reconfiguration_Guide/adding_storage-device-or-path.html

7. Adding a Storage Device or Path

When adding a device, be aware that the path-based device name (/dev/sd name,major:minor number, and /dev/disk/by-path name, for example) the system assigns to the new device may have been previously in use by a device that has since been removed. As such, ensure that all old references to the path-based device name have been removed. Otherwise, the new device may be mistaken for the old device.

The first step in adding a storage device or path is to physically enable access to the new storage device, or a new path to an existing device. This is done using vendor-specific commands at the Fibre Channel or iSCSI storage server. When doing so, note the LUN value for the new storage that will be presented to your host. If the storage server is Fibre Channel, also take note of theWorld Wide Node Name (WWNN) of the storage server, and determine whether there is a single WWNN for all ports on the storage server. If this is not the case, note theWorld Wide Port Name (WWPN) for each port that will be used to access the new LUN.

Next, make the operating system aware of the new storage device, or path to an existing device. The recommended command to use is:

echo "c t l" > /sys/class/scsi_host/hosth/scan

In the previous command, h is the HBA number,c is the channel on the HBA,t is the SCSI target ID, andl is the LUN.

Note

The older form of this command, echo "scsi add-single-device 0 0 0 0" > /proc/scsi/scsi, is deprecated.

For Fibre Channel storage servers that implement a single WWNN for all ports, you can determine the correcth,c,andt values (i.e. HBA number, HBA channel, and SCSI target ID) by searching for the WWNN insysfs. For example, if the WWNN of the storage server is0x5006016090203181, use:

grep 5006016090203181 /sys/class/fc_transport/*/node_name

This should display output similar to the following:

/sys/class/fc_transport/target5:0:2/node_name:0x5006016090203181 /sys/class/fc_transport/target5:0:3/node_name:0x5006016090203181 /sys/class/fc_transport/target6:0:2/node_name:0x5006016090203181 /sys/class/fc_transport/target6:0:3/node_name:0x5006016090203181

This indicates there are four Fibre Channel routes to this target (two single-channel HBAs, each leading to two storage ports). Assuming a LUN value is56, then the following command will configure the first path:

echo "0 2 56" > /sys/class/scsi_host/host5/scan

This must be done for each path to the new device.

For Fibre Channel storage servers that do not implement a single WWNN for all ports, you can determine the correct HBA number, HBA channel, and SCSI target ID by searching for each of the WWPNs insysfs.

Another way to determine the HBA number, HBA channel, and SCSI target ID is to refer to another device that is already configured on the same path as the new device. This can be done with various commands, such aslsscsi, scsi_id, multipath -l, and ls -l /dev/disk/by-*. This information, plus the LUN number of the new device, can be used as shown above to probe and configure that path to the new device.

After adding all the SCSI paths to the device, execute the multipath command, and check to see that the device has been properly configured. At this point, the device can be added tomd, LVM, mkfs, or mount, for example.

If the steps above are followed, then a device can safely be added to a running system. It is not necessary to stop I/O to other devices while this is done. Other procedures involving a rescan (or a reset) of the SCSI bus, which cause the operating system to update its state to reflect the current device connectivity, are not recommended while storage I/O is in progress.