NATIVE SCSI MULTIPATH: This series introduces native SCSI multipath support. It is intended as an alternative to dm-mpath.
This support aims to provide a multipath-enabled SCSI block device/ gendisk. For a SCSI device to support native multipath, either of the following conditions must be satisfied: a. unique ID in VPD page 83 and ALUA support and scsi_multipath modparam enabled b. unique ID in VPD page 83 and scsi_multipath_always modparam enabled This series relies on reading sdev->access_state to get path information. This path information would be provided by ALUA. ALUA support which does not rely on device handlers has already been discussed at https://lore.kernel.org/linux-scsi/[email protected]/ and support will be added in the next phase. New classes of devices are added: - scsi_mpath_device - scsi_mpath_disk These are required since a multipath scsi_device has no common scsi host. An example of the sysfs files and directories for these new classes is as follows: $ ls -l /sys/class/scsi_mpath_device/scsi_mpath_device0/ total 0 -rw-r--r-- 1 root root 4096 Feb 25 11:59 iopolicy drwxr-xr-x 2 root root 0 Feb 25 11:59 multipath drwxr-xr-x 2 root root 0 Feb 25 11:59 power lrwxrwxrwx 1 root root 0 Feb 25 11:59 subsystem -> ../../../../class/scsi_mpath_device -rw-r--r-- 1 root root 4096 Feb 25 11:58 uevent -r--r--r-- 1 root root 4096 Feb 25 11:59 vpd_id $ ls -l /sys/class/scsi_mpath_device/scsi_mpath_device0/multipath/ total 0 lrwxrwxrwx 1 root root 0 Feb 25 11:59 8:0:0:0 -> ../../../../platform/host8/session1/target8:0:0/8:0:0:0 lrwxrwxrwx 1 root root 0 Feb 25 11:59 9:0:0:0 -> ../../../../platform/host9/session2/target9:0:0/9:0:0:0 $ cat /sys/class/scsi_mpath_device/scsi_mpath_device0/vpd_id naa.600140505200a986f0043c9afa1fd077 $ cat /sys/class/scsi_mpath_device/scsi_mpath_device0/iopolicy numa $ $ ls -l /sys/class/scsi_mpath_disk/scsi_mpath_disk0/ total 0 drwxr-xr-x 2 root root 0 Feb 25 12:00 power drwxr-xr-x 11 root root 0 Feb 25 11:58 sdc lrwxrwxrwx 1 root root 0 Feb 25 11:58 subsystem -> ../../../../class/scsi_mpath_disk -rw-r--r-- 1 root root 4096 Feb 25 11:58 uevent $ ls -l /sys/class/scsi_mpath_disk/scsi_mpath_disk0/sdc/multipath/ total 0 lrwxrwxrwx 1 root root 0 Feb 25 12:00 sdc:0 -> ../../../../../platform/host8/session1/target8:0:0/8:0:0:0/block/sdc:0 lrwxrwxrwx 1 root root 0 Feb 25 12:00 sdc:1 -> ../../../../../platform/host9/session2/target9:0:0/9:0:0:0/block/sdc:1 $ ls -l /dev/sdc brw-rw---- 1 root disk 8, 32 Feb 25 11:58 /dev/sdc The scsi_device and scsi_disk classes otherwise remain unmodified. However, the per-path block device is hidden in /dev/. Furthermore, multipathed block devices have a new naming scheme, sdX:Y, where X is the scsi multipath device index and Y is the path index. No multipath sg support is added. We still have a per-path sg device. Since the SCSI block device is multipath enabled, we can access multipathed scsi_ioctl() through that block device. For failover, we take the approach of cloning bio's and re-submitting them in full (for failover errors). LIBMULTIPATH: This series introduces also libmultipath. It is essentially a refactoring of NVME multipath support, so we can have a common library to also support native SCSI multipath. Much of the code is taken directly from the NVMe multipath code. However, NVMe specifics are removed. A template structure is provided so the driver -may provide callbacks for driver specifics, like ANA support for NVMe. Important new structures introduced include: - mpath_head These contain much of the multipath-specific functionality from nvme_ns_head, including a pointer to the gendisk structure and a path SRCU-based array. - mpath_device This is the per-path structure, and contains much the same multipath-specific functionality in nvme_ns libmultipath provides functionality for path management, path selection, data path, and failover handling. Full series also available at https://github.com/johnpgarry/linux/commits/scsi-multipath-v7.2-v4/ Differences to v3: - drop for now any libmultipath specific NVMe support, like cdev and PR support - deal with issues sashiko/codex found in libmultipath: - add checks in mpath_add_device() for setting pointers - add mpath_delete_device() -> mpath_synchronize() call - add numa iopolicy check for disabled path - fix trace in mpath_bdev_submit_bio() - fix race in mpath_head_uninit() / mpath_put_head() - add ioctl check in mpath_bdev_ioctl() - fix sync ordering in mpath_remove_disk() - drop unused arg in mpath_iopolicy_store() - fix mpath_delayed_removal_secs_show() print format - memset mpath_head in mpath_head_init() - deal with issues sashiko/codex found in scsi-multipath support: - change TPG check in scsi_mpath_dev_alloc() - use scsi_multipath_param_set() -> __sysfs_match_string() - add scsi_mpath_revalidate_paths() - add sdev check in scsi_mpath_clone_end_io() - fix sdev->access_state masking in scsi_mpath_is_disabled() and scsi_mpath_is_optimized() - properly set default iopolicy in scsi_mpath_alloc_head() - fix error path in scsi_mpath_alloc_head() - fix possible race in scsi_mpath_find_head() - fix scsi_mpath_{start, end}_request() for partitions - fix scsi_end_request() -> scsi_mpath_end_request() order to deal with partial completions - don't call __scsi_internal_device_block_nowait() -> scsi_mpath_dev_clear_path() - call scsi_internal_device_unblock() -> scsi_mpath_revalidate_paths() - fix sd.c sys attr references to sd_mpath_disk - handle delayed removal race in sd_mpath_remove_head() - default index = -1 in sd_probe() John Garry (28): libmultipath: Add initial framework libmultipath: Add basic gendisk support libmultipath: Add path selection support libmultipath: Add bio handling libmultipath: Add support for mpath_device management libmultipath: Add delayed removal support libmultipath: Add sysfs helpers libmultipath: Add mpath_bdev_report_zones() libmultipath: Add support for block device IOCTL libmultipath: Add mpath_bdev_getgeo() libmultipath: Add mpath_bdev_get_unique_id() scsi-multipath: introduce basic SCSI device support scsi-multipath: introduce scsi_device head structure scsi-multipath: provide sysfs link from to scsi_device scsi-multipath: support iopolicy scsi-multipath: clone each bio scsi-multipath: clear path when device is blocked scsi-multipath: revalidate paths upon device unblock scsi-multipath: failover handling scsi-multipath: provide callbacks for path state scsi-multipath: add scsi_mpath_{start,end}_request() scsi-multipath: add delayed disk removal support scsi: sd: add multipath disk class scsi: sd: add multipath disk attr groups scsi: sd: support multipath disk scsi: sd: add mpath_dev file scsi: sd: add mpath_numa_nodes dev attribute scsi: sd: add mpath_queue_depth dev attribute drivers/scsi/Kconfig | 10 + drivers/scsi/Makefile | 1 + drivers/scsi/scsi.c | 8 +- drivers/scsi/scsi_lib.c | 16 +- drivers/scsi/scsi_multipath.c | 640 +++++++++++++++++++++++ drivers/scsi/scsi_scan.c | 4 + drivers/scsi/scsi_sysfs.c | 10 + drivers/scsi/sd.c | 585 ++++++++++++++++++++- drivers/scsi/sd.h | 3 + include/linux/multipath.h | 158 ++++++ include/scsi/scsi_cmnd.h | 6 + include/scsi/scsi_device.h | 2 + include/scsi/scsi_driver.h | 4 + include/scsi/scsi_host.h | 4 + include/scsi/scsi_multipath.h | 119 +++++ lib/Kconfig | 6 + lib/Makefile | 2 + lib/multipath.c | 960 ++++++++++++++++++++++++++++++++++ 18 files changed, 2514 insertions(+), 24 deletions(-) create mode 100644 drivers/scsi/scsi_multipath.c create mode 100644 include/linux/multipath.h create mode 100644 include/scsi/scsi_multipath.h create mode 100644 lib/multipath.c -- 2.43.7
