Re: Multipathing with SCSI on FCP
It looks like you only have one real FCP path to that LUN, correct? Devices 601 and 603 are probably on the same chpid, and you are accessing lun 0001 through the same WWPN. ( BTW - is that LUN number right? If it's on a Shark device, every LUN I've ever seen has been in the 5000-5FFF range. But if you've tried accessing it with a single path and it works, then never mind... ) Also - I think you may run into a problem accessing the same LUN more than once through a single WWPN. You may have to have more than one WWPN to do multipathing - otherwise the zfcp driver will throw that error message about accessing the same LUN more than once. Basically - this means you probably need to have more than one adapter in your Shark ( Or other back end dasd ) defined as open systems SCSI FCP. Also - at the moment, Linux on Z needs to have a FCP switch in between the Z and the back end storage system. Direct connect is not supported with FCP - it needs the switch's nameserver to work. I'm not sure what benifit you'd get by having multiple paths to a single LUN - I can't say for sure if you get multiple IO's in flight at one time to the one device ( like with PAV ) or not. The real benifit of SCSI multipathing from my understanding is from having multiple LUNs accessed by multiple paths so you can drive more than one device simultaneously. Well - that and the no single point of failure. Anyway - heres what I would suggest to do Multipathing the way I understand it: Device 701 is on a different FCP chpid than 601, and I've changed the WWPN that 701 talks to to be different ( ...c7...) than the one on adapter 601 ( ...c5...) I've also changed the LUN number to be in the 5000-5FFF range ( to match what I know about zLinux's SCSI needs related Sharkness ) and and added another lun to show how you'd do more than one. rmmod zfcp modprobe qdio modprobe scsi_mod insmod zfcp map=\ 0x601 0x01:0x500060e802c5d603 0x00:0x5001;\ 0x601 0x01:0x500060e802c5d603 0x01:0x5002;\ 0x701 0x01:0x500060e802c7d603 0x00:0x5001;\ 0x701 0x01:0x500060e802c7d603 0x01:0x5002 modprobe sd_mod modprobe st This should get you up with multiple paths to the devices. You'll end up with something like /dev/sda pointing to the first path to 5001, /dev/sdb pointing to the first path to 5002, /dev/sdc pointing to the second path to 5001, and /dev/sdd pointing to the second path to 5002. Look at the startup messages carefully, or at the dmesg output to verify this. Then you'll probably want to use LVM ( on SuSE ) or mdadm ( on RedHat ) to setup consolidated devices to access your disks. With LVM just create your logical volume on the first set of paths to the devices as you normally would, then use the pvpath command to enable the extra paths. With mdadm - sorry - I can't help there. I've never used that tool. If you need directions on how to do multipathing on RedHat contact me off list and I'll forward you to the right people. Jay Brenneman Linux Test and Integration Center T/L: 295 - 7745 Extern: 845 - 435 - 7745 [EMAIL PROTECTED] Herve Bonvin [EMAIL PROTECTED] Sent by: Linux on 390 Port [EMAIL PROTECTED] 03/27/2004 02:00 AM Please respond to Linux on 390 Port To: [EMAIL PROTECTED] cc: Subject:Multipathing with SCSI on FCP Hi, Has anybody tried multipathing with SCSI on FCP ? I tried to defined 2 path to the same WWPN/LUN with 2 differents devices numbers on the same linux and I get the following error : Mar 27 07:59:04 sbe12127 kernel: Loaded 10178 symbols from /boot/System.map-2.4.21-83-default. Mar 27 07:59:04 sbe12127 kernel: Symbols match kernel version 2.4.21. Mar 27 07:59:04 sbe12127 kernel: Loaded 516 symbols from 18 modules. Mar 27 07:59:04 sbe12127 kernel: scsi : 0 hosts left. Mar 27 07:59:04 sbe12127 kernel: zfcp: zfcp_module_init: driver version 0x3009d Mar 27 07:59:05 sbe12127 kernel: zfcp: FSF: zfcp_fsf_open_unit_handler: FSF_LUN_IN_USE Mar 27 07:59:05 sbe12127 kernel: zfcp: FSF: zfcp_fsf_open_unit_handler: error: FCP_LUN 0x0001 at the remote port with WWPN 0x500060e802c5d603 connected to the adapter with devno 0x0601 is already owned by another operating system instance (LPAR or VM guest) Mar 27 07:59:05 sbe12127 kernel: zfcp: FSF: zfcp_fsf_open_unit_handler: Additional sense data is presented: Mar 27 07:59:05 sbe12127 kernel: 00bb0f03 Mar 27 07:59:05 sbe12127 kernel: zfcp: ERP: zfcp_erp_unit_failed: Unit recovery failed on the unit with FCP_LUN 0x0001 connected to the port with WWPN 0x500060e802c5d603 at the adapter with devno 0x0601. Mar 27 07:59:05 sbe12127 kernel: scsi0 : zfcp Mar 27 07:59:05 sbe12127 kernel: scsi1 : zfcp here the scripts : rmmod zfcp modprobe qdio modprobe scsi_mod insmod zfcp map=\ 0x601 0x01:0x500060e802c5d603 0x00:0x0001;\ 0x603 0x02:0x500060e802c5d603 0x01:0x0001 modprobe sd_mod modprobe st Thanks in advance, Herve Bonvin
RE : Multipathing with SCSI on FCP
Jay, Thank you very much for your answer. It really helps. For the LUN Number, we don't use Shark but Hitachi systems though a SAN. Your right, Devices 601 and 603 are on the same chpid. I did for test purpose, the second chpid is not ready for the moment. I thougth it was possible to test the multipathing functionality by using 2 differents devices. I will have to wait for our installation to be finished before going on with my tests. Best Regards, Herve Bonvin -Original Message- From: Linux on 390 Port [mailto:[EMAIL PROTECTED] On Behalf Of Robert J Brenneman Sent: Sunday, March 28, 2004 8:00 PM To: [EMAIL PROTECTED] Subject: Re: Multipathing with SCSI on FCP It looks like you only have one real FCP path to that LUN, correct? Devices 601 and 603 are probably on the same chpid, and you are accessing lun 0001 through the same WWPN. ( BTW - is that LUN number right? If it's on a Shark device, every LUN I've ever seen has been in the 5000-5FFF range. But if you've tried accessing it with a single path and it works, then never mind... ) Also - I think you may run into a problem accessing the same LUN more than once through a single WWPN. You may have to have more than one WWPN to do multipathing - otherwise the zfcp driver will throw that error message about accessing the same LUN more than once. Basically - this means you probably need to have more than one adapter in your Shark ( Or other back end dasd ) defined as open systems SCSI FCP. Also - at the moment, Linux on Z needs to have a FCP switch in between the Z and the back end storage system. Direct connect is not supported with FCP - it needs the switch's nameserver to work. I'm not sure what benifit you'd get by having multiple paths to a single LUN - I can't say for sure if you get multiple IO's in flight at one time to the one device ( like with PAV ) or not. The real benifit of SCSI multipathing from my understanding is from having multiple LUNs accessed by multiple paths so you can drive more than one device simultaneously. Well - that and the no single point of failure. Anyway - heres what I would suggest to do Multipathing the way I understand it: Device 701 is on a different FCP chpid than 601, and I've changed the WWPN that 701 talks to to be different ( ...c7...) than the one on adapter 601 ( ...c5...) I've also changed the LUN number to be in the 5000-5FFF range ( to match what I know about zLinux's SCSI needs related Sharkness ) and and added another lun to show how you'd do more than one. rmmod zfcp modprobe qdio modprobe scsi_mod insmod zfcp map=\ 0x601 0x01:0x500060e802c5d603 0x00:0x5001;\ 0x601 0x01:0x500060e802c5d603 0x01:0x5002;\ 0x701 0x01:0x500060e802c7d603 0x00:0x5001;\ 0x701 0x01:0x500060e802c7d603 0x01:0x5002 modprobe sd_mod modprobe st This should get you up with multiple paths to the devices. You'll end up with something like /dev/sda pointing to the first path to 5001, /dev/sdb pointing to the first path to 5002, /dev/sdc pointing to the second path to 5001, and /dev/sdd pointing to the second path to 5002. Look at the startup messages carefully, or at the dmesg output to verify this. Then you'll probably want to use LVM ( on SuSE ) or mdadm ( on RedHat ) to setup consolidated devices to access your disks. With LVM just create your logical volume on the first set of paths to the devices as you normally would, then use the pvpath command to enable the extra paths. With mdadm - sorry - I can't help there. I've never used that tool. If you need directions on how to do multipathing on RedHat contact me off list and I'll forward you to the right people. Jay Brenneman Linux Test and Integration Center T/L: 295 - 7745 Extern: 845 - 435 - 7745 [EMAIL PROTECTED] Herve Bonvin [EMAIL PROTECTED] Sent by: Linux on 390 Port [EMAIL PROTECTED] 03/27/2004 02:00 AM Please respond to Linux on 390 Port To: [EMAIL PROTECTED] cc: Subject:Multipathing with SCSI on FCP Hi, Has anybody tried multipathing with SCSI on FCP ? I tried to defined 2 path to the same WWPN/LUN with 2 differents devices numbers on the same linux and I get the following error : Mar 27 07:59:04 sbe12127 kernel: Loaded 10178 symbols from /boot/System.map-2.4.21-83-default. Mar 27 07:59:04 sbe12127 kernel: Symbols match kernel version 2.4.21. Mar 27 07:59:04 sbe12127 kernel: Loaded 516 symbols from 18 modules. Mar 27 07:59:04 sbe12127 kernel: scsi : 0 hosts left. Mar 27 07:59:04 sbe12127 kernel: zfcp: zfcp_module_init: driver version 0x3009d Mar 27 07:59:05 sbe12127 kernel: zfcp: FSF: zfcp_fsf_open_unit_handler: FSF_LUN_IN_USE Mar 27 07:59:05 sbe12127 kernel: zfcp: FSF: zfcp_fsf_open_unit_handler: error: FCP_LUN 0x0001 at the remote port with WWPN 0x500060e802c5d603 connected to the adapter with devno 0x0601 is already owned by another operating system instance (LPAR or VM guest) Mar 27 07:59:05 sbe12127 kernel: zfcp: FSF
Multipathing with SCSI on FCP
Hi, Has anybody tried multipathing with SCSI on FCP ? I tried to defined 2 path to the same WWPN/LUN with 2 differents devices numbers on the same linux and I get the following error : Mar 27 07:59:04 sbe12127 kernel: Loaded 10178 symbols from /boot/System.map-2.4.21-83-default. Mar 27 07:59:04 sbe12127 kernel: Symbols match kernel version 2.4.21. Mar 27 07:59:04 sbe12127 kernel: Loaded 516 symbols from 18 modules. Mar 27 07:59:04 sbe12127 kernel: scsi : 0 hosts left. Mar 27 07:59:04 sbe12127 kernel: zfcp: zfcp_module_init: driver version 0x3009d Mar 27 07:59:05 sbe12127 kernel: zfcp: FSF: zfcp_fsf_open_unit_handler: FSF_LUN_IN_USE Mar 27 07:59:05 sbe12127 kernel: zfcp: FSF: zfcp_fsf_open_unit_handler: error: FCP_LUN 0x0001 at the remote port with WWPN 0x500060e802c5d603 connected to the adapter with devno 0x0601 is already owned by another operating system instance (LPAR or VM guest) Mar 27 07:59:05 sbe12127 kernel: zfcp: FSF: zfcp_fsf_open_unit_handler: Additional sense data is presented: Mar 27 07:59:05 sbe12127 kernel: 00bb0f03 Mar 27 07:59:05 sbe12127 kernel: zfcp: ERP: zfcp_erp_unit_failed: Unit recovery failed on the unit with FCP_LUN 0x0001 connected to the port with WWPN 0x500060e802c5d603 at the adapter with devno 0x0601. Mar 27 07:59:05 sbe12127 kernel: scsi0 : zfcp Mar 27 07:59:05 sbe12127 kernel: scsi1 : zfcp here the scripts : rmmod zfcp modprobe qdio modprobe scsi_mod insmod zfcp map=\ 0x601 0x01:0x500060e802c5d603 0x00:0x0001;\ 0x603 0x02:0x500060e802c5d603 0x01:0x0001 modprobe sd_mod modprobe st Thanks in advance, Herve Bonvin -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
