Re: Problems with SCSI-over-FCP after machine upgrade
We have fixed this problem but made more than one change so we do not know the cause - it would be nice to know who to blame ! Thanks for all the advice. I followed up Scott's suggestion of creating an exec to delete the EDEV' s paths and to do SET EDEV CLEAR so that I could vary the CHPIDs offline. (I found that to redefine the EDEVs I had to use 'SET EDEV ' providing the Type, Attributes, FCP_dev, wwpn and LUN without the text 'ADD PATH' for the first path and with the text 'add path' for the other paths). There is no zoning or LUN masking involved and the NPIV wwpns had not changed (the physical NPIVs had changed - they contains the new PCHIPID ids). We had already tried switching everything in the paths off/on. The changes which we made to resolve the problem were: - switch firmware was upgraded from 6.1.1a to 6.3.0d . Neither had been qualified for zBC12. (We later upgraded to 7.0.0d)- removed the link between the two SAN switches. The SAN Volume Controller documentation says that the fibre networks should be independent. When I disconnected the link I found that one of the SFPs came out too - it had not 'clicked' into place.- cleaned the z12 end of the fibre links. All of the zlinuxes then came up and stayed up, except for some which relied on the ISL being in place. ie the LOADDEV parameters (for loading from chpid 18) specified an SVC wwpn which was now accessible only from the other CHPID, which was easily fixed. Maybe the problem was due to FCP channel microcode in z12 (or z114/196). Or maybe there were some fabric errors which were exacerbated by the faster z12 IFLs. Keith On Monday, 8 June 2015, 15:39, Raymond Higgs rayhi...@us.ibm.com wrote: Hi Keith, Please check zoning and lun masking. I think the SVC management interface calls them hosts. If youdidn't do an MES upgrade, then the WWPNs of your channels changed. Regards, Ray Higgs System z FCP Firmware Development Bld. 706, B42 2455 South Road Poughkeepsie, NY 12601 (845) 435-8666, T/L 295-8666 rayhi...@us.ibm.com From: Keith Gooding kw...@yahoo.co.uk To: LINUX-390@VM.MARIST.EDU Date: 06/04/2015 04:23 PM Subject: Problems withSCSI-over-FCP after machine upgrade Sent by: Linux on 390Port LINUX-390@VM.MARIST.EDU This may not be the proper forum but maybe someonecan help. We have a small number of linux systems (32) under z/VM 6.3 whichuse SCSI connections to LUNs on a SAN Volume Controller via a couple ofIBM SAN24B switches (the equivalent of Brocade 300). There are also somesystems which use EDEVs on the same SVC. This had worked on z10 BC forabout 5 years without problems. Last week the z10 was upgraded to a zBC12, retaining the same FICON cards(4Gbs), but not necessarily associated with the same CHPIDs. Since thena number of the LUN connections have been 'lost', cauing linux systemsto fail. SCSIDISC displays eg HCPRXS975I Virtual FCP device 1A05ignored because the adapter was not able to connect to the fibre channelnetwork. It is then not possible to rebot the linux system. Restarting 'everything' - ie SVC nodes, SAN switches, CHIPD vary off/on- cleared the problem for a while. Any ideas where to start looking ?. I have discovered that we have 32 FCP subchannels defined on the CHPID (but highest used unit addressis 1f, and there are only about a dozen in use). Also the switch has notbeen 'qualified' for use on z12 (but it appears that it was not qualifiedfor z10 either). Any advice greatly appreciated ! -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 orvisit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: Problems with SCSI-over-FCP after machine upgrade
Use SET EDEV commands to delete all the paths defined over the FCP subchannels... Then use SET EDEV to define them all back when done.. A simple EXEC to do one or the other or both will save your fingers if you need to do this more then once. Scott Rohling On Thu, Jun 4, 2015 at 2:07 PM, kwg kw...@yahoo.co.uk wrote: There is no zoning. These are all development systems used only by the z/vm and its linux guests. It was a machine upgrade rather than a new machine and the NPIVs for the z/vm systems were preserved. I did check them and I will check again but all the linux systems did start ok initially. The physical WWPNs may have changed, especially if they are associated with the ficon cards, which were permitted during the upgrade. Btw can anyone tell me how I can stop the EDEV devices so that I can vary the Chris's offline without shutting down z/VM (which has some z/os guests). Keith On 4 Jun 2015, at 21:30, David Kreuter dkreu...@vm-resources.com wrote: Hi Keith: Check the zoning and the NPIVs. The NPIVs presented to SVC from the BC12 could have changed. David Kreuter Original Message Subject: Problems with SCSI-over-FCP after machine upgrade From: Keith Gooding kw...@yahoo.co.uk Date: Thu, June 04, 2015 4:21 pm To: LINUX-390@VM.MARIST.EDU This may not be the proper forum but maybe someone can help. We have a small number of linux systems (32) under z/VM 6.3 which use SCSI connections to LUNs on a SAN Volume Controller via a couple of IBM SAN24B switches (the equivalent of Brocade 300). There are also some systems which use EDEVs on the same SVC. This had worked on z10 BC for about 5 years without problems. Last week the z10 was upgraded to a zBC12, retaining the same FICON cards (4Gbs), but not necessarily associated with the same CHPIDs. Since then a number of the LUN connections have been 'lost', cauing linux systems to fail. SCSIDISC displays eg HCPRXS975I Virtual FCP device 1A05 ignored because the adapter was not able to connect to the fibre channel network. It is then not possible to rebot the linux system. Restarting 'everything' - ie SVC nodes, SAN switches, CHIPD vary off/on - cleared the problem for a while. Any ideas where to start looking ?. I have discovered that we have 32 FCP subchannels defined on the CHPID (but highest used unit address is 1f, and there are only about a dozen in use). Also the switch has not been 'qualified' for use on z12 (but it appears that it was not qualified for z10 either). Any advice greatly appreciated ! -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: Problems with SCSI-over-FCP after machine upgrade
There is no zoning. These are all development systems used only by the z/vm and its linux guests. It was a machine upgrade rather than a new machine and the NPIVs for the z/vm systems were preserved. I did check them and I will check again but all the linux systems did start ok initially. The physical WWPNs may have changed, especially if they are associated with the ficon cards, which were permitted during the upgrade. Btw can anyone tell me how I can stop the EDEV devices so that I can vary the Chris's offline without shutting down z/VM (which has some z/os guests). Keith On 4 Jun 2015, at 21:30, David Kreuter dkreu...@vm-resources.com wrote: Hi Keith: Check the zoning and the NPIVs. The NPIVs presented to SVC from the BC12 could have changed. David Kreuter Original Message Subject: Problems with SCSI-over-FCP after machine upgrade From: Keith Gooding kw...@yahoo.co.uk Date: Thu, June 04, 2015 4:21 pm To: LINUX-390@VM.MARIST.EDU This may not be the proper forum but maybe someone can help. We have a small number of linux systems (32) under z/VM 6.3 which use SCSI connections to LUNs on a SAN Volume Controller via a couple of IBM SAN24B switches (the equivalent of Brocade 300). There are also some systems which use EDEVs on the same SVC. This had worked on z10 BC for about 5 years without problems. Last week the z10 was upgraded to a zBC12, retaining the same FICON cards (4Gbs), but not necessarily associated with the same CHPIDs. Since then a number of the LUN connections have been 'lost', cauing linux systems to fail. SCSIDISC displays eg HCPRXS975I Virtual FCP device 1A05 ignored because the adapter was not able to connect to the fibre channel network. It is then not possible to rebot the linux system. Restarting 'everything' - ie SVC nodes, SAN switches, CHIPD vary off/on - cleared the problem for a while. Any ideas where to start looking ?. I have discovered that we have 32 FCP subchannels defined on the CHPID (but highest used unit address is 1f, and there are only about a dozen in use). Also the switch has not been 'qualified' for use on z12 (but it appears that it was not qualified for z10 either). Any advice greatly appreciated ! -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: Problems with SCSI-over-FCP after machine upgrade
I should have said you will probably need to use SET EDEV .. CLEAR to delete the last path -- it may even remove them all - but as I've never issued it myself - I'm not sure. When I've messed with things it was to vary off particular paths - not get rid of the whole device. You can DEFINE/DELETE EDEVICE --- but I'm fairly sure that with SET EDEV DELETE PATH and CLEAR --- and conversely, ADD PATH ... you should be able to release the subchannels without necessarily redefining the EDEVICE and just dealing with the paths. On Thu, Jun 4, 2015 at 2:13 PM, Scott Rohling scott.rohl...@gmail.com wrote: Use SET EDEV commands to delete all the paths defined over the FCP subchannels... Then use SET EDEV to define them all back when done.. A simple EXEC to do one or the other or both will save your fingers if you need to do this more then once. Scott Rohling On Thu, Jun 4, 2015 at 2:07 PM, kwg kw...@yahoo.co.uk wrote: There is no zoning. These are all development systems used only by the z/vm and its linux guests. It was a machine upgrade rather than a new machine and the NPIVs for the z/vm systems were preserved. I did check them and I will check again but all the linux systems did start ok initially. The physical WWPNs may have changed, especially if they are associated with the ficon cards, which were permitted during the upgrade. Btw can anyone tell me how I can stop the EDEV devices so that I can vary the Chris's offline without shutting down z/VM (which has some z/os guests). Keith On 4 Jun 2015, at 21:30, David Kreuter dkreu...@vm-resources.com wrote: Hi Keith: Check the zoning and the NPIVs. The NPIVs presented to SVC from the BC12 could have changed. David Kreuter Original Message Subject: Problems with SCSI-over-FCP after machine upgrade From: Keith Gooding kw...@yahoo.co.uk Date: Thu, June 04, 2015 4:21 pm To: LINUX-390@VM.MARIST.EDU This may not be the proper forum but maybe someone can help. We have a small number of linux systems (32) under z/VM 6.3 which use SCSI connections to LUNs on a SAN Volume Controller via a couple of IBM SAN24B switches (the equivalent of Brocade 300). There are also some systems which use EDEVs on the same SVC. This had worked on z10 BC for about 5 years without problems. Last week the z10 was upgraded to a zBC12, retaining the same FICON cards (4Gbs), but not necessarily associated with the same CHPIDs. Since then a number of the LUN connections have been 'lost', cauing linux systems to fail. SCSIDISC displays eg HCPRXS975I Virtual FCP device 1A05 ignored because the adapter was not able to connect to the fibre channel network. It is then not possible to rebot the linux system. Restarting 'everything' - ie SVC nodes, SAN switches, CHIPD vary off/on - cleared the problem for a while. Any ideas where to start looking ?. I have discovered that we have 32 FCP subchannels defined on the CHPID (but highest used unit address is 1f, and there are only about a dozen in use). Also the switch has not been 'qualified' for use on z12 (but it appears that it was not qualified for z10 either). Any advice greatly appreciated ! -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: Problems with SCSI-over-FCP after machine upgrade
Hi Keith: Check the zoning and the NPIVs. The NPIVs presented to SVC from the BC12 could have changed. David Kreuter Original Message Subject: Problems with SCSI-over-FCP after machine upgrade From: Keith Gooding kw...@yahoo.co.uk Date: Thu, June 04, 2015 4:21 pm To: LINUX-390@VM.MARIST.EDU This may not be the proper forum but maybe someone can help. We have a small number of linux systems (32) under z/VM 6.3 which use SCSI connections to LUNs on a SAN Volume Controller via a couple of IBM SAN24B switches (the equivalent of Brocade 300). There are also some systems which use EDEVs on the same SVC. This had worked on z10 BC for about 5 years without problems. Last week the z10 was upgraded to a zBC12, retaining the same FICON cards (4Gbs), but not necessarily associated with the same CHPIDs. Since then a number of the LUN connections have been 'lost', cauing linux systems to fail. SCSIDISC displays eg HCPRXS975I Virtual FCP device 1A05 ignored because the adapter was not able to connect to the fibre channel network. It is then not possible to rebot the linux system. Restarting 'everything' - ie SVC nodes, SAN switches, CHIPD vary off/on - cleared the problem for a while. Any ideas where to start looking ?. I have discovered that we have 32 FCP subchannels defined on the CHPID (but highest used unit address is 1f, and there are only about a dozen in use). Also the switch has not been 'qualified' for use on z12 (but it appears that it was not qualified for z10 either). Any advice greatly appreciated ! -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Problems with SCSI-over-FCP after machine upgrade
This may not be the proper forum but maybe someone can help. We have a small number of linux systems (32) under z/VM 6.3 which use SCSI connections to LUNs on a SAN Volume Controller via a couple of IBM SAN24B switches (the equivalent of Brocade 300). There are also some systems which use EDEVs on the same SVC. This had worked on z10 BC for about 5 years without problems. Last week the z10 was upgraded to a zBC12, retaining the same FICON cards (4Gbs), but not necessarily associated with the same CHPIDs. Since then a number of the LUN connections have been 'lost', cauing linux systems to fail. SCSIDISC displays eg HCPRXS975I Virtual FCP device 1A05 ignored because the adapter was not able to connect to the fibre channel network. It is then not possible to rebot the linux system. Restarting 'everything' - ie SVC nodes, SAN switches, CHIPD vary off/on - cleared the problem for a while. Any ideas where to start looking ?. I have discovered that we have 32 FCP subchannels defined on the CHPID (but highest used unit address is 1f, and there are only about a dozen in use). Also the switch has not been 'qualified' for use on z12 (but it appears that it was not qualified for z10 either). Any advice greatly appreciated ! -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
