Re: FCP device

[Alan Altmark]

Oh, and if this is a reference to running 2nd level with SCSI, then you simply 
ATTACH/DEDICATE the FCP subchannels from the 1st level system to your 2nd level 
guest.   They can be attached with virtual addresses that match your 
configuration at home, but your 2nd level system and its guests still must be 
configured to work with the DR storage configuration.

Regards,
Alan

Alan Altmark
Senior Managing z/VM Consultant
IBM Technology Services
1 607 321 7556  (Mobile)
alan_altm...@us.ibm.com

> -Original Message-
> From: Linux on 390 Port  On Behalf Of Peter
> Sent: Tuesday, June 28, 2022 5:04 AM
> To: LINUX-390@VM.MARIST.EDU
> Subject: [EXTERNAL] [LINUX-390] FCP device
> 
> Hello
> 
> Good morning
> 
> How do we create WWPN or FBA device in DR site ?
> 
> As this is the first time we are planning to replicate Linux guest to DR site.
> 
> Is there a way to assign or attach a FBA device to the Linux guest ?
> 
> Peter

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: FCP device

[Alan Altmark]

The term "FBA" is ambiguous in this context.   Are you using z/VM FBA EDEVICEs 
or are you giving Linux guests direct access to SCSI LUNs?

I know this sounds trite, but you do it in DR the same way you do it at home.   
To connect to a SCSI device, there are 4 elements:
1. The initiator (client) WWPN assigned to a specific FCP subchannel on the 
Z/LinuxONE box
2. The target (server) WWPNs assigned to the storage controller 
3. The LUN within the storage controller
4. The permissions within the SAN fabric that allows the initiator WWPNs to 
talk to the target WWPNs.

So at home your z/VM and/or Linux configurations include the target WWPNs and 
LUNs.   The storage controller configuration gives one or more initiator WWPNs 
access to one or more LUNs.  And of course the SAN fabric is configured to 
allow the WWPN pairs to talk to each other.

All of that must be done in DR where the equipment is different and so are the 
WWPNs and, possibly, LUN numbers.

On top of that you have the mechanism that copies the application data from 
home to DR and back.

How you handle the differences between home and DR depend on whether you 
maintain a separate system for each site with its own configuration or whether 
you use a single system (or set of systems) that adapt themselves to their 
location when they IPL based on environmental information that can tell it 
whether it is in DR or at home.

Regards,
Alan

Alan Altmark
Senior Managing z/VM Consultant
IBM Technology Services
1 607 321 7556  (Mobile)
alan_altm...@us.ibm.com

> -Original Message-
> From: Linux on 390 Port  On Behalf Of Peter
> Sent: Tuesday, June 28, 2022 5:04 AM
> To: LINUX-390@VM.MARIST.EDU
> Subject: [EXTERNAL] [LINUX-390] FCP device
> 
> Hello
> 
> Good morning
> 
> How do we create WWPN or FBA device in DR site ?
> 
> As this is the first time we are planning to replicate Linux guest to DR site.
> 
> Is there a way to assign or attach a FBA device to the Linux guest ?
> 
> Peter
> 
> --
> For LINUX-390 subscribe / signoff / archive access instructions, send email to
> lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
> INVALID URI REMOVED
> 3A__www2.marist.edu_htbin_wlvindex-3FLINUX-
> 2D390&d=DwIBaQ&c=jf_iaSHvJObTbx-
> siA1ZOg&r=XX3LPhXj6Fv4hkzdpbonTd1gcy88ea-
> vqLQGEWWoD4M&m=D7M27xBYK0kyVF6ngfV1WiIGu_PJ9Z_KoI0R7oMo954
> k1nBVU2orBFoqyC6G-BZv&s=8-u-w3BBig5CkivElIL3wR7AY4Im8kHBBri-
> iCPhvEM&e=

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: FCP device

[Steffen Maier]

Hi Peter,

On 6/28/22 11:03, Peter wrote:

How do we create WWPN or FBA device in DR site ?

As this is the first time we are planning to replicate Linux guest to DR
site.

Is there a way to assign or attach a FBA device to the Linux guest ?


What's replicated in the DR site? IBM Z CPC, storage (using e.g. PPRC), or both?

Which WWPN: initiator of the host, or target of the storage?
The host initiator WWPNs are managed by the machine and can e.g. be exported on
the SE (example for non-DPM machine):
https://public.dhe.ibm.com/software/dw/linux390/lvc/zFCP_Best_Practices-BB-Webcast_201805.pdf#page=40

I suppose you use z/VM EDEV to represent multiple paths to an FCP-attached
volume as emulated FBA DASD to a guest as opposed to dedicated FCP devices? If
so, I think you would just use a different set of FCP devices on the DR CPC to
assemble the EDEV device and dedicate that resulting FBA DASD to the
"replicated" guest for use on cold failover, i.e. on IPL of the guest on the DR
CPC.

--
Mit freundlichen Gruessen / Kind regards
Steffen Maier

Linux on IBM Z and LinuxONE

https://www.ibm.com/privacy/us/en/
IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Gregor Pillen
Geschaeftsfuehrung: David Faller
Sitz der Gesellschaft: Boeblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

FCP device

[Peter]

Hello

Good morning

How do we create WWPN or FBA device in DR site ?

As this is the first time we are planning to replicate Linux guest to DR
site.

Is there a way to assign or attach a FBA device to the Linux guest ?

Peter

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: FCP disks and IOCDS

[Robert J Brenneman]

I also strongly advocate that you enable NPIV on your FCP pchids. Since the
SAN and storage ACLs are all based on WWPN, you want a unique WWPN for each
FCP device on a channel if the devices will be used by multiple
administrative domains.

If you will only ever use FCP devices from z/VM LPARs for z/VM itself, then
NPIV on those chpids is not terribly critical since its the z/VM Sysprog
responsible for all the SAN config for all the z/VM systems.

On the other hand, if you will be attaching FCP devices to Linux virtual
machines and those linux virtual machines are admind by different teams who
don't talk to or like each other, then you want NPIV enabled on all the
linux FCP chpids so that they cannot accidentally or maliciously attach to
each others disk in the SAN.

On Tue, Feb 8, 2022 at 11:31 AM Steffen Maier  wrote:

> On 2/8/22 00:16, Robert J Brenneman wrote:
> > IODF devices for FCP do not point to disks - they are simply a hole to
> dump
> > frames into the network and pull frames out of the network with.  Its a
> lot
> > more like OSA devices than DASD.   the OSA device is not an endpoint in
> the
> > IP network that you are talking to, and neither is the FCP device in the
> > IODF.
> >
> > you add device 4000 on chpid 40 , and 4100 on chpid 41 to your IODF as
> you
> > have drawn. each chpid plugs to a different switch, and each of those
> > switches in turn plugs to the storage device.
> >
> > When you configure devices 4000 and 4100 online in linux, no storage
> > appears. All that happens is that the 2 pchids log in to the switch that
> > each is plugged to.
> > Once the devices have logged in to the switch - the SAN admin can create
> a
> > zone that permits the WWPN of the Z adapter to talk to the WWPN of the
> > storage box. A WWPN is kinda the SAN equivalent of an IP address - it
> > uniquely identifies this endpoint to the storage network. The FCP devices
> > on the Z have WWPNs, and the storage device ports also have WWPNs, and
> when
> > you create a zone in the storage network with one each of Host WWPN and
> > Storage WWPN the SAN will permit those two endpoints to talk to each
> other.
> > This is very different from FICON CUP network management where you permit
> > port 23 to talk to port 4D - the FICON method is a switch-port to
> > switch-port access control method. It does not care what is at the other
> > end of the port, just that this port can talk to that port - in the
> > switch.   A SAN WWPN zone is that "this end point WWPN" in the network
> can
> > talk to "that end point WWPN" in the network - no matter what physical
> > switch ports there are between them.
> >
> > Once the zone is in place and active the linux OS will see the storage
> > controller - and now in the storage controller you have to tell it about
> > the WWPNs of the Linux host and what virtual disk it is allowed to use.
> > This is called lun masking or mapping, depending on whos storage you're
> > using.  Once you create a host definition on the storage controller that
> > lists the WWPNs of the Linux FCP devices, and maps a LUN to that host
> > definition, Linux will see scsi disk devices appear on the FCP channels.
> >
> > you will see one /dev/sdXX device appear for each path to each lun.  for
> > example 1 LUN with 2 paths will show you /dev/sda and /dev/sdb.   The
> Linux
> > OS can see both paths to the disk - and now you need to enable the linux
> > multipath driver to manage those paths for you. It will create a new
> > /dev/mapper/ device for you to format or use for LVM or
> > whatever and the driver will handle spreading IO across /dev/sda and
> > /dev/sdb paths for you, as well as path recovery if you take a switch
> down
> > for maintenance.
>
> well said
>
> > ref:
> >
> https://www.vm.ibm.com/education/lvc/LVC0924.pdf?cm_sp=dw-dwtv-_-linuxonz-_-PDF-for-3rdpartyhost-videos%20PDFs
>
> just a little update with a more recent version:
>
>
> https://public.dhe.ibm.com/software/dw/linux390/lvc/zFCP_Best_Practices-BB-Webcast_201805.pdf
>
> On top, you might find this useful for zoning:
> Exporting host WWPNs:
> https://listserv.uark.edu/scripts/wa-uarkedu.exe?A2=IBMVM;3c30da04.2110
>
> > On Mon, Feb 7, 2022 at 5:37 PM Davis, Larry (National VM Capability) <
> > larry.dav...@dxc.com> wrote:
> >
> >> Cross posting to Linux for s390 and z/VM List Serves
> >>
> >>
> >> We are looking at using a SAN and connect our z/VM system to it to allow
> >> our Linux servers on z/VM to use it
> >>
> >> We are having questions on Connectivity in the I

Re: FCP disks and IOCDS

[Steffen Maier]

On 2/8/22 00:16, Robert J Brenneman wrote:

IODF devices for FCP do not point to disks - they are simply a hole to dump
frames into the network and pull frames out of the network with.  Its a lot
more like OSA devices than DASD.   the OSA device is not an endpoint in the
IP network that you are talking to, and neither is the FCP device in the
IODF.

you add device 4000 on chpid 40 , and 4100 on chpid 41 to your IODF as you
have drawn. each chpid plugs to a different switch, and each of those
switches in turn plugs to the storage device.

When you configure devices 4000 and 4100 online in linux, no storage
appears. All that happens is that the 2 pchids log in to the switch that
each is plugged to.
Once the devices have logged in to the switch - the SAN admin can create a
zone that permits the WWPN of the Z adapter to talk to the WWPN of the
storage box. A WWPN is kinda the SAN equivalent of an IP address - it
uniquely identifies this endpoint to the storage network. The FCP devices
on the Z have WWPNs, and the storage device ports also have WWPNs, and when
you create a zone in the storage network with one each of Host WWPN and
Storage WWPN the SAN will permit those two endpoints to talk to each other.
This is very different from FICON CUP network management where you permit
port 23 to talk to port 4D - the FICON method is a switch-port to
switch-port access control method. It does not care what is at the other
end of the port, just that this port can talk to that port - in the
switch.   A SAN WWPN zone is that "this end point WWPN" in the network can
talk to "that end point WWPN" in the network - no matter what physical
switch ports there are between them.

Once the zone is in place and active the linux OS will see the storage
controller - and now in the storage controller you have to tell it about
the WWPNs of the Linux host and what virtual disk it is allowed to use.
This is called lun masking or mapping, depending on whos storage you're
using.  Once you create a host definition on the storage controller that
lists the WWPNs of the Linux FCP devices, and maps a LUN to that host
definition, Linux will see scsi disk devices appear on the FCP channels.

you will see one /dev/sdXX device appear for each path to each lun.  for
example 1 LUN with 2 paths will show you /dev/sda and /dev/sdb.   The Linux
OS can see both paths to the disk - and now you need to enable the linux
multipath driver to manage those paths for you. It will create a new
/dev/mapper/ device for you to format or use for LVM or
whatever and the driver will handle spreading IO across /dev/sda and
/dev/sdb paths for you, as well as path recovery if you take a switch down
for maintenance.


well said


ref:
https://www.vm.ibm.com/education/lvc/LVC0924.pdf?cm_sp=dw-dwtv-_-linuxonz-_-PDF-for-3rdpartyhost-videos%20PDFs


just a little update with a more recent version:

https://public.dhe.ibm.com/software/dw/linux390/lvc/zFCP_Best_Practices-BB-Webcast_201805.pdf

On top, you might find this useful for zoning:
Exporting host WWPNs:
https://listserv.uark.edu/scripts/wa-uarkedu.exe?A2=IBMVM;3c30da04.2110


On Mon, Feb 7, 2022 at 5:37 PM Davis, Larry (National VM Capability) <
larry.dav...@dxc.com> wrote:


Cross posting to Linux for s390 and z/VM List Serves


We are looking at using a SAN and connect our z/VM system to it to allow
our Linux servers on z/VM to use it

We are having questions on Connectivity in the IOCP to backend LUN's in
the SAN

For FCP Channels you are not allowed to do Multi-Pathing like we normally
see on the Mainframe by having a control unit have multiple Path to the
same device.

If this is correct then two  SAN switch's that connect to the SAN disks
can map to the same LUN in the back end

Using this will allow 2 separate device addresses to be created that point
to the same backend LUN correct?


Then Linux Multi-pathing  will see 2 separate device addresses as a single
LUN and associate both addresses to the same SAN Device

CHPID 40 (CSS0) -> Switch A ->--

   | SAN  Devices   |
CHPID 41 (CSS0) -> Switch B ->--

CHPID 40, CU 4000, devices 4000-401F
CHPID 41, CU 4100, devices 4100-411F

If the Zone Fabric relates all the out link switch ports to the same
xx00-xx1F devices in the backend, does this allow for Linux Multi-pathing
and eliminate the single CHPID point of failure

Are there any documents that will clarify this possibly for me?


Larry Davis
Senior z/VM Systems Architect,
Leveraged Mainframe Team
DXC Technology

T +1.813.394.4240
larry.dav...@dxc.com<mailto:larry.dav...@dxc.com>




--
Mit freundlichen Gruessen / Kind regards
Steffen Maier

Linux on IBM Z and LinuxONE

https://www.ibm.com/privacy/us/en/
IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Gregor Pillen
Geschaeftsfuehrung: David Faller
Sitz der Gesellschaft

Re: FCP disks and IOCDS

[Davis, Larry (National VM Capability)]

Thanks for the great explanation and the reference

Larry Davis (z/VM Team)

-Original Message-
From: Linux on 390 Port  On Behalf Of Robert J 
Brenneman
Sent: Monday, February 7, 2022 6:16 PM
To: LINUX-390@VM.MARIST.EDU
Subject: Re: FCP disks and IOCDS

IODF devices for FCP do not point to disks - they are simply a hole to dump 
frames into the network and pull frames out of the network with.  Its a lot
more like OSA devices than DASD.   the OSA device is not an endpoint in the
IP network that you are talking to, and neither is the FCP device in the IODF.

you add device 4000 on chpid 40 , and 4100 on chpid 41 to your IODF as you have 
drawn. each chpid plugs to a different switch, and each of those switches in 
turn plugs to the storage device.

When you configure devices 4000 and 4100 online in linux, no storage appears. 
All that happens is that the 2 pchids log in to the switch that each is plugged 
to.
Once the devices have logged in to the switch - the SAN admin can create a zone 
that permits the WWPN of the Z adapter to talk to the WWPN of the storage box. 
A WWPN is kinda the SAN equivalent of an IP address - it uniquely identifies 
this endpoint to the storage network. The FCP devices on the Z have WWPNs, and 
the storage device ports also have WWPNs, and when you create a zone in the 
storage network with one each of Host WWPN and Storage WWPN the SAN will permit 
those two endpoints to talk to each other.
This is very different from FICON CUP network management where you permit port 
23 to talk to port 4D - the FICON method is a switch-port to switch-port access 
control method. It does not care what is at the other end of the port, just 
that this port can talk to that port - in the
switch.   A SAN WWPN zone is that "this end point WWPN" in the network can
talk to "that end point WWPN" in the network - no matter what physical switch 
ports there are between them.

Once the zone is in place and active the linux OS will see the storage 
controller - and now in the storage controller you have to tell it about the 
WWPNs of the Linux host and what virtual disk it is allowed to use.
This is called lun masking or mapping, depending on whos storage you're using.  
Once you create a host definition on the storage controller that lists the 
WWPNs of the Linux FCP devices, and maps a LUN to that host definition, Linux 
will see scsi disk devices appear on the FCP channels.

you will see one /dev/sdXX device appear for each path to each lun.  for
example 1 LUN with 2 paths will show you /dev/sda and /dev/sdb.   The Linux
OS can see both paths to the disk - and now you need to enable the linux 
multipath driver to manage those paths for you. It will create a new 
/dev/mapper/ device for you to format or use for LVM or whatever 
and the driver will handle spreading IO across /dev/sda and /dev/sdb paths for 
you, as well as path recovery if you take a switch down for maintenance.

ref:
https://clicktime.symantec.com/3Ra5CNjXqdX5owcopq55Vp7VN?u=https%3A%2F%2Fwww.vm.ibm.com%2Feducation%2Flvc%2FLVC0924.pdf%3Fcm_sp%3Ddw-dwtv-_-linuxonz-_-PDF-for-3rdpartyhost-videos%2520PDFs

On Mon, Feb 7, 2022 at 5:37 PM Davis, Larry (National VM Capability) < 
larry.dav...@dxc.com> wrote:

> Cross posting to Linux for s390 and z/VM List Serves
>
>
> We are looking at using a SAN and connect our z/VM system to it to
> allow our Linux servers on z/VM to use it
>
> We are having questions on Connectivity in the IOCP to backend LUN's
> in the SAN
>
> For FCP Channels you are not allowed to do Multi-Pathing like we
> normally see on the Mainframe by having a control unit have multiple
> Path to the same device.
>
> If this is correct then two  SAN switch's that connect to the SAN
> disks can map to the same LUN in the back end
>
> Using this will allow 2 separate device addresses to be created that
> point to the same backend LUN correct?
>
>
> Then Linux Multi-pathing  will see 2 separate device addresses as a
> single LUN and associate both addresses to the same SAN Device
>
> CHPID 40 (CSS0) -> Switch A ->--
>
>   | SAN  Devices   |
> CHPID 41 (CSS0) -> Switch B ->--
>
> CHPID 40, CU 4000, devices 4000-401F
> CHPID 41, CU 4100, devices 4100-411F
>
> If the Zone Fabric relates all the out link switch ports to the same
> xx00-xx1F devices in the backend, does this allow for Linux
> Multi-pathing and eliminate the single CHPID point of failure
>
> Are there any documents that will clarify this possibly for me?
>
>
> Larry Davis
> Senior z/VM Systems Architect,
> Leveraged Mainframe Team
> DXC Technology
>
> T +1.813.394.4240
> larry.dav...@dxc.com<mailto:larry.dav...@dxc.com>
>
>
>
>
> 

Re: FCP disks and IOCDS

[Robert J Brenneman]

IODF devices for FCP do not point to disks - they are simply a hole to dump
frames into the network and pull frames out of the network with.  Its a lot
more like OSA devices than DASD.   the OSA device is not an endpoint in the
IP network that you are talking to, and neither is the FCP device in the
IODF.

you add device 4000 on chpid 40 , and 4100 on chpid 41 to your IODF as you
have drawn. each chpid plugs to a different switch, and each of those
switches in turn plugs to the storage device.

When you configure devices 4000 and 4100 online in linux, no storage
appears. All that happens is that the 2 pchids log in to the switch that
each is plugged to.
Once the devices have logged in to the switch - the SAN admin can create a
zone that permits the WWPN of the Z adapter to talk to the WWPN of the
storage box. A WWPN is kinda the SAN equivalent of an IP address - it
uniquely identifies this endpoint to the storage network. The FCP devices
on the Z have WWPNs, and the storage device ports also have WWPNs, and when
you create a zone in the storage network with one each of Host WWPN and
Storage WWPN the SAN will permit those two endpoints to talk to each other.
This is very different from FICON CUP network management where you permit
port 23 to talk to port 4D - the FICON method is a switch-port to
switch-port access control method. It does not care what is at the other
end of the port, just that this port can talk to that port - in the
switch.   A SAN WWPN zone is that "this end point WWPN" in the network can
talk to "that end point WWPN" in the network - no matter what physical
switch ports there are between them.

Once the zone is in place and active the linux OS will see the storage
controller - and now in the storage controller you have to tell it about
the WWPNs of the Linux host and what virtual disk it is allowed to use.
This is called lun masking or mapping, depending on whos storage you're
using.  Once you create a host definition on the storage controller that
lists the WWPNs of the Linux FCP devices, and maps a LUN to that host
definition, Linux will see scsi disk devices appear on the FCP channels.

you will see one /dev/sdXX device appear for each path to each lun.  for
example 1 LUN with 2 paths will show you /dev/sda and /dev/sdb.   The Linux
OS can see both paths to the disk - and now you need to enable the linux
multipath driver to manage those paths for you. It will create a new
/dev/mapper/ device for you to format or use for LVM or
whatever and the driver will handle spreading IO across /dev/sda and
/dev/sdb paths for you, as well as path recovery if you take a switch down
for maintenance.

ref:
https://www.vm.ibm.com/education/lvc/LVC0924.pdf?cm_sp=dw-dwtv-_-linuxonz-_-PDF-for-3rdpartyhost-videos%20PDFs

On Mon, Feb 7, 2022 at 5:37 PM Davis, Larry (National VM Capability) <
larry.dav...@dxc.com> wrote:

> Cross posting to Linux for s390 and z/VM List Serves
>
>
> We are looking at using a SAN and connect our z/VM system to it to allow
> our Linux servers on z/VM to use it
>
> We are having questions on Connectivity in the IOCP to backend LUN's in
> the SAN
>
> For FCP Channels you are not allowed to do Multi-Pathing like we normally
> see on the Mainframe by having a control unit have multiple Path to the
> same device.
>
> If this is correct then two  SAN switch's that connect to the SAN disks
> can map to the same LUN in the back end
>
> Using this will allow 2 separate device addresses to be created that point
> to the same backend LUN correct?
>
>
> Then Linux Multi-pathing  will see 2 separate device addresses as a single
> LUN and associate both addresses to the same SAN Device
>
> CHPID 40 (CSS0) -> Switch A ->--
>
>   | SAN  Devices   |
> CHPID 41 (CSS0) -> Switch B ->--
>
> CHPID 40, CU 4000, devices 4000-401F
> CHPID 41, CU 4100, devices 4100-411F
>
> If the Zone Fabric relates all the out link switch ports to the same
> xx00-xx1F devices in the backend, does this allow for Linux Multi-pathing
> and eliminate the single CHPID point of failure
>
> Are there any documents that will clarify this possibly for me?
>
>
> Larry Davis
> Senior z/VM Systems Architect,
> Leveraged Mainframe Team
> DXC Technology
>
> T +1.813.394.4240
> larry.dav...@dxc.com<mailto:larry.dav...@dxc.com>
>
>
>
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www2.marist.edu/htbin/wlvindex?LINUX-390
>


--
Jay Brenneman

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

FCP disks and IOCDS

[Davis, Larry (National VM Capability)]

Cross posting to Linux for s390 and z/VM List Serves


We are looking at using a SAN and connect our z/VM system to it to allow our 
Linux servers on z/VM to use it

We are having questions on Connectivity in the IOCP to backend LUN's in the SAN

For FCP Channels you are not allowed to do Multi-Pathing like we normally see 
on the Mainframe by having a control unit have multiple Path to the same device.

If this is correct then two  SAN switch's that connect to the SAN disks can map 
to the same LUN in the back end

Using this will allow 2 separate device addresses to be created that point to 
the same backend LUN correct?


Then Linux Multi-pathing  will see 2 separate device addresses as a single LUN 
and associate both addresses to the same SAN Device

CHPID 40 (CSS0) -> Switch A ->--

| SAN  Devices   |
CHPID 41 (CSS0) -> Switch B ->--

CHPID 40, CU 4000, devices 4000-401F
CHPID 41, CU 4100, devices 4100-411F

If the Zone Fabric relates all the out link switch ports to the same xx00-xx1F 
devices in the backend, does this allow for Linux Multi-pathing and eliminate 
the single CHPID point of failure

Are there any documents that will clarify this possibly for me?


Larry Davis
Senior z/VM Systems Architect,
Leveraged Mainframe Team
DXC Technology

T +1.813.394.4240
larry.dav...@dxc.com<mailto:larry.dav...@dxc.com>




--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: CP overhead of using FCP attached SCSI SAN

[Rob van der Heij]

On Thu, 5 Nov 2020 at 16:23, Dave Jones  wrote:

> So handling this approximately 1K times could drive the %steal up, I
> think.
>

That appears to be nmon failing, probably picking up a stale pointer or
following some bunny trail. It's something to pick up with Nigel. It does
not directly contribute to CP overhead, but it's possible that the recovery
consumes good CPU resources that make the guest compete harder for CPU
resources to get the real work done, and find more contention when getting
beyond allocated capacity.

Rob

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: CP overhead of using FCP attached SCSI SAN

[Dave Jones]

No offense taking, Rob! :-)
We now have another clue, this time from the console log. This error
keeps recurring:

[82053.035436] User process fault: interruption code 0x60010 in
nmon_mainframe_6
[82053.035448] failing address: 3FFFD379000
[82053.035453] CPU: 24 PID: 17149 Comm: nmon_mainframe_ Kdump: loaded
Not tainte
[82053.035456] task: 0041402d8000 ti: 0042b55d8000 task.ti:
0042b55d
[82053.035459] User PSW : 070520018000 80007114 (0x80007114)
[82053.035477] R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:1 AS:0 CC:2 PM:0 EA:
User GPRS: 441c 0073e750 00670680
00077000
[82053.035484] afe8da70 afe89b90 80067a50 000
[82053.035488] 01f9 03fffcbc1440 00035430 000
[82053.035492] 01fc 800282b0 0672 000
[82053.035505] User Code: 80007106: eb210003000d sllg %r2,%r1,
8000710c: b9080012 agr %r1,%r2
#80007110: 41717000 la %r7,0(%r1,%r7)

80007114: e31071a4 lg %r1,416(%r7)

8000711a: ec1300442065 clgrj %r1,%r3,2,800071a2
80007120: b9e91013 sgrk %r1,%r3,%r1
80007124: e32071a80004 lg %r2,424(%r7)
8000712a: e33091a80004 lg %r3,424(%r9)
[82053.035558] Last Breaking-Event-Address:
[82053.035560] <8000707e> 0x8000707e

So handling this approximately 1K times could drive the %steal up, I
think.
Thanks again.
DJ
---
DAVID JONES | MANAGING DIRECTOR FOR ZSYSTEMS SERVICES | z/VM, Linux, and
Cloud
703.237.7370 (Office) | 281.578.7544 (CELL)

INFORMATION TECHNOLOGY COMPANY

On 11.05.2020 12:48 AM, Rob van der Heij wrote:

On Wed, 4 Nov 2020 at 20:06, Rob van der Heij 
wrote:



On Wed, 4 Nov 2020 at 19:30, Dave Jones  wrote

What is the CP overhead of managing this?  The one Linux guest that is

running here reports a %steal of 15-17%, which I think is a bit high.




At the risk of teaching granny... the most common cause for reported
high
"steal percentage" in Linux is not CP overhead on I/O, but contention
for
CPU resources on z/VM and PR/SM level.
And when you see high %steal combined with %idle in the guest, then it
may
well be due to an application causing a high amount of polling (and
lowering the number of virtual CPUs closer to the allocated capacity
may be
helpful).

Rob

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390
or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: CP overhead of using FCP attached SCSI SAN

[Rob van der Heij]

On Wed, 4 Nov 2020 at 20:06, Rob van der Heij  wrote:

>
> On Wed, 4 Nov 2020 at 19:30, Dave Jones  wrote
>
> What is the CP overhead of managing this?  The one Linux guest that is
>> running here reports a %steal of 15-17%, which I think is a bit high.
>>
>
At the risk of teaching granny... the most common cause for reported high
"steal percentage" in Linux is not CP overhead on I/O, but contention for
CPU resources on z/VM and PR/SM level.
And when you see high %steal combined with %idle in the guest, then it may
well be due to an application causing a high amount of polling (and
lowering the number of virtual CPUs closer to the allocated capacity may be
helpful).

Rob

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: CP overhead of using FCP attached SCSI SAN

[Rob van der Heij]

On Wed, 4 Nov 2020 at 19:30, Dave Jones  wrote

What is the CP overhead of managing this?  The one Linux guest that is
> running here reports a %steal of 15-17%, which I think is a bit high.
> Could this be configured better?
> Thanks, appreciate it.


You’re right that EDEV overhead would show in the guest’s TTIME and thus
noticed by Linux as steal. Your configuration with application data on FCP
and just OS on EDEV is good practice.

I would not expect OS disk I/O as high that you notice, unless when you’re
very tight on memory and drop libraries and executables from page cache as
soon as the program terminates. You’d see those things with stateless
agents that fire up a program every 10 seconds to report some data.

My rules of thumb are too dated, but you should look with vmstat -d or
other tools whether there really is enough I/O to worry. Then look at what
is being read or written (application logs could be an issue and can be
moved to FCP)

Rob

>
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: CP overhead of using FCP attached SCSI SAN

[barton]

I don't see how steal time and the scsi would be related. There's an old 
article about what steal time is at "http://velocitysoftware.com/STEAL.html";



On 11/4/2020 10:30 AM, Dave Jones wrote:

Hello, all.

I have a site that is using SCSI SAN (a V7000) to hold all z/VM
storagethe system z/VM and Linux software is installed on emulated
FBA dasd, and the Oracle database is stored on 360 or so SCSI disks,
attached via FCP.

What is the CP overhead of managing this?  The one Linux guest that is
running here reports a %steal of 15-17%, which I think is a bit high.
Could this be configured better?
Thanks, appreciate it.
DJ

--
DAVID JONES | MANAGING DIRECTOR FOR ZSYSTEMS SERVICES | z/VM, Linux, and
Cloud
703.237.7370 (Office) | 281.578.7544 (CELL)

INFORMATION TECHNOLOGY COMPANY

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 
or visit

http://www2.marist.edu/htbin/wlvindex?LINUX-390




--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

CP overhead of using FCP attached SCSI SAN

[Dave Jones]

Hello, all.

I have a site that is using SCSI SAN (a V7000) to hold all z/VM
storagethe system z/VM and Linux software is installed on emulated
FBA dasd, and the Oracle database is stored on 360 or so SCSI disks,
attached via FCP.

What is the CP overhead of managing this?  The one Linux guest that is
running here reports a %steal of 15-17%, which I think is a bit high.
Could this be configured better?
Thanks, appreciate it.
DJ

--
DAVID JONES | MANAGING DIRECTOR FOR ZSYSTEMS SERVICES | z/VM, Linux, and
Cloud
703.237.7370 (Office) | 281.578.7544 (CELL)

INFORMATION TECHNOLOGY COMPANY

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: KVM and FCP disk without SAN switch

[Robert J Brenneman]

aahhh yea. a dark corner of memory gets its first ray of light
in a while

V7000 and other storwize related products ( SVC, FS9xxx ) now do NPIV on
the storage side by default if you are doing a new install. You have to
explicitly enable it if you are upgrading from older versions.

Ostensibly this feature is there to enable easier storage migrations down
the line, since the WWPN you're configuring into the SAN zone and on the
operating system side is now virtual. It is supposed to make switching the
physical storage box out from under the running OS by using box managed
copy services like metro-mirror easier and more automate-able without
having to take a planned outage.

This obviously also requires a SAN switch for it to work since it uses NPIV.

--
Jay Brenneman

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: KVM and FCP disk without SAN switch

[Christian Borntraeger]

On 17.09.19 17:18, Johan Schelling wrote:
> I remember that when we started with KVM on LinuxONE around 3/4 years ago we 
> connected our V7000 directly (without a SAN switch) to the LinuxONE.  zVM 
> worked like a charm, but we had a lot of problems with (the then IBM 
> supplied) KVM…..  We had a lot of discussion with both the LinuxONE and the 
> storage guys as the documentation wasn’t quite clear on what was supported or 
> not. Problems we faced had to do with (hope I recall correctly) with WWPN’s 
> returned from the V7000 were  all mixed up… At the moment we added a SAN 
> switch all our problems were gone … ;-)
> Haven’t had any problems with KVM since …..   but haven’t checked the 
> situation without a SAN switch in the last years.

Yes, there was a firmware issue with the V7000s back then. But this has been 
fixed.
I still have a v7000 directly connected to my test z13 as an outcome of that 
bug back then.
But as I said, unless this is officially supported by all components you should 
better 
use a switch. 

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: KVM and FCP disk without SAN switch

[Johan Schelling]

I remember that when we started with KVM on LinuxONE around 3/4 years ago we 
connected our V7000 directly (without a SAN switch) to the LinuxONE.  zVM 
worked like a charm, but we had a lot of problems with (the then IBM supplied) 
KVM…..  We had a lot of discussion with both the LinuxONE and the storage guys 
as the documentation wasn’t quite clear on what was supported or not. Problems 
we faced had to do with (hope I recall correctly) with WWPN’s returned from the 
V7000 were  all mixed up… At the moment we added a SAN switch all our 
problems were gone … ;-)
Haven’t had any problems with KVM since …..   but haven’t checked the situation 
without a SAN switch in the last years.

Regards
Johan Schelling
Infrastructure Solution Architect


[cid:image001.png@01D008CB.2EEA8890]



ICU IT Services BV
Transistorstraat 55b  I  1322 CK  ALMERE

M 06 – 21 245 992  I  E 
johan.schell...@icu-it.nl<mailto:johan.schell...@icu-it.nl>
T 088 – 5 234 123  I  www.icu-it.nl<http://www.icu-it.nl/>  I  KvK 32135776
ICU Proclaimer<http://www.icu-it.nl/proclaimer/>

Op 17 sep. 2019, om 12:56 heeft Christian Borntraeger 
mailto:borntrae...@linux.ibm.com>> het volgende 
geschreven:

On 16.09.19 21:19, Jim Elliott wrote:
For z/VM you must have a SAN switch to connect FCP attached disk. Is this
also true for KVM (Ubuntu if it matters)?

As KVM borrows the FCP support from Linux the answer is the same as for Linux 
in LPAR.
In general the Linux code supports switched fabric and direct attachment. The 
code does
not support arbitrated loop.
What you also need is a support statement that tells about the storage server 
and the
Z system. For example a V7000 DOES work when connected to a z Box (there is no 
fencing)
but it is not a supported config (at least it was not when I check 3 years ago).
So in general you are on the safe-side with a switch. Without a switch requires 
to check
the certifications.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu<mailto:lists...@vm.marist.edu> with the 
message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: KVM and FCP disk without SAN switch

[Robert J Brenneman]

Also be aware that if you want to use NPIV you /must/ have a switch and the
switch must explicitly support NPIV since that is a function that requires
both FCP Host adapter and switch support to work.

On Tue, Sep 17, 2019 at 7:35 AM Jim Elliott  wrote:

> Christian:
>
> Thanks. We will go with a switch to attach the V5010 and decided on an
> IBM/Cisco SAN32C-6 with 8-port 16 Gb bundle.
>
> Jim Elliott
> Senior Consultant - GlassHouse Systems Inc.
>
>
> On Tue, Sep 17, 2019 at 6:58 AM Christian Borntraeger <
> borntrae...@linux.ibm.com> wrote:
>
> > On 16.09.19 21:19, Jim Elliott wrote:
> > > For z/VM you must have a SAN switch to connect FCP attached disk. Is
> this
> > > also true for KVM (Ubuntu if it matters)?
> >
> > As KVM borrows the FCP support from Linux the answer is the same as for
> > Linux in LPAR.
> > In general the Linux code supports switched fabric and direct attachment.
> > The code does
> > not support arbitrated loop.
> > What you also need is a support statement that tells about the storage
> > server and the
> > Z system. For example a V7000 DOES work when connected to a z Box (there
> > is no fencing)
> > but it is not a supported config (at least it was not when I check 3
> years
> > ago).
> > So in general you are on the safe-side with a switch. Without a switch
> > requires to check
> > the certifications.
> >
> > --
> > For LINUX-390 subscribe / signoff / archive access instructions,
> > send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> > visit
> > http://www2.marist.edu/htbin/wlvindex?LINUX-390
> >
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www2.marist.edu/htbin/wlvindex?LINUX-390
>


--
Jay Brenneman

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: KVM and FCP disk without SAN switch

[Jim Elliott]

Christian:

Thanks. We will go with a switch to attach the V5010 and decided on an
IBM/Cisco SAN32C-6 with 8-port 16 Gb bundle.

Jim Elliott
Senior Consultant - GlassHouse Systems Inc.


On Tue, Sep 17, 2019 at 6:58 AM Christian Borntraeger <
borntrae...@linux.ibm.com> wrote:

> On 16.09.19 21:19, Jim Elliott wrote:
> > For z/VM you must have a SAN switch to connect FCP attached disk. Is this
> > also true for KVM (Ubuntu if it matters)?
>
> As KVM borrows the FCP support from Linux the answer is the same as for
> Linux in LPAR.
> In general the Linux code supports switched fabric and direct attachment.
> The code does
> not support arbitrated loop.
> What you also need is a support statement that tells about the storage
> server and the
> Z system. For example a V7000 DOES work when connected to a z Box (there
> is no fencing)
> but it is not a supported config (at least it was not when I check 3 years
> ago).
> So in general you are on the safe-side with a switch. Without a switch
> requires to check
> the certifications.
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www2.marist.edu/htbin/wlvindex?LINUX-390
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: KVM and FCP disk without SAN switch

[Christian Borntraeger]

On 16.09.19 21:19, Jim Elliott wrote:
> For z/VM you must have a SAN switch to connect FCP attached disk. Is this
> also true for KVM (Ubuntu if it matters)?

As KVM borrows the FCP support from Linux the answer is the same as for Linux 
in LPAR.
In general the Linux code supports switched fabric and direct attachment. The 
code does
not support arbitrated loop.
What you also need is a support statement that tells about the storage server 
and the
Z system. For example a V7000 DOES work when connected to a z Box (there is no 
fencing)
but it is not a supported config (at least it was not when I check 3 years ago).
So in general you are on the safe-side with a switch. Without a switch requires 
to check
the certifications.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: KVM and FCP disk without SAN switch

[Martha McConaghy]

Carlos is right.  It really doesn't matter what system is on the Z (VM 
or Linux), it has more to do with the FCP FICON adapters. The v 
boxes don't have the protocol to talk to FCP directly, while the more 
sophistcated DS8xxx do.  So, no matter what system you are talking to on 
Z, you would need a switch for the v5010.


Martha

On 9/16/2019 4:44 PM, Jim Elliott wrote:

Carlos:
  
Thanks. It will be a V5010 so we will configure a small SAN switch.
  
Jim Elliott

Senior Consultant - GlassHouse Systems Inc.
  
  
On Mon, Sep 16, 2019 at 4:39 PM Bodra - Pessoal  wrote:
  

 From our experience with KVM running on a zBox if you use as storage
devices IBM V5000 or V7000 you need to use a San Switch (2498-Bxx) to
connect. If you use storage devices IBM DS8800/8870, you can connect
directly. These are some tests that we did here at our shop.


Carlos Bodra
IBM zEnterprise Certified
São Paulo – SP – Brazil


-Mensagem original-
De: Linux on 390 Port  Em nome de Jim Elliott
Enviada em: segunda-feira, 16 de setembro de 2019 16:20
Para: LINUX-390@VM.MARIST.EDU
Assunto: KVM and FCP disk without SAN switch

For z/VM you must have a SAN switch to connect FCP attached disk. Is this
also true for KVM (Ubuntu if it matters)?

Jim Elliott
Senior Consultant - GlassHouse Systems Inc.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

  
--

For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390


--
Martha McConaghy
Marist: System Architect/Technical Lead
SHARE Association: Vice President
Marist College IT
Poughkeepsie, NY 12601

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: KVM and FCP disk without SAN switch

[Jim Elliott]

Carlos:

Thanks. It will be a V5010 so we will configure a small SAN switch.

Jim Elliott
Senior Consultant - GlassHouse Systems Inc.


On Mon, Sep 16, 2019 at 4:39 PM Bodra - Pessoal  wrote:

> From our experience with KVM running on a zBox if you use as storage
> devices IBM V5000 or V7000 you need to use a San Switch (2498-Bxx) to
> connect. If you use storage devices IBM DS8800/8870, you can connect
> directly. These are some tests that we did here at our shop.
>
>
> Carlos Bodra
> IBM zEnterprise Certified
> São Paulo – SP – Brazil
>
>
> -Mensagem original-
> De: Linux on 390 Port  Em nome de Jim Elliott
> Enviada em: segunda-feira, 16 de setembro de 2019 16:20
> Para: LINUX-390@VM.MARIST.EDU
> Assunto: KVM and FCP disk without SAN switch
>
> For z/VM you must have a SAN switch to connect FCP attached disk. Is this
> also true for KVM (Ubuntu if it matters)?
>
> Jim Elliott
> Senior Consultant - GlassHouse Systems Inc.
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www2.marist.edu/htbin/wlvindex?LINUX-390
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www2.marist.edu/htbin/wlvindex?LINUX-390
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

RES: KVM and FCP disk without SAN switch

[Bodra - Pessoal]

>From our experience with KVM running on a zBox if you use as storage devices 
>IBM V5000 or V7000 you need to use a San Switch (2498-Bxx) to connect. If you 
>use storage devices IBM DS8800/8870, you can connect directly. These are some 
>tests that we did here at our shop.


Carlos Bodra
IBM zEnterprise Certified
São Paulo – SP – Brazil


-Mensagem original-
De: Linux on 390 Port  Em nome de Jim Elliott
Enviada em: segunda-feira, 16 de setembro de 2019 16:20
Para: LINUX-390@VM.MARIST.EDU
Assunto: KVM and FCP disk without SAN switch

For z/VM you must have a SAN switch to connect FCP attached disk. Is this
also true for KVM (Ubuntu if it matters)?

Jim Elliott
Senior Consultant - GlassHouse Systems Inc.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: KVM and FCP disk without SAN switch

[Adolph Kahan]

Yes, it is. With KVM you have more switch options than with z/VM

Adolph Kahan | Senior Consultant
Tel: +1 (416) 229-2950 x304 | Mobile: +1 (416) 822-3329
  

-Original Message-
From: Linux on 390 Port  On Behalf Of Jim Elliott
Sent: Monday, September 16, 2019 3:20 PM
To: LINUX-390@VM.MARIST.EDU
Subject: KVM and FCP disk without SAN switch

For z/VM you must have a SAN switch to connect FCP attached disk. Is this also 
true for KVM (Ubuntu if it matters)?

Jim Elliott
Senior Consultant - GlassHouse Systems Inc.

--
For LINUX-390 subscribe / signoff / archive access instructions, send email to 
lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww2.marist.edu%2Fhtbin%2Fwlvindex%3FLINUX-390&data=01%7C01%7Cakahan%40GHSYSTEMS.COM%7C3c38157fe09f4553da0d08d73adb28c2%7C1c932bc9c5af4366811d012126996da4%7C0&sdata=DwHDjMM7P9M4Rbc0M4TBx%2BYYHt9KlLhRfp9ncaYFMmM%3D&reserved=0

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

KVM and FCP disk without SAN switch

[Jim Elliott]

For z/VM you must have a SAN switch to connect FCP attached disk. Is this
also true for KVM (Ubuntu if it matters)?

Jim Elliott
Senior Consultant - GlassHouse Systems Inc.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: Problems with SCSI-over-FCP after machine upgrade

[Keith Gooding]

We have fixed this problem but made more than one change so we do not know the 
cause - it would be nice to know who to blame !

Thanks for all the advice. I followed up Scott's suggestion of creating an exec 
to delete the EDEV' s paths and to do SET EDEV  CLEAR so that I could vary 
the CHPIDs offline. (I found that to redefine the EDEVs I had to use 'SET EDEV 
' providing the   Type, Attributes, FCP_dev, wwpn and LUN without the text 
'ADD PATH' for the first path and with the text 'add path' for the other paths).
There is no zoning or LUN masking involved and the NPIV wwpns had not changed 
(the physical NPIVs had changed - they contains the new PCHIPID ids). We had 
already tried switching everything in the paths off/on.

 The changes which we made to resolve the problem were: - switch firmware was 
upgraded from 6.1.1a to 6.3.0d . Neither had been qualified for zBC12. (We 
later upgraded to 7.0.0d)- removed the link between the two SAN switches. The 
SAN Volume Controller documentation says that the fibre networks should be 
independent. When I disconnected the link I found that one of the SFPs came out 
too - it had not 'clicked' into place.- cleaned the z12 end of the fibre links.
All of the zlinuxes then came up and stayed up, except for some which relied on 
the ISL being in place. ie the LOADDEV parameters (for loading from  chpid 18) 
specified an SVC wwpn which was now accessible only from the other CHPID, which 
was easily fixed.
Maybe the problem was due to FCP channel microcode in z12 (or z114/196). Or 
maybe there were some fabric errors which were exacerbated by the faster z12 
IFLs.
Keith


  


 On Monday, 8 June 2015, 15:39, Raymond Higgs  wrote:
   

 Hi Keith,

Please check zoning and lun masking. I think the SVC management interface calls 
them hosts.  If youdidn't do an MES upgrade, then the WWPNs of your channels 
changed.

Regards,

Ray Higgs
System z FCP Firmware Development
Bld. 706, B42
2455 South Road
Poughkeepsie, NY 12601
(845) 435-8666,  T/L 295-8666
rayhi...@us.ibm.com



From:       Keith Gooding 
To:       LINUX-390@VM.MARIST.EDU
Date:       06/04/2015 04:23 PM
Subject:       Problems withSCSI-over-FCP after machine upgrade
Sent by:       Linux on 390Port 



This may not be the proper forum but maybe someonecan help.
 We have a small number of linux systems (<32) under z/VM 6.3 whichuse SCSI 
connections to LUNs on a SAN Volume Controller via a couple ofIBM SAN24B 
switches (the equivalent of Brocade 300). There are also somesystems which use 
EDEVs on the same SVC. This had worked on z10 BC forabout 5 years without 
problems.
 Last week the z10 was upgraded to a zBC12, retaining the same FICON 
cards(4Gbs), but not necessarily associated with the same CHPIDs. Since thena 
number of the LUN connections have been 'lost', cauing linux systemsto fail. 
SCSIDISC displays eg "HCPRXS975I Virtual FCP device 1A05ignored because the 
adapter was not able to connect to the fibre channelnetwork". It is then not 
possible to rebot the linux system.

Restarting 'everything' - ie SVC nodes, SAN switches, CHIPD vary off/on-  
cleared the problem for a while.
Any ideas where to start looking ?. I have discovered that we have >32 FCP 
subchannels defined on the CHPID (but highest used unit addressis 1f, and there 
are only about a dozen in use). Also the switch has notbeen 'qualified' for use 
on z12 (but it appears that it was not qualifiedfor z10 either).
Any advice greatly appreciated !
 

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 orvisit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/





--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Problems with SCSI-over-FCP after machine upgrade

[Scott Rohling]

I should have said you will probably need to use SET EDEV .. CLEAR to
delete the last path -- it may even remove them all - but as I've never
issued it myself - I'm not sure.  When I've messed with things it was to
vary off particular paths - not get rid of the whole device.   You can
DEFINE/DELETE EDEVICE --- but I'm fairly sure that with SET EDEV DELETE
PATH and CLEAR ---   and conversely, ADD PATH  ... you should be able to
release the subchannels without necessarily redefining the EDEVICE and just
dealing with the paths.

On Thu, Jun 4, 2015 at 2:13 PM, Scott Rohling 
wrote:

> Use SET EDEV commands to delete all the paths defined over the FCP
> subchannels... Then use SET EDEV to define them all back when done..
> A simple EXEC to do one or the other or both will save your fingers if you
> need to do this more then once.
>
> Scott Rohling
>
> On Thu, Jun 4, 2015 at 2:07 PM, kwg  wrote:
>
>> There is no zoning. These are all development systems used only by the
>> z/vm and its linux guests. It was a machine upgrade rather than a new
>> machine and the NPIVs for the z/vm systems were preserved. I did check them
>> and I will check again but all the linux systems did start ok initially.
>> The physical WWPNs may have changed, especially if they are associated with
>> the ficon cards, which were permitted during the upgrade.
>>
>> Btw can anyone tell me how I can stop the EDEV devices so that I can vary
>> the Chris's offline without shutting down z/VM (which has some z/os guests).
>>
>> Keith
>>
>>
>>
>>
>> > On 4 Jun 2015, at 21:30, David Kreuter 
>> wrote:
>> >
>> > Hi Keith: Check the zoning and the NPIVs. The NPIVs presented to SVC
>> > from the BC12 could have changed.
>> > David Kreuter
>> >
>> >
>> >  Original Message 
>> > Subject: Problems with SCSI-over-FCP after machine upgrade
>> > From: Keith Gooding 
>> > Date: Thu, June 04, 2015 4:21 pm
>> > To: LINUX-390@VM.MARIST.EDU
>> >
>> > This may not be the proper forum but maybe someone can help.
>> > We have a small number of linux systems (<32) under z/VM 6.3 which use
>> > SCSI connections to LUNs on a SAN Volume Controller via a couple of IBM
>> > SAN24B switches (the equivalent of Brocade 300). There are also some
>> > systems which use EDEVs on the same SVC. This had worked on z10 BC for
>> > about 5 years without problems.
>> > Last week the z10 was upgraded to a zBC12, retaining the same FICON
>> > cards (4Gbs), but not necessarily associated with the same CHPIDs. Since
>> > then a number of the LUN connections have been 'lost', cauing linux
>> > systems to fail. SCSIDISC displays eg "HCPRXS975I Virtual FCP device
>> > 1A05 ignored because the adapter was not able to connect to the fibre
>> > channel network". It is then not possible to rebot the linux system.
>> >
>> > Restarting 'everything' - ie SVC nodes, SAN switches, CHIPD vary off/on
>> > -  cleared the problem for a while.
>> > Any ideas where to start looking ?. I have discovered that we have > 32
>> > FCP subchannels defined on the CHPID (but highest used unit address is
>> > 1f, and there are only about a dozen in use). Also the switch has not
>> > been 'qualified' for use on z12 (but it appears that it was not
>> > qualified for z10 either).
>> > Any advice greatly appreciated !
>> >
>> >
>> > --
>> > For LINUX-390 subscribe / signoff / archive access instructions,
>> > send email to lists...@vm.marist.edu with the message: INFO LINUX-390
>> or
>> > visit
>> > http://www.marist.edu/htbin/wlvindex?LINUX-390
>> > --
>> > For more information on Linux on System z, visit
>> > http://wiki.linuxvm.org/
>> >
>> > --
>> > For LINUX-390 subscribe / signoff / archive access instructions,
>> > send email to lists...@vm.marist.edu with the message: INFO LINUX-390
>> or visit
>> > http://www.marist.edu/htbin/wlvindex?LINUX-390
>> > --
>> > For more information on Linux on System z, visit
>> > http://wiki.linuxvm.org/
>>
>> --
>> For LINUX-390 subscribe / 

Re: Problems with SCSI-over-FCP after machine upgrade

[Scott Rohling]

Use SET EDEV commands to delete all the paths defined over the FCP
subchannels... Then use SET EDEV to define them all back when done..
A simple EXEC to do one or the other or both will save your fingers if you
need to do this more then once.

Scott Rohling

On Thu, Jun 4, 2015 at 2:07 PM, kwg  wrote:

> There is no zoning. These are all development systems used only by the
> z/vm and its linux guests. It was a machine upgrade rather than a new
> machine and the NPIVs for the z/vm systems were preserved. I did check them
> and I will check again but all the linux systems did start ok initially.
> The physical WWPNs may have changed, especially if they are associated with
> the ficon cards, which were permitted during the upgrade.
>
> Btw can anyone tell me how I can stop the EDEV devices so that I can vary
> the Chris's offline without shutting down z/VM (which has some z/os guests).
>
> Keith
>
>
>
>
> > On 4 Jun 2015, at 21:30, David Kreuter 
> wrote:
> >
> > Hi Keith: Check the zoning and the NPIVs. The NPIVs presented to SVC
> > from the BC12 could have changed.
> > David Kreuter
> >
> >
> >  Original Message 
> > Subject: Problems with SCSI-over-FCP after machine upgrade
> > From: Keith Gooding 
> > Date: Thu, June 04, 2015 4:21 pm
> > To: LINUX-390@VM.MARIST.EDU
> >
> > This may not be the proper forum but maybe someone can help.
> > We have a small number of linux systems (<32) under z/VM 6.3 which use
> > SCSI connections to LUNs on a SAN Volume Controller via a couple of IBM
> > SAN24B switches (the equivalent of Brocade 300). There are also some
> > systems which use EDEVs on the same SVC. This had worked on z10 BC for
> > about 5 years without problems.
> > Last week the z10 was upgraded to a zBC12, retaining the same FICON
> > cards (4Gbs), but not necessarily associated with the same CHPIDs. Since
> > then a number of the LUN connections have been 'lost', cauing linux
> > systems to fail. SCSIDISC displays eg "HCPRXS975I Virtual FCP device
> > 1A05 ignored because the adapter was not able to connect to the fibre
> > channel network". It is then not possible to rebot the linux system.
> >
> > Restarting 'everything' - ie SVC nodes, SAN switches, CHIPD vary off/on
> > -  cleared the problem for a while.
> > Any ideas where to start looking ?. I have discovered that we have > 32
> > FCP subchannels defined on the CHPID (but highest used unit address is
> > 1f, and there are only about a dozen in use). Also the switch has not
> > been 'qualified' for use on z12 (but it appears that it was not
> > qualified for z10 either).
> > Any advice greatly appreciated !
> >
> >
> > --
> > For LINUX-390 subscribe / signoff / archive access instructions,
> > send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> > visit
> > http://www.marist.edu/htbin/wlvindex?LINUX-390
> > --
> > For more information on Linux on System z, visit
> > http://wiki.linuxvm.org/
> >
> > --
> > For LINUX-390 subscribe / signoff / archive access instructions,
> > send email to lists...@vm.marist.edu with the message: INFO LINUX-390
> or visit
> > http://www.marist.edu/htbin/wlvindex?LINUX-390
> > --
> > For more information on Linux on System z, visit
> > http://wiki.linuxvm.org/
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Problems with SCSI-over-FCP after machine upgrade

[kwg]

There is no zoning. These are all development systems used only by the z/vm and 
its linux guests. It was a machine upgrade rather than a new machine and the 
NPIVs for the z/vm systems were preserved. I did check them and I will check 
again but all the linux systems did start ok initially. The physical WWPNs may 
have changed, especially if they are associated with the ficon cards, which 
were permitted during the upgrade.

Btw can anyone tell me how I can stop the EDEV devices so that I can vary the 
Chris's offline without shutting down z/VM (which has some z/os guests).

Keith




> On 4 Jun 2015, at 21:30, David Kreuter  wrote:
> 
> Hi Keith: Check the zoning and the NPIVs. The NPIVs presented to SVC
> from the BC12 could have changed.
> David Kreuter
> 
> 
>  Original Message 
> Subject: Problems with SCSI-over-FCP after machine upgrade
> From: Keith Gooding 
> Date: Thu, June 04, 2015 4:21 pm
> To: LINUX-390@VM.MARIST.EDU
> 
> This may not be the proper forum but maybe someone can help.
> We have a small number of linux systems (<32) under z/VM 6.3 which use
> SCSI connections to LUNs on a SAN Volume Controller via a couple of IBM
> SAN24B switches (the equivalent of Brocade 300). There are also some
> systems which use EDEVs on the same SVC. This had worked on z10 BC for
> about 5 years without problems.
> Last week the z10 was upgraded to a zBC12, retaining the same FICON
> cards (4Gbs), but not necessarily associated with the same CHPIDs. Since
> then a number of the LUN connections have been 'lost', cauing linux
> systems to fail. SCSIDISC displays eg "HCPRXS975I Virtual FCP device
> 1A05 ignored because the adapter was not able to connect to the fibre
> channel network". It is then not possible to rebot the linux system.
> 
> Restarting 'everything' - ie SVC nodes, SAN switches, CHIPD vary off/on
> -  cleared the problem for a while.
> Any ideas where to start looking ?. I have discovered that we have > 32
> FCP subchannels defined on the CHPID (but highest used unit address is
> 1f, and there are only about a dozen in use). Also the switch has not
> been 'qualified' for use on z12 (but it appears that it was not
> qualified for z10 either).
> Any advice greatly appreciated !
> 
> 
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
> 
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Problems with SCSI-over-FCP after machine upgrade

[David Kreuter]

Hi Keith: Check the zoning and the NPIVs. The NPIVs presented to SVC
from the BC12 could have changed.
David Kreuter


 Original Message 
Subject: Problems with SCSI-over-FCP after machine upgrade
From: Keith Gooding 
Date: Thu, June 04, 2015 4:21 pm
To: LINUX-390@VM.MARIST.EDU

This may not be the proper forum but maybe someone can help.
 We have a small number of linux systems (<32) under z/VM 6.3 which use
SCSI connections to LUNs on a SAN Volume Controller via a couple of IBM
SAN24B switches (the equivalent of Brocade 300). There are also some
systems which use EDEVs on the same SVC. This had worked on z10 BC for
about 5 years without problems.
 Last week the z10 was upgraded to a zBC12, retaining the same FICON
cards (4Gbs), but not necessarily associated with the same CHPIDs. Since
then a number of the LUN connections have been 'lost', cauing linux
systems to fail. SCSIDISC displays eg "HCPRXS975I Virtual FCP device
1A05 ignored because the adapter was not able to connect to the fibre
channel network". It is then not possible to rebot the linux system.

Restarting 'everything' - ie SVC nodes, SAN switches, CHIPD vary off/on
-  cleared the problem for a while.
Any ideas where to start looking ?. I have discovered that we have > 32
FCP subchannels defined on the CHPID (but highest used unit address is
1f, and there are only about a dozen in use). Also the switch has not
been 'qualified' for use on z12 (but it appears that it was not
qualified for z10 either).
Any advice greatly appreciated !
 

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Problems with SCSI-over-FCP after machine upgrade

[Keith Gooding]

This may not be the proper forum but maybe someone can help.
 We have a small number of linux systems (<32) under z/VM 6.3 which use SCSI 
connections to LUNs on a SAN Volume Controller via a couple of IBM SAN24B 
switches (the equivalent of Brocade 300). There are also some systems which use 
EDEVs on the same SVC. This had worked on z10 BC for about 5 years without 
problems.
 Last week the z10 was upgraded to a zBC12, retaining the same FICON cards 
(4Gbs), but not necessarily associated with the same CHPIDs. Since then a 
number of the LUN connections have been 'lost', cauing linux systems to fail. 
SCSIDISC displays eg "HCPRXS975I Virtual FCP device 1A05 ignored because the 
adapter was not able to connect to the fibre channel network". It is then not 
possible to rebot the linux system.

Restarting 'everything' - ie SVC nodes, SAN switches, CHIPD vary off/on -  
cleared the problem for a while.
Any ideas where to start looking ?. I have discovered that we have > 32 FCP 
subchannels defined on the CHPID (but highest used unit address is 1f, and 
there are only about a dozen in use). Also the switch has not been 'qualified' 
for use on z12 (but it appears that it was not qualified for z10 either).
Any advice greatly appreciated !
 

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: relocation put read only FCP

[Victor Echavarry Diaz]

The problem was a the dev_loss_tmo. It using the default causing the read only 
issue. The correct dev_loss_tmo is 120 solving the problem.

Thanks for your help Robert, Sam and Gregory.

Victor Echavarry
System Programmer, EVERTEC LLC

-Original Message-
From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of Grzegorz 
Powiedziuk
Sent: Wednesday, January 21, 2015 12:19 PM
To: LINUX-390@VM.MARIST.EDU
Subject: Re: relocation put read only FCP

What Robert said might be correct. The other thing I would check if that won't 
help is:
 - hosts definition on the XIV and Zonning in SAN switches When you relocate 
virtual machine from one LPAR to another you are also switching FCP channels 
under the hood for the relocated linux guest. Which means your linux guest 
after the reloctation will talk to the XIV from different FCP channels with  
different WWPNs.  XIV must have these new WWPN in the host deffinition for this 
guest (LUN Masking). Also zone definitions in SAN Switches have to contain 
"new" WWPNs.

Regards
Gregory Powiedziuk


2015-01-21 9:43 GMT-05:00 Victor Echavarry Diaz :

> This is a multi-part MIME message.
>
> Content-Type: text/plain; charset="us-ascii"
> Content-Transfer-Encoding: quoted-printable
>
>
> We are doing relocation testing under z/VM 6.3. We test a SLES 11SP2  
> and o= racle 10.2. We bring up on both z/vm LPARS in standalone way 
> and works fine=  on both LPARS. When we begin relocation from any of 
> the LPAR, immediately = the SAN file system goes in read only mode. 
> I'm attaching part of the log a= t the moment of failure. The SAN is a 
> XIV We are running z/VM 6.3.
>
> Thanks for your help
>
> Victor Echavarry
> System Programmer, EVERTEC LLC
>
>
>
>
>
> WARNING: This email and any files transmitted with it are confidential 
> and intended solely for the use of the individual or entity to whom 
> they are addressed. If you have received this email in error please 
> delete it immedi= ately.
> Please note that any views or opinions presented in this email are 
> solely t= hose of the author and do not necessarily represent those of 
> EVERTEC, Inc. or its affiliates. Finally, the integrity and security 
> of this message cannot be guaranteed on the Internet, and as such 
> EVERTEC, Inc. and its affiliates ac= cept no liability for any damage 
> caused by any virus transmitted by this email.
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions, send 
> email to lists...@vm.marist.edu with the message: INFO LINUX-390 or 
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit 
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions, send email to 
lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit http://wiki.linuxvm.org/



WARNING: This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they are
addressed. If you have received this email in error please delete it 
immediately.
Please note that any views or opinions presented in this email are solely those
of the author and do not necessarily represent those of EVERTEC, Inc. or its
affiliates. Finally, the integrity and security of this message cannot be
guaranteed on the Internet, and as such EVERTEC, Inc. and its affiliates accept
no liability for any damage caused by any virus transmitted by this email.

Re: relocation put read only FCP

[Cohen, Sam]

Also, be sure your EQID for the FCP subchannels to the specific LUN are the 
same in both LPARs.

Thanks,


Sam Cohen
Levi, Ray & Shoup, Inc.

-Original Message-
From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of Grzegorz 
Powiedziuk
Sent: Wednesday, January 21, 2015 9:19 AM
To: LINUX-390@VM.MARIST.EDU
Subject: Re: relocation put read only FCP

What Robert said might be correct. The other thing I would check if that won't 
help is:
 - hosts definition on the XIV and Zonning in SAN switches When you relocate 
virtual machine from one LPAR to another you are also switching FCP channels 
under the hood for the relocated linux guest. Which means your linux guest 
after the reloctation will talk to the XIV from different FCP channels with  
different WWPNs.  XIV must have these new WWPN in the host deffinition for this 
guest (LUN Masking). Also zone definitions in SAN Switches have to contain 
"new" WWPNs.

Regards
Gregory Powiedziuk


2015-01-21 9:43 GMT-05:00 Victor Echavarry Diaz :

> This is a multi-part MIME message.
>
> Content-Type: text/plain; charset="us-ascii"
> Content-Transfer-Encoding: quoted-printable
>
>
> We are doing relocation testing under z/VM 6.3. We test a SLES 11SP2  
> and o= racle 10.2. We bring up on both z/vm LPARS in standalone way 
> and works fine=  on both LPARS. When we begin relocation from any of 
> the LPAR, immediately = the SAN file system goes in read only mode. 
> I'm attaching part of the log a= t the moment of failure. The SAN is a 
> XIV We are running z/VM 6.3.
>
> Thanks for your help
>
> Victor Echavarry
> System Programmer, EVERTEC LLC
>
>
>
>
>
> WARNING: This email and any files transmitted with it are confidential 
> and intended solely for the use of the individual or entity to whom 
> they are addressed. If you have received this email in error please 
> delete it immedi= ately.
> Please note that any views or opinions presented in this email are 
> solely t= hose of the author and do not necessarily represent those of 
> EVERTEC, Inc. or its affiliates. Finally, the integrity and security 
> of this message cannot be guaranteed on the Internet, and as such 
> EVERTEC, Inc. and its affiliates ac= cept no liability for any damage 
> caused by any virus transmitted by this email.
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions, send 
> email to lists...@vm.marist.edu with the message: INFO LINUX-390 or 
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit 
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions, send email to 
lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit http://wiki.linuxvm.org/

Re: relocation put read only FCP

[Grzegorz Powiedziuk]

What Robert said might be correct. The other thing I would check if that
won't help is:
 - hosts definition on the XIV and Zonning in SAN switches
When you relocate virtual machine from one LPAR to another you are also
switching FCP channels under the hood for the relocated linux guest. Which
means your linux guest after the reloctation will talk to the XIV from
different FCP channels with  different WWPNs.  XIV must have these new WWPN
in the host deffinition for this guest (LUN Masking). Also zone definitions
in SAN Switches have to contain "new" WWPNs.

Regards
Gregory Powiedziuk


2015-01-21 9:43 GMT-05:00 Victor Echavarry Diaz :

> This is a multi-part MIME message.
>
> Content-Type: text/plain; charset="us-ascii"
> Content-Transfer-Encoding: quoted-printable
>
>
> We are doing relocation testing under z/VM 6.3. We test a SLES 11SP2  and
> o=
> racle 10.2. We bring up on both z/vm LPARS in standalone way and works
> fine=
>  on both LPARS. When we begin relocation from any of the LPAR, immediately
> =
> the SAN file system goes in read only mode. I'm attaching part of the log
> a=
> t the moment of failure. The SAN is a XIV We are running z/VM 6.3.
>
> Thanks for your help
>
> Victor Echavarry
> System Programmer, EVERTEC LLC
>
>
>
>
>
> WARNING: This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they are
> addressed. If you have received this email in error please delete it
> immedi=
> ately.
> Please note that any views or opinions presented in this email are solely
> t=
> hose
> of the author and do not necessarily represent those of EVERTEC, Inc. or
> its
> affiliates. Finally, the integrity and security of this message cannot be
> guaranteed on the Internet, and as such EVERTEC, Inc. and its affiliates
> ac=
> cept
> no liability for any damage caused by any virus transmitted by this email.
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: relocation put read only FCP

[Robert J Brenneman]

Attachments don't make it through the listserve software, but I can guess
that what is happening to your system is that the relocation involves a
pause while the processor state is relocated to the new target system,
followed by an I/O recovery interrupt.

the FCP / SCSI / Multipath driver stack interprets this as an I/O error and
sets the file systems read only to preserve your data.

You can open up the timeout window that multipath uses to look for these
errors by making some settings as mentioned by Steffen on PG 30 here:
http://www.vm.ibm.com/education/lvc/LVC0924.pdf

and
http://www-01.ibm.com/support/knowledgecenter/SSB27U_6.3.0/com.ibm.zvm.v630.hcpa5/hcpa5313.htm
regarding the queue_if_no_path multipath option

On Wed, Jan 21, 2015 at 9:43 AM, Victor Echavarry Diaz <
vechava...@evertecinc.com> wrote:

> This is a multi-part MIME message.
>
> Content-Type: text/plain; charset="us-ascii"
> Content-Transfer-Encoding: quoted-printable
>
>
> We are doing relocation testing under z/VM 6.3. We test a SLES 11SP2  and
> o=
> racle 10.2. We bring up on both z/vm LPARS in standalone way and works
> fine=
>  on both LPARS. When we begin relocation from any of the LPAR, immediately
> =
> the SAN file system goes in read only mode. I'm attaching part of the log
> a=
> t the moment of failure. The SAN is a XIV We are running z/VM 6.3.
>
> Thanks for your help
>
> Victor Echavarry
> System Programmer, EVERTEC LLC
>
>
>
>
>
> WARNING: This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they are
> addressed. If you have received this email in error please delete it
> immedi=
> ately.
> Please note that any views or opinions presented in this email are solely
> t=
> hose
> of the author and do not necessarily represent those of EVERTEC, Inc. or
> its
> affiliates. Finally, the integrity and security of this message cannot be
> guaranteed on the Internet, and as such EVERTEC, Inc. and its affiliates
> ac=
> cept
> no liability for any damage caused by any virus transmitted by this email.
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>



--
Jay Brenneman

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

relocation put read only FCP

[Victor Echavarry Diaz]

This is a multi-part MIME message.

Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable


We are doing relocation testing under z/VM 6.3. We test a SLES 11SP2  and o=
racle 10.2. We bring up on both z/vm LPARS in standalone way and works fine=
 on both LPARS. When we begin relocation from any of the LPAR, immediately =
the SAN file system goes in read only mode. I'm attaching part of the log a=
t the moment of failure. The SAN is a XIV We are running z/VM 6.3.

Thanks for your help

Victor Echavarry
System Programmer, EVERTEC LLC





WARNING: This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they are
addressed. If you have received this email in error please delete it immedi=
ately.
Please note that any views or opinions presented in this email are solely t=
hose
of the author and do not necessarily represent those of EVERTEC, Inc. or its
affiliates. Finally, the integrity and security of this message cannot be
guaranteed on the Internet, and as such EVERTEC, Inc. and its affiliates ac=
cept
no liability for any damage caused by any virus transmitted by this email.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
Jan 20 17:19:02 L209T kernel: qeth.3acf0c: 0.0.0360: The qeth device driver 
failed to recover an error on the device
Jan 20 17:19:02 L209T kernel: qeth: irb : 00 c2 40 17 7e 05 10 38 0e 02 
00 00 00 80 00 00  ..@.~..8
Jan 20 17:19:02 L209T kernel: qeth: irb 0010: 01 02 00 00 00 00 00 00 00 00 
00 00 00 00 00 00  
Jan 20 17:19:02 L209T kernel: qeth: sense data : 02 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00  
Jan 20 17:19:02 L209T kernel: qeth: sense data 0010: 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00  
Jan 20 17:19:02 L209T kernel: qeth.3acf0c: 0.0.0360: The qeth device driver 
failed to recover an error on the device
Jan 20 17:19:02 L209T kernel: qeth: irb : 00 c2 40 17 7e 05 10 38 0e 02 
00 00 00 80 00 00  ..@.~..8
Jan 20 17:19:02 L209T kernel: qeth: irb 0010: 01 02 00 00 00 00 00 00 00 00 
00 00 00 00 00 00  
Jan 20 17:19:02 L209T kernel: qeth: irb 0020: 02 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00  
Jan 20 17:19:02 L209T kernel: qeth: irb 0030: 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00  
Jan 20 17:19:02 L209T kernel: zfcp.e78dec: 0.0.7548: A QDIO problem occurred
Jan 20 17:19:02 L209T kernel: zfcp.e78dec: 0.0.7508: A QDIO problem occurred
Jan 20 17:19:02 L209T kernel: qeth.fd0b7c: 0.0.0360: A recovery process has 
been started for the device
Jan 20 17:19:02 L209T kernel: qdio: 0.0.0362 OSA on SC 2 using AI:1 QEBSM:0 
PCI:1 TDD:1 SIGA:RW A 
Jan 20 17:19:02 L209T kernel: qdio: 0.0.7508 ZFCP on SC 15 using AI:1 QEBSM:1 
PCI:1 TDD:1 SIGA: W A 
Jan 20 17:19:02 L209T kernel: qdio: 0.0.7548 ZFCP on SC 16 using AI:1 QEBSM:1 
PCI:1 TDD:1 SIGA: W A 
Jan 20 17:19:02 L209T kernel: qeth.cc0c57: 0.0.0360: MAC address 
02:78:c1:02:09:00 successfully registered on device eth0
Jan 20 17:19:02 L209T kernel: qeth.736dae: 0.0.0360: Device is a Guest LAN QDIO 
card (level: V630)
Jan 20 17:19:02 L209T kernel: with link type GuestLAN QDIO (portname: )
Jan 20 17:19:02 L209T kernel: qeth.bad88b: 0.0.0360: Device successfully 
recovered!
Jan 20 17:19:07 L209T multipathd: 8:0: mark as failed
Jan 20 17:19:07 L209T multipathd: 20017380009c51b8f: remaining active paths: 5
Jan 20 17:19:07 L209T multipathd: 8:16: mark as failed
Jan 20 17:19:07 L209T multipathd: 20017380009c51b90: remaining active paths: 5
Jan 20 17:19:07 L209T multipathd: 8:64: mark as failed
Jan 20 17:19:07 L209T multipathd: 20017380009c51cfb: remaining active paths: 5
Jan 20 17:19:07 L209T multipathd: 20017380009c51b90: sdb - directio checker 
reports path is down
Jan 20 17:19:07 L209T multipathd: 20017380009c51b8f: sda - directio checker 
reports path is down
Jan 20 17:19:07 L209T multipathd: 20017380009c51cfa: sdc - directio checker 
reports path is down
Jan 20 17:19:07 L209T multipathd: checker failed path 8:32 in map 
20017380009c51cfa
Jan 20 17:19:07 L209T multipathd: 20017380009c51cfa: remaining active paths: 5
Jan 20 17:19:07 L209T multipathd: 20017380009c51cf9: sdd - directio checker 
reports path is down
Jan 20 17:19:07 L209T multipathd: checker failed path 8:48 in map 
20017380009c51cf9
Jan 20 17:19:07 L209T multipathd: 20017380009c51cf9: remaining active paths: 5
Jan 20 17:19:07 L209T multipathd: 20017380009c51cfb: sde - directio checker 
reports path is down
Jan 20 17:19:07 L209T multipathd: 200173800

Re: Trying to share 2 FCP on 2 zVM 6.2 SSI 2nd level

[Alan Altmark]

On Thursday, 05/23/2013 at 10:28 EDT, Sam Bass 
wrote:
> Running z/VM 5.4 with ZVMA62 and ZVMB62 guests on 1 CEC.
> Successfully brought of a z/Linux guest under ZVMA62 without FCP and did
VMRELO
> to ZVMB62.
>
> How can we test FCP for this z/Linux guest on these two second level
z/VM 6.2
> systems (ZVMA62 and ZVMB62)?

Well, you have to ATTACH/DEDICATE some of 1st level FCP subchannels to
your 2nd level guests and assign EQIDs to them.  To really validate
things, you need to have some with EQID1 and some with EQID2 on the target
system so that you can watch CP selecting one that matches the origin
system's EQID.

Alan Altmark

Senior Managing z/VM and Linux Consultant
IBM System Lab Services and Training
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Trying to share 2 FCP on 2 zVM 6.2 SSI 2nd level

[Sam Bass]

Running z/VM 5.4 with ZVMA62 and ZVMB62 guests on 1 CEC.
Successfully brought of a z/Linux guest under ZVMA62 without FCP and did VMRELO 
to ZVMB62.

How can we test FCP for this z/Linux guest on these two second level z/VM 6.2 
systems (ZVMA62 and ZVMB62)?

Sam Bass

Re: SCSI/FCP disk size

[Jon Miller]

And to add to the conversation, I've also used UDEV to help query disk
information for me in the past. Say your SCSI disk is "/dev/sdf", then you
can query all sorts of information with udev via:
udevadm info -a -p $(udevadm info -q path -n /dev/sdf) | less -Ip size

 I choose "less" in my sample CLI invocation to help point out the "size"
attribute but also promote exploration of the other attributes available.

-- Jon Miller

On Wed, Oct 10, 2012 at 12:27 PM, Mark Post  wrote:

> >>> On 10/9/2012 at 07:32 PM, Thang Pham  wrote:
> > Is there a way to find out the size of a native SCSI device attached via
> > FCP channel?  I do not see lszfcp or lsscsi having an option that lets
> you
> > see the size of the disk you have attached to a VM.
>
> The simplest and most direct method is simply "fdisk -l /dev/sd?".  After
> all, it's just a SCSI disk.
>
>
> Mark Post
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: SCSI/FCP disk size

[Brad Hinson]

On Oct 9, 2012, at 7:32 PM, Thang Pham wrote:

> Hello List,
> 
> Is there a way to find out the size of a native SCSI device attached via
> FCP channel?  I do not see lszfcp or lsscsi having an option that lets you
> see the size of the disk you have attached to a VM.
> 

I always liked "sfdisk -s ".  Just returns the size as one 
number, easy to parse in a script.

> Thanks,
> Thang Pham
> 
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: SCSI/FCP disk size

[Mark Post]

>>> On 10/9/2012 at 07:32 PM, Thang Pham  wrote: 
> Is there a way to find out the size of a native SCSI device attached via
> FCP channel?  I do not see lszfcp or lsscsi having an option that lets you
> see the size of the disk you have attached to a VM.

The simplest and most direct method is simply "fdisk -l /dev/sd?".  After all, 
it's just a SCSI disk.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: SCSI/FCP disk size

[Alan Altmark]

On Wednesday, 10/10/2012 at 09:40 EDT, Steffen Maier
 wrote:

> I'm not quite sure what you mean in your second sentence with regard to
> a disk attached to a VM.
> To a VM as a z/VM userid, i.e. attached to the virtual machine by the
> hypervisor? AFAIK this means EDEV under z/VM. I don't know of any other
> way of attaching a scsi disk to a VM (usually users only attach the FCP
> host bus adapter to a VM).
> To a VM as in guest operating system? Then the previous paragraphs
apply.
> Or would you like to get the lun size without even attaching them to
> Linux or the VM? If so, then this depends on the storage type since
> you'd have to use out of band mechanisms (i.e. non-scsi) to query the
> storage target server.

Regrettably absent is the ability of CP to perform standard (and common
extension) inquiries of a remote LUN without having to ATTACH it to a
guest.  I.e. CP QUERY LUN 5 WWPN x  DETAILS.  Right now,
LUNs are very much like tape drives: they have to be attached to a guest
that knows how to talk to it in detail in order to get useful information
out of it.

Alan Altmark

Senior Managing z/VM and Linux Consultant
IBM System Lab Services and Training
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: SCSI/FCP disk size

[Scott Rohling]

I use /proc/partitions all the time - but stopped recommending it to others
when I was told (can't remember the source) it was deprecated.   I'd be
glad to be wrong here.. as you say - it's quick and easy and no running
through /dev or /sys structures..

Scott Rohling

On Wed, Oct 10, 2012 at 7:02 AM, Rick Troth wrote:

> On 10/10/2012 09:27 AM, Peter Oberparleiter wrote:
> > # cat /proc/partitions
> > major minor  #blocks  name
> >80   10485760 sda
> >81   10485743 sda1
> >
> > Note: the #blocks is the size in 1k-blocks
>
> Yep.  I was going to suggest exactly what Peter suggested.  It's quick
> and easy.
>
> Note that a disk need not be "partitioned" to show up under
> /proc/partitions.  (I say, since I have a penchant for pointing it
> out.)  In fact, if you're going to use these volumes as "LVM fodder"
> (that is, let them be PVs in a volume group), then you will find it
> better to *not* partition them.  Just stamp the requisite LVM magic on
> "sda" (instead of "sda1") and proceed.
>
>
> --
>
> Rick Troth
> Senior Software Developer
>
> Velocity Software Inc.
> Mountain View, CA 94041
> Main: (877) 964-8867
> Direct: (614) 594-9768
> ri...@velocitysoftware.com 
> 
> Signature 
> *Follow us:*
> Facebook
> 
> LinkedIn  Twitter
>  Xing
> 
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: SCSI/FCP disk size

[Rick Troth]

<><><><><>

signature.asc
Description: OpenPGP digital signature

Re: SCSI/FCP disk size

[Peter Oberparleiter]

On 10.10.2012 01:32, Thang Pham wrote:

Is there a way to find out the size of a native SCSI device attached via
FCP channel?  I do not see lszfcp or lsscsi having an option that lets you
see the size of the disk you have attached to a VM.


You can view the usable size of any block device (not just SCSI) using 
the following command:


# cat /proc/partitions
major minor  #blocks  name
   80   10485760 sda
   81   10485743 sda1

Note: the #blocks is the size in 1k-blocks

Newer distributions provide a tool called 'lsblk' which also shows disk 
and partition sizes.


# lsblk
NAME MAJ:MIN RM   SIZE RO MOUNTPOINT
sda8:0010G  0
└─sda1 8:1010G  0 /


Regards,
  Peter Oberparleiter

--
Peter Oberparleiter
Linux on System z Development - IBM Germany

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: SCSI/FCP disk size

[Steffen Maier]

On 10/10/2012 02:00 AM, Thang Pham wrote:

That works, thanks.

From:   Raymond Higgs/Poughkeepsie/IBM@IBMUS
Date:   10/09/2012 07:56 PM

It is in /var/log/messages:

Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.656933] scsi 1:0:26:1082146832: 
Direct-Access IBM  2107900  36.5 PQ: 0 ANSI: 5
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.657047] sd 1:0:26:1082146832: 
Attached scsi generic sg21 type 0
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.659692] sd 1:0:26:1082146832: 
[sdv] 4194304 512-byte logical blocks: (2.14 GB/2.00 GiB)
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.660473] sd 1:0:26:1082146832: 
[sdv] Write Protect is off
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.660839] sd 1:0:26:1082146832: 
[sdv] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.663606]  sdv: unknown partition 
table
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.665817] sd 1:0:26:1082146832: 
[sdv] Attached SCSI disk

Or send the SCSI read capacity command like this:

root@4e1d-laplace-48.1:sg_readcap /dev/sdv
Read Capacity results:
Last logical block address=4194303 (0x3f), Number of blocks=4194304
Logical block length=512 bytes
Hence:
Device size: 2147483648 bytes, 2048.0 MiB, 2.15 GB



Linux on 390 Port  wrote on 10/09/2012 07:32:19 PM:


From: Thang Pham/Poughkeepsie/IBM@IBMUS
Date: 10/09/2012 07:40 PM



Is there a way to find out the size of a native SCSI device attached via
FCP channel?  I do not see lszfcp or lsscsi having an option that lets you
see the size of the disk you have attached to a VM.


The size of a scsi (disk) device is the same as the block device size. 
This is not specific to Linux on System z nor zfcp nor SCSI. It's just 
common block subsystem code.


Depending on what kind of code you want to process this information with;
for scripting you could use /sys/block//size (and adhere to 
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=blob;f=Documentation/sysfs-rules.txt;hb=HEAD) 
(or parse the somewhat older /proc/partitions);
for C code you could just use the ioctl BLKGETSIZE64 from 
include/linux/fs.h [see e.g. the source of 'blockdev --getsz ' as 
an example].
All other suggestions so far basically boil down to this in the end, so 
I'd probably prefer to use the user space interface directly.


This requires that you had attached the LUN to Linux previously and that 
would include lun probing in the kernel which already did inquiry and 
readcapacity (if the device is a disk, e.g.) among other things so 
there's no need to resend those scsi commands again explicitly.


I'm not quite sure what you mean in your second sentence with regard to 
a disk attached to a VM.
To a VM as a z/VM userid, i.e. attached to the virtual machine by the 
hypervisor? AFAIK this means EDEV under z/VM. I don't know of any other 
way of attaching a scsi disk to a VM (usually users only attach the FCP 
host bus adapter to a VM).

To a VM as in guest operating system? Then the previous paragraphs apply.
Or would you like to get the lun size without even attaching them to 
Linux or the VM? If so, then this depends on the storage type since 
you'd have to use out of band mechanisms (i.e. non-scsi) to query the 
storage target server.


Steffen Maier

Linux on System z Development

IBM Deutschland Research & Development GmbH
Vorsitzende des Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: SCSI/FCP disk size

[Michael MacIsaac]

fdisk can help.

Here is a little hack I named "lunsizes" that assumes up to 26 mounted
LUNs and "friendly names" (/dev/mapper/mapth):

#!/bin/bash
# find LUNs in /dev/mapper and list in bytes and GiB
ls /dev/mapper/mpath[a-z] >/dev/null 2>&1
if [ $? != 0 ]; then # no LUNs found
  echo "No LUNs found in /dev/mapper/mpath*"
  exit 1
fi

echo -e "LUN   \tBytes  \t~GiB"
for nextLUN in /dev/mapper/mpath[a-z]; do
  bytes=`fdisk -l $nextLUN 2>/dev/null | grep "Disk $nextLUN" | awk
'{print $5}'`
  let GB=bytes/1024/1024/1024
  echo -e "$nextLUN \t$bytes \t$GB"
done

"Mike MacIsaac" 



From:
Thang Pham/Poughkeepsie/IBM@IBMUS
To:
LINUX-390@vm.marist.edu,
Date:
10/09/2012 07:33 PM
Subject:
[LINUX-390] SCSI/FCP disk size
Sent by:
Linux on 390 Port 



Hello List,

Is there a way to find out the size of a native SCSI device attached via
FCP channel?  I do not see lszfcp or lsscsi having an option that lets you
see the size of the disk you have attached to a VM.

Thanks,
Thang Pham

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/




--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: SCSI/FCP disk size

[Thang Pham]

That works, thanks.




From:   Raymond Higgs/Poughkeepsie/IBM@IBMUS
To: LINUX-390@vm.marist.edu,
Date:   10/09/2012 07:56 PM
Subject:Re: SCSI/FCP disk size
Sent by:Linux on 390 Port 



It is in /var/log/messages:

Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.656933] scsi
1:0:26:1082146832: Direct-Access IBM  2107900  36.5 PQ: 0
ANSI: 5
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.657047] sd
1:0:26:1082146832: Attached scsi generic sg21 type 0
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.659692] sd
1:0:26:1082146832: [sdv] 4194304 512-byte logical blocks: (2.14 GB/2.00
GiB)
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.660473] sd
1:0:26:1082146832: [sdv] Write Protect is off
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.660839] sd
1:0:26:1082146832: [sdv] Write cache: enabled, read cache: enabled,
doesn't support DPO or FUA
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.663606]  sdv: unknown
partition table
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.665817] sd
1:0:26:1082146832: [sdv] Attached SCSI disk

Or send the SCSI read capacity command like this:

root@4e1d-laplace-48.1:sg_readcap /dev/sdv
Read Capacity results:
   Last logical block address=4194303 (0x3f), Number of blocks=4194304
   Logical block length=512 bytes
Hence:
   Device size: 2147483648 bytes, 2048.0 MiB, 2.15 GB

Regards,

Ray Higgs
System z FCP Firmware Development
Bld. 706, B42
2455 South Road
Poughkeepsie, NY 12601
(845) 435-8666,  T/L 295-8666
rayhi...@us.ibm.com

Linux on 390 Port  wrote on 10/09/2012 07:32:19
PM:

> From: Thang Pham/Poughkeepsie/IBM@IBMUS
> To: LINUX-390@vm.marist.edu
> Date: 10/09/2012 07:40 PM
> Subject: SCSI/FCP disk size
> Sent by: Linux on 390 Port 
>
> Hello List,
>
> Is there a way to find out the size of a native SCSI device attached via
> FCP channel?  I do not see lszfcp or lsscsi having an option that lets
you
> see the size of the disk you have attached to a VM.
>
> Thanks,
> Thang Pham
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
<>

Re: SCSI/FCP disk size

[Raymond Higgs]

It is in /var/log/messages:

Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.656933] scsi
1:0:26:1082146832: Direct-Access IBM  2107900  36.5 PQ: 0
ANSI: 5
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.657047] sd
1:0:26:1082146832: Attached scsi generic sg21 type 0
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.659692] sd
1:0:26:1082146832: [sdv] 4194304 512-byte logical blocks: (2.14 GB/2.00
GiB)
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.660473] sd
1:0:26:1082146832: [sdv] Write Protect is off
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.660839] sd
1:0:26:1082146832: [sdv] Write cache: enabled, read cache: enabled,
doesn't support DPO or FUA
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.663606]  sdv: unknown
partition table
Oct  9 19:43:21 4e1d-laplace-48 kernel: [23044.665817] sd
1:0:26:1082146832: [sdv] Attached SCSI disk

Or send the SCSI read capacity command like this:

root@4e1d-laplace-48.1:sg_readcap /dev/sdv
Read Capacity results:
   Last logical block address=4194303 (0x3f), Number of blocks=4194304
   Logical block length=512 bytes
Hence:
   Device size: 2147483648 bytes, 2048.0 MiB, 2.15 GB

Regards,

Ray Higgs
System z FCP Firmware Development
Bld. 706, B42
2455 South Road
Poughkeepsie, NY 12601
(845) 435-8666,  T/L 295-8666
rayhi...@us.ibm.com

Linux on 390 Port  wrote on 10/09/2012 07:32:19
PM:

> From: Thang Pham/Poughkeepsie/IBM@IBMUS
> To: LINUX-390@vm.marist.edu
> Date: 10/09/2012 07:40 PM
> Subject: SCSI/FCP disk size
> Sent by: Linux on 390 Port 
>
> Hello List,
>
> Is there a way to find out the size of a native SCSI device attached via
> FCP channel?  I do not see lszfcp or lsscsi having an option that lets
you
> see the size of the disk you have attached to a VM.
>
> Thanks,
> Thang Pham
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

SCSI/FCP disk size

[Thang Pham]

Hello List,

Is there a way to find out the size of a native SCSI device attached via
FCP channel?  I do not see lszfcp or lsscsi having an option that lets you
see the size of the disk you have attached to a VM.

Thanks,
Thang Pham

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP, multipath and LVM order at boot time

[Michael MacIsaac]

Bruce,

> If SUSE, did you also enable boot.multipath?
Yes SuSE, No I did not enable.  Yes, that fixed it. Thanks!

"Mike MacIsaac"(845) 433-7061

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP, multipath and LVM order at boot time

[Bruce Hayden]

SUSE or Red Hat?  If SUSE, did you also enable boot.multipath?  I
usually use "insserv" instead of chkconfig on SUSE, so enter "insserv
boot.multipath".

On Tue, Mar 27, 2012 at 3:04 PM, Michael MacIsaac  wrote:
> So I did not use the FCP devices at install time. After install, I added
> three LUNs, set up multipathing, set up an LVM and edited /etc/fstab so it
> is mounted over /opt.
>
> I did a "chkconfig multipathd on" and rebooted.  The LV did not mount. It
> seems multipathd ran *after* /etc/fstab was read. After logging on, if I
> then do a "mount /opt", it works fine
>
> Has anyone seen this?  Does multipathd perhaps need some additional
> "comments" in the insserv header so it starts earlier?
>
> Thanks.
>
> "Mike MacIsaac"    (845) 433-7061
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/



-- 
Bruce Hayden
z/VM and Linux on System z ATS
IBM, Endicott, NY

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

FCP, multipath and LVM order at boot time

[Michael MacIsaac]

So I did not use the FCP devices at install time. After install, I added
three LUNs, set up multipathing, set up an LVM and edited /etc/fstab so it
is mounted over /opt.

I did a "chkconfig multipathd on" and rebooted.  The LV did not mount. It
seems multipathd ran *after* /etc/fstab was read. After logging on, if I
then do a "mount /opt", it works fine

Has anyone seen this?  Does multipathd perhaps need some additional
"comments" in the insserv header so it starts earlier?

Thanks.

"Mike MacIsaac"(845) 433-7061

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP vs EDEV for system - best practice?

[Alan Altmark]

On Tuesday, 03/20/2012 at 10:33 EDT, Rick Troth  wrote:
> On Tue, Mar 20, 2012 at 9:56 AM, Alan Altmark 
wrote:
> > SCSI-only configurations require some consideration:
> > - You can't use the new Single System Image feature
> > - You can't share the RACF database (not even with other VM systems)
> ...
>
> I suspect that #1 is because XLINK support has not yet been ported to
> FBA.  Is that correct?  (I won't say anything about VM having
> supported FBA in other areas for thirty years.  Too late!)
>
> Gotta wonder if #2 is further fallout from the XLINK challenge.
> Someone should submit a req against this.

No, it's not about XLINK.  As Neale says, for SSI, CP uses atomic channel
programs to "compare and swap" on the volume that contains the Persistent
Data Record (PDR).  That logic isn't available for SCSI.

As far as RACF database sharing goes, there are three issues:
1.  Lack of RESERVE/RELEASE for virtual FBA.
2.  Lack of real RESERVE/RELEASE for SCSI in the SCSI command set CP must
use (advisory locking only, I believe).
3.  Use of CMS files to contain the database.  RACF support for FBA was
not written with the idea that the database would be shared.  Significant
RACF updates would be required, even after the CP restrictions were
lifted.

Alan Altmark

Senior Managing z/VM and Linux Consultant
IBM System Lab Services and Training
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP vs EDEV for system - best practice?

[Neale Ferguson]

No, they use an atomic CCW program to do work on the PDR.

On 3/20/12 10:28 AM, "Rick Troth"  wrote:

> On Tue, Mar 20, 2012 at 9:56 AM, Alan Altmark  wrote:
>> SCSI-only configurations require some consideration:
>> - You can't use the new Single System Image feature
>> - You can't share the RACF database (not even with other VM systems)
>  ...
> 
> I suspect that #1 is because XLINK support has not yet been ported to
> FBA.  Is that correct?  (I won't say anything about VM having
> supported FBA in other areas for thirty years.  Too late!)

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP vs EDEV for system - best practice?

[Rick Troth]

On Tue, Mar 20, 2012 at 9:56 AM, Alan Altmark  wrote:
> SCSI-only configurations require some consideration:
> - You can't use the new Single System Image feature
> - You can't share the RACF database (not even with other VM systems)
 ...

I suspect that #1 is because XLINK support has not yet been ported to
FBA.  Is that correct?  (I won't say anything about VM having
supported FBA in other areas for thirty years.  Too late!)

Gotta wonder if #2 is further fallout from the XLINK challenge.
Someone should submit a req against this.

-- R;   <><

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP vs EDEV for system - best practice?

[Alan Altmark]

On Tuesday, 03/20/2012 at 02:06 EDT, Grzegorz Powiedziuk
 wrote:
> I found a lot of very useful information here but this time I have some
> doubts so I decided to subscribe and ask.
>
> Recently we upgraded our hardware (z9 + DS6000 -> z114 + DS6000 + XIV)
and
> we have to make some choices.
> There is a chance that at some point we will have to get rid of DS6000
and
> we will have to store store everything on XIV.
> I know that it is not best choice for z/vm but DS is a pretty old guy
and
> maintaining it may become expensive.

I always suggest having both ECKD (for "z/VM data") and SCSI (for "Linux
data").  The ability to create large LUNs and the raw speed of data
transfer via FC make SCSI ideal for databases.

SCSI-only configurations require some consideration:
- You can't use the new Single System Image feature
- You can't share the RACF database (not even with other VM systems)
- z/OS can't provide any backups (FDRSOS with DS8000 can, but that's not
your config)
- No hyperswap capability
- Rudimentary performance instrumentation for the devices
- Increased CPU consumption for EDEVs

If you don't have any of those concerns, then SCSI-only is an easy choice.

Alan Altmark

Senior Managing z/VM and Linux Consultant
IBM System Lab Services and Training
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP vs EDEV for system - best practice?

[Richard Gasiorowski]

I applaud you for your decision to place the user DB data on FCP SCSI. Out
performs ECKD and allows you to have ALL LUN after formatting and inherent
multi pathing NOT the 70% or so.   the best part is that you now have made
zlinux less "mainframe" and more main stream.  Other platforms can now
access without having to go thru hoops. For far too long typical mainframe
bigots (i have been working on MF since IBM and my PSR days ala 1974) have
brandished and espoused MF technology in the distributed world.  In this
world to beat them you have to join them and play nicely in the sea of
data.  My experience in the z/VM owned DASD world is its really not a
necessity so long as you maintain PAV in the real disk mode.  Not a big
fan of PAV on mini disk for performance.  PLus what is left which would
reside on MDISK that needs that so called performance boost?

Richard (Gaz) Gasiorowski
Lead Architect UTC Global
CSC
3170 Fairview Park Dr., Falls Church, VA 22042
845-889-8533|Work|845-392-7889 Cell|rgasi...@csc.com|www.csc.com




This is a PRIVATE message. If you are not the intended recipient, please
delete without copying and kindly advise us by e-mail of the mistake in
delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind CSC to
any order or other contract unless pursuant to explicit written agreement
or government initiative expressly permitting the use of e-mail for such
purpose.



From:   Rob van der Heij 
To: LINUX-390@vm.marist.edu
Date:   03/20/2012 04:23 AM
Subject:        Re: FCP vs EDEV for system - best practice?
Sent by:Linux on 390 Port 



On Tue, Mar 20, 2012 at 7:04 AM, Grzegorz Powiedziuk
 wrote:

> Data disks for oracle I will build using direct attached FCP/SCSI. It is
> pretty easy to manage, multipathing is working fine - no doubts over
here.
> But what about linux system disks?
> I know that in general ECKD win over FCP.  But does FCP win over EDEV
when
> we are talking about linux system?

I am not sure "ECKD win over FCP" unless you talk about having
performance instrumentation in the control unit and channel...

Since you already have some data on FCP attached SCSI, you seem to
have mastered some of the issues related to that (NPIV, performance
instrumentation, etc). Some parts of that would be avoided when you
have ECKD or EDEV. For some installations there is a big advantage in
having access control in CP and RACF and space management in DIRMAINT.
As long as you're talking about moderate I/O rate, the extra CPU cost
of EDEV may be worth it. If you want to run your Linux systems
management processes from CMS, then being able to handle the disks
there is very attractive.

As for multi-pathing etc, I'm not sure that is a definite must-have
for everyone. I have seen several configurations where multi-pathing
and PAV made things slower. The issues to make this work well for real
life workload are not trivial. I'm currently looking at an ECKD device
that I can read at 250 MB/s (single device, no PAV). With a lot of
workloads this means disk I/O is not the limiting factor and you can
invest your time on other things.

--
Rob van der Heij
Velocity Software
http://www.velocitysoftware.com/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP vs EDEV for system - best practice?

[Rob van der Heij]

On Tue, Mar 20, 2012 at 10:47 AM, Philipp Kern  wrote:

> On Tue, Mar 20, 2012 at 09:15:11AM +0100, Rob van der Heij wrote:
> > As for multi-pathing etc, I'm not sure that is a definite must-have
> > for everyone. I have seen several configurations where multi-pathing
> > and PAV made things slower. The issues to make this work well for real
> > life workload are not trivial. I'm currently looking at an ECKD device
> > that I can read at 250 MB/s (single device, no PAV). With a lot of
> > workloads this means disk I/O is not the limiting factor and you can
> > invest your time on other things.
>
> Out of curiosity: That 250 MB/s is sequential read, not random read, right
> I found the advantage of PAV over single ECKD pretty severe with random
> read
> workloads.
>

Sure, sequential read or at least enough that we can do fairly large I/O's.
IIRC with 64K per I/O it dropped to half.

If you're doing 4K per I/O, the device pend time and dispatching delay will
become a significant factor. Last time I saw a real application do 4K I/O's
a small change in the application improved things by a factor of 10. YMMV.

I agree you can avoid some of that if you queue the I/O's in the control
unit, but you need quite a few moving parts aligned to let an application
take advantage of that. It's very well possible such a setup will have a
negative impact on other workload. When your focus is mixed workload rather
than single application throughput, there may be other aspects more
important.

Rob

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP vs EDEV for system - best practice?

[Philipp Kern]

On Tue, Mar 20, 2012 at 09:15:11AM +0100, Rob van der Heij wrote:
> As for multi-pathing etc, I'm not sure that is a definite must-have
> for everyone. I have seen several configurations where multi-pathing
> and PAV made things slower. The issues to make this work well for real
> life workload are not trivial. I'm currently looking at an ECKD device
> that I can read at 250 MB/s (single device, no PAV). With a lot of
> workloads this means disk I/O is not the limiting factor and you can
> invest your time on other things.

Out of curiosity: That 250 MB/s is sequential read, not random read, right
I found the advantage of PAV over single ECKD pretty severe with random read
workloads.

Kind regards,
Philipp Kern

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP vs EDEV for system - best practice?

[Rob van der Heij]

On Tue, Mar 20, 2012 at 7:04 AM, Grzegorz Powiedziuk
 wrote:

> Data disks for oracle I will build using direct attached FCP/SCSI. It is
> pretty easy to manage, multipathing is working fine - no doubts over here.
> But what about linux system disks?
> I know that in general ECKD win over FCP.  But does FCP win over EDEV when
> we are talking about linux system?

I am not sure "ECKD win over FCP" unless you talk about having
performance instrumentation in the control unit and channel...

Since you already have some data on FCP attached SCSI, you seem to
have mastered some of the issues related to that (NPIV, performance
instrumentation, etc). Some parts of that would be avoided when you
have ECKD or EDEV. For some installations there is a big advantage in
having access control in CP and RACF and space management in DIRMAINT.
As long as you're talking about moderate I/O rate, the extra CPU cost
of EDEV may be worth it. If you want to run your Linux systems
management processes from CMS, then being able to handle the disks
there is very attractive.

As for multi-pathing etc, I'm not sure that is a definite must-have
for everyone. I have seen several configurations where multi-pathing
and PAV made things slower. The issues to make this work well for real
life workload are not trivial. I'm currently looking at an ECKD device
that I can read at 250 MB/s (single device, no PAV). With a lot of
workloads this means disk I/O is not the limiting factor and you can
invest your time on other things.

-- 
Rob van der Heij
Velocity Software
http://www.velocitysoftware.com/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP vs EDEV for system - best practice?

[Philipp Kern]

On Tue, Mar 20, 2012 at 12:04:40AM -0600, Grzegorz Powiedziuk wrote:
> But what about linux system disks?
> I know that in general ECKD win over FCP.

I'd be not so sure about that.  You need at least a bunch of aliases on the
ECKD side to get into the same ballpark of speed.  (With FCP AIUI Linux can
just queue up multiple tagged I/O requests and the storage server will reply in
an order it sees fit.  Which is good for latency.)

> But does FCP win over EDEV when we are talking about linux system?

Are we just talking about speed or isolation/security?  FCP clearly wins with
the former, but loses with the latter.  EDEVs don't do aliases AFAIK and you
need FCP processing from CP, which isn't better than Linux's native support.

I'd be curious what options are available to limit a single VM to specific LUNs
when passing through the HBA.  I know that I could limit the LPAR through NPIV,
but single VMs?

Kind regards,
Philipp Kern

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

FCP vs EDEV for system - best practice?

[Grzegorz Powiedziuk]

Hello everyone.
I found a lot of very useful information here but this time I have some
doubts so I decided to subscribe and ask.

Recently we upgraded our hardware (z9 + DS6000 -> z114 + DS6000 + XIV) and
we have to make some choices.
There is a chance that at some point we will have to get rid of DS6000 and
we will have to store store everything on XIV.
I know that it is not best choice for z/vm but DS is a pretty old guy and
maintaining it may become expensive.

But anyway, to the point.
I am building linux guests under z/vm (right now 5.4 but it will change
soon).
Data disks for oracle I will build using direct attached FCP/SCSI. It is
pretty easy to manage, multipathing is working fine - no doubts over here.
But what about linux system disks?
I know that in general ECKD win over FCP.  But does FCP win over EDEV when
we are talking about linux system?

I've already encountered some problems with direct attached FCP/SCSI. Main
one is multipathing when you want to use a clone/snapshot. WWID for scsi
device changes and multipathing brakes.
I can work around this but who knows If I am gonna be able to do this in 1
year from know, under time pressure when something happens.
In case like this I have to boot a clone without multipathing (option
multipath=off in zipl), update fstab, zipl with new wwid , reboot few times
etc, boot updated guest with multipath. A little dirty but works.

Now I am looking into EDEV. I know it can impact performance but it is
'just' a system drive. We don't need a lot of iops here right?

So the question is : should I stick to FCP/SCSI as much as possible or just
go with EDEV for my linux system disks (and put oracle on FCP direct
attached).

And I also would like to ask - what is difference between attributes "XIV"
and "SCSI" in "EDEVICE" statement? Both are scsci over fiber so why there
is an extra one for XIV?

Thanks!
Grzegorz Powiedziuk

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: EMC fcp problem

[Victor Echavarry Diaz]

Mark:
You're right is IPL the old one. For some reason when update the service pack, 
the IPL disk change from 150 to 151. Is there a way to change it on the 
mkinitrd or the zipl?
Regards,
V�ctor Echavarry
System Programmer
Technology Systems & Operations Division
EVERTEC



-Original Message-
From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of Mark Post
Sent: Thursday, September 08, 2011 4:34 PM
To: LINUX-390@VM.MARIST.EDU
Subject: Re: EMC fcp problem

>>> On 9/8/2011 at 03:40 PM, Victor Echavarry Diaz 
wrote: 
> Mark:
> I begin the update and when I reboot, the system give the following message
> FATAL: Could not load /lib/modules/2.6.16.60-0.21-default/modules.dep: No 
> such f
> ile or directory 
> 
> The module lib is /lib/modules/2.6.16.60-0.42.10-default how I solve this.

Did you boot the right kernel?  It looks like you booted the old one.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



-
CONFIDENTIALITY NOTE: This email communication and its attachments
contain information that are proprietary and confidential to
EVERTEC, INC., its affiliates or its clients.  They may not be
disclosed, distributed, used, copied or modified in any way without
EVERTEC, Inc.’s authorization. If you are not the intended
recipient of this email, you are not an authorized person.  Please
delete it and notify the sender immediately. EVERTEC, Inc. and its
affiliates do not assume any liability for damages resulting from
emails that have been sent or altered without their consent.
Moreover, EVERTEC, Inc. has taken precautions to safeguard its
email communications, but cannot assure that such is the case and
disclaim any responsibility attributable thereto.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: EMC fcp problem

[Mark Post]

>>> On 9/8/2011 at 04:55 PM, Victor Echavarry Diaz 
wrote: 
> Mark:
> You're right is IPL the old one. For some reason when update the service 
> pack, the IPL disk change from 150 to 151. Is there a way to change it on the 
> mkinitrd or the zipl?

What volume is the "IPL volume" is determined by this entry in /etc/zipl.conf:
:menu
target = /boot/zipl

For that to somehow change between device numbers sounds like /boot was not 
mounted when the kernel was installed.  If /boot is mounted now, just 
re-running zipl should correct it.  Or, if /boot is _not_ normally mounted (for 
whatever reason), then make sure /boot is not mounted and re-run zipl.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: EMC fcp problem

[Victor Echavarry Diaz]

cat /etc/zipl.conf  
# Modified by YaST2. Last modification on Thu Sep  8 16:19:41 AST 2011  
[defaultboot]   
defaultmenu = menu  

###Don't change this comment - YaST2 identifier: Original name: linux###
[SLES_10_SP3]   
image = /boot/image-2.6.16.60-0.54.5-default
target = /boot/zipl 
ramdisk = /boot/initrd-2.6.16.60-0.54.5-default,0x400   
parameters = "root=/dev/dasdb1 TERM=dumb possible_cpus=4"   


:menu   
default = 1 
prompt = 1  
target = /boot/zipl 
timeout = 10
1 = SLES_10_SP3 
2 = ipl  
 
###Don't change this comment - YaST2 identifier: Original name: ipl###   
[ipl]
image = /boot/image  
target = /boot/zipl  
ramdisk = /boot/initrd,0x100 
parameters = "root=/dev/dasdb1   TERM=dumb possible_cpus=4"  

V�ctor Echavarry 
System Programmer
Technology Systems & Operations Division
EVERTEC



-Original Message-
From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of Mark Post
Sent: Thursday, September 08, 2011 4:50 PM
To: LINUX-390@VM.MARIST.EDU
Subject: Re: EMC fcp problem

>>> On 9/8/2011 at 04:38 PM, Victor Echavarry Diaz 
wrote: 
> Is pointing to the new one
> [SLES_10_SP3]
>
> image = /boot/image-2.6.16.60-0.54.5-default 
>
> target = /boot/zipl  
>
> ramdisk = /boot/initrd-2.6.16.60-0.54.5-default,0x400
>
> parameters = "root=/dev/dasdb1 TERM=dumb possible_cpus=4"

Is that they only section in zipl.conf?


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



-
CONFIDENTIALITY NOTE: This email communication and its attachments
contain information that are proprietary and confidential to
EVERTEC, INC., its affiliates or its clients.  They may not be
disclosed, distributed, used, copied or modified in any way without
EVERTEC, Inc.’s authorization. If you are not the intended
recipient of this email, you are not an authorized person.  Please
delete it and notify the sender immediately. EVERTEC, Inc. and its
affiliates do not assume any liability for damages resulting from
emails that have been sent or altered without their consent.
Moreover, EVERTEC, Inc. has taken precautions to safeguard its
email communications, but cannot assure that such is the case and
disclaim any responsibility attributable thereto.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: EMC fcp problem

[Mark Post]

>>> On 9/8/2011 at 04:38 PM, Victor Echavarry Diaz 
wrote: 
> Is pointing to the new one
> [SLES_10_SP3]
>
> image = /boot/image-2.6.16.60-0.54.5-default 
>
> target = /boot/zipl  
>
> ramdisk = /boot/initrd-2.6.16.60-0.54.5-default,0x400
>
> parameters = "root=/dev/dasdb1 TERM=dumb possible_cpus=4"

Is that they only section in zipl.conf?


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: EMC fcp problem

[Victor Echavarry Diaz]

Is pointing to the new one
[SLES_10_SP3]   
image = /boot/image-2.6.16.60-0.54.5-default
target = /boot/zipl 
ramdisk = /boot/initrd-2.6.16.60-0.54.5-default,0x400   
parameters = "root=/dev/dasdb1 TERM=dumb possible_cpus=4"   

V�ctor Echavarry
System Programmer
Technology Systems & Operations Division
EVERTEC



-Original Message-
From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of Mark Post
Sent: Thursday, September 08, 2011 4:34 PM
To: LINUX-390@VM.MARIST.EDU
Subject: Re: EMC fcp problem

>>> On 9/8/2011 at 03:40 PM, Victor Echavarry Diaz 
wrote: 
> Mark:
> I begin the update and when I reboot, the system give the following message
> FATAL: Could not load /lib/modules/2.6.16.60-0.21-default/modules.dep: No 
> such f
> ile or directory 
> 
> The module lib is /lib/modules/2.6.16.60-0.42.10-default how I solve this.

Did you boot the right kernel?  It looks like you booted the old one.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



-
CONFIDENTIALITY NOTE: This email communication and its attachments
contain information that are proprietary and confidential to
EVERTEC, INC., its affiliates or its clients.  They may not be
disclosed, distributed, used, copied or modified in any way without
EVERTEC, Inc.’s authorization. If you are not the intended
recipient of this email, you are not an authorized person.  Please
delete it and notify the sender immediately. EVERTEC, Inc. and its
affiliates do not assume any liability for damages resulting from
emails that have been sent or altered without their consent.
Moreover, EVERTEC, Inc. has taken precautions to safeguard its
email communications, but cannot assure that such is the case and
disclaim any responsibility attributable thereto.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: EMC fcp problem

[Mark Post]

>>> On 9/8/2011 at 03:40 PM, Victor Echavarry Diaz 
wrote: 
> Mark:
> I begin the update and when I reboot, the system give the following message
> FATAL: Could not load /lib/modules/2.6.16.60-0.21-default/modules.dep: No 
> such f
> ile or directory 
> 
> The module lib is /lib/modules/2.6.16.60-0.42.10-default how I solve this.

Did you boot the right kernel?  It looks like you booted the old one.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: EMC fcp problem

[Victor Echavarry Diaz]

Mark:
I begin the update and when I reboot, the system give the following message
FATAL: Could not load /lib/modules/2.6.16.60-0.21-default/modules.dep: No such f
ile or directory 

The module lib is /lib/modules/2.6.16.60-0.42.10-default how I solve this.
Regards. 
V�ctor Echavarry
vechava...@evertecinc.com
System Programmer
Technology Systems & Operations Division
EVERTEC
787-759- Ext. 3401



-Original Message-
From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of Mark Post
Sent: Thursday, September 08, 2011 1:35 PM
To: LINUX-390@VM.MARIST.EDU
Subject: Re: EMC fcp problem

>>> On 9/8/2011 at 11:13 AM, Victor Echavarry Diaz 
wrote: 
> Every time we try to define a LVM at the mapper its cancel.

You shouldn't be trying to use /dev/mapper, you should be using 
/dev/disk/by-id/.

> The Box is an EMC CLARiiON VMX 5700. We are running SLES 10SP2. Any 
> suggestions to solve this?

Try using a supported version of the operating system?  Say SLES10 SP4 or 
SLES11 SP1?  There have been an awful lot of zFCP changes and fixes since SP2 
was out of service.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



-
CONFIDENTIALITY NOTE: This email communication and its attachments
contain information that are proprietary and confidential to
EVERTEC, INC., its affiliates or its clients.  They may not be
disclosed, distributed, used, copied or modified in any way without
EVERTEC, Inc.’s authorization. If you are not the intended
recipient of this email, you are not an authorized person.  Please
delete it and notify the sender immediately. EVERTEC, Inc. and its
affiliates do not assume any liability for damages resulting from
emails that have been sent or altered without their consent.
Moreover, EVERTEC, Inc. has taken precautions to safeguard its
email communications, but cannot assure that such is the case and
disclaim any responsibility attributable thereto.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: EMC fcp problem

[Mark Post]

>>> On 9/8/2011 at 11:13 AM, Victor Echavarry Diaz 
wrote: 
> Every time we try to define a LVM at the mapper its cancel.

You shouldn't be trying to use /dev/mapper, you should be using 
/dev/disk/by-id/.

> The Box is an EMC CLARiiON VMX 5700. We are running SLES 10SP2. Any 
> suggestions to solve this?

Try using a supported version of the operating system?  Say SLES10 SP4 or 
SLES11 SP1?  There have been an awful lot of zFCP changes and fixes since SP2 
was out of service.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

EMC fcp problem

[Victor Echavarry Diaz]

We are installing a FCP on a zLinux guest. When we bring up the fcp with 
multipath we get the following error messages
SCSI device sdc: drive cache: write through
 sdc:<5>  Vendor: DGC   Model: VRAID Rev: 0531
  Type:   Direct-Access  ANSI SCSI revision: 04
 sdc1
sd 0:0:0:3: Attached scsi disk sdc
SCSI device sdd: 60817408 512-byte hdwr sectors (31139 MB)
sdd: Write Protect is off
  Vendor: DGC   Model: VRAID Rev: 0531
  Type:   Direct-Access  ANSI SCSI revision: 04
SCSI device sdd: drive cache: write through
SCSI device sdd: 60817408 512-byte hdwr sectors (31139 MB)
sdd: Write Protect is off
SCSI device sdd: drive cache: write through
 sdd: sdd1
Starting slpd ..done
audit(1315493425.823:3): audit_pid=3897 old=0 by auid=4294967295
Sep 08 10:50:25 | sdbl: emc prio: path not correctly configured for failover
Sep 08 10:50:25 | sdbk: emc prio: path not correctly configured for failover
Sep 08 10:50:25 | sdbj: emc prio: path not correctly configured for failover
Sep 08 10:50:26 | sda: emc prio: path not correctly configured for failover
device-mapper: multipath emc: version 0.0.4 loaded
device-mapper: multipath emc: long trespass command will be send
device-mapper: multipath emc: honor reservation bit will not be set (default)
device-mapper: multipath round-robin: version 1.0.0 loaded
device-mapper: multipath emc: long trespass command will be send
device-mapper: multipath emc: honor reservation bit will not be set (default)
device-mapper: multipath emc: emc_pg_init: sending switch-over command
device-mapper: multipath emc: long trespass command will be send
device-mapper: multipath emc: honor reservation bit will not be set (default)
device-mapper: multipath emc: long trespass command will be send
device-mapper: multipath emc: honor reservation bit will not be set (default)
device-mapper: multipath emc: emc_pg_init: sending switch-over command
device-mapper: multipath emc: honor reservation bit will not be set (default)
device-mapper: multipath: Failing path 66:128.
device-mapper: multipath: Failing path 65:224.
device-mapper: multipath: Failing path 65:48.
device-mapper: multipath: Failing path 8:112.
device-mapper: multipath: Failing path 67:240.
device-mapper: multipath: Failing path 67:192.
device-mapper: multipath: Failing path 67:16.
device-mapper: multipath: Failing path 66:96.
device-mapper: multipath: Failing path 67:224.
device-mapper: multipath: Failing path 67:176.
device-mapper: multipath: Failing path 66:224.
device-mapper: multipath: Failing path 66:80.
t will not be set (default)
Sep  8 10:50:26 l212t kernel: device-mapper: multipath emc: long trespass comman
d will be send
Sep  8 10:50:26 l212t kernel: device-mapper: multipath emc: honor reservation bi
t will not be set (default)
Sep  8 10:50:26 l212t kernel: device-mapper: multipath emc: long trespass comman
d will be send
Sep  8 10:50:26 l212t kernel: device-mapper: multipath emc: honor reservation bi
t will not be set (default)

multipath -ll

sdt: emc prio: path not correctly configured for failover
sdh: emc prio: path not correctly configured for failover
sdbl: emc prio: path not correctly configured for failover
sdbi: emc prio: path not correctly configured for failover
sdax: emc prio: path not correctly configured for failover
sdam: emc prio: path not correctly configured for failover
36006016074c02c00ea4f59cc6fd5e011 dm-9 DGC,VRAID
[size=1.0G][features=1 queue_if_no_path][hwhandler=1 emc]
\_ round-robin 0 [prio=0][enabled]
 \_ 0:0:1:8   sdao  66:128 [failed][faulty]
 \_ 1:0:1:8   sdae  65:224 [failed][faulty]
 \_ 1:0:0:8   sdt   65:48  [failed][faulty]
 \_ 0:0:0:8   sdh   8:112  [failed][faulty]
\_ round-robin 0 [prio=0][enabled]
 \_ 1:0:3:8   sdbl  67:240 [failed][faulty]
 \_ 0:0:3:8   sdbi  67:192 [failed][faulty]
 \_ 0:0:2:8   sdax  67:16  [failed][faulty]
 \_ 1:0:2:8   sdam  66:96  [failed][faulty]
sdbd: emc prio: path not correctly configured for failover
sdaz: emc prio: path not correctly configured for failover

ls -l /dev/mapper
total 0
brw--- 1 root root 253, 12 Sep  8 10:50 36006016074c02c0016b57e1570d5e011
brw--- 1 root root 253, 11 Sep  8 10:50 36006016074c02c00263f5d526fd5e011
brw--- 1 root root 253, 14 Sep  8 10:50 36006016074c02c004a54a8e16fd5e011
brw--- 1 root root 253, 16 Sep  8 10:50 36006016074c02c006e68a0082fd3e011
brw--- 1 root root 253, 15 Sep  8 10:50 36006016074c02c007068a0082fd3e011
brw--- 1 root root 253, 13 Sep  8 10:50 36006016074c02c007468a0082fd3e011
brw--- 1 root root 253, 10 Sep  8 10:50 36006016074c02c0096a342c06fd5e011
brw--- 1 root root 253,  9 Sep  8 10:50 36006016074c02c00ea4f59cc6fd5e011
lrwxrwxrwx 1 root root  16 Sep  8 10:48 control -> ../device-mapper
brw--- 1 root root 253,  0 Sep  8 10:49 oracledb-oraclevol

lszfcp
0.0.754c host0
0.0.750c host1
l212t:~ # lszfcp -D
lszfcp -D
0.0.754c/0x5006016046e05b86/0x0001 0:0:0:1
0.0.754c/0x5006016046e05b86/0x0002000

Re: Using FCP for RHEL 5.6 kickstart install (solved/work around)

[Donald Russell]

On Mon, Apr 25, 2011 at 22:17, Donald Russell  wrote:

> When I install RHEL on FCP/SCSI if I specify only FCP_1 in the CMS file
> parm file everything goes fine. When I specify a second (or more) path
> (FCP_2 etc) everything goes fine right up until the reboot after installing
> everything.
>
> At that point I get CP Interrupt loop detected, and it goes to CP READ.
>



The Reader's Digest version of this is I put /boot on a 3390 minidisk, and /
on mapper/mpath0 and specified all the paths on the FCP_n= statements in the
parm file.

Kickstart was happy, and the firstboot worked correctly.

/boot cannot be on an mpath device. This changes in RHEL 6 apparently, but
I've not had the opportunity to try that yet.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Using FCP for RHEL 5.6 kickstart install

[Donald Russell]

When I install RHEL on FCP/SCSI if I specify only FCP_1 in the CMS file parm
file everything goes fine. When I specify a second (or more) path (FCP_2
etc) everything goes fine right up until the reboot after installing
everything.

At that point I get CP Interrupt loop detected, and it goes to CP READ.

What I'm hearing now is that zipl does not support having the boot partition
on an mpath device... is that true? If so, I think I'll just create an
EDEVICE for than LUN and define all the paths in VM.

But, can anybody point me to some (current) doc that describes how to
setup/perform a kickstart install on FCP using multiple paths, and have a
working system at the end. ;-)

Sorry if I sound a little terse, I've just lost a lot of time due to
out-of-date and/or incorrect documentation. (Yes, I have cases open with Red
Hat and they are in the process of updating/correcting the doc)

So, I'm hoping somebody has blazed this trail before me and can share some
insight. :-)

Cheers

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP port closed [SOLVED]

[Rafal Hanzel]

Hi,

We solved our problem. Storage admin brought a borrowed switch and we 
made connection with switch. Everything works fine.


But first we had strange behaviour with this working connection. When we 
pull out cable and put on back, connection stop working. (It was some 
kind of instance that it worked, and therefore we did these all attempts).


Storage admin told me that configuration for point-to-point on EMC 
Storage System is exactly the same as for switch.
If it is true it is weird for me that it works with switch but 
point-to-point doesn't work.


thanks to all for your help.

Pozdrawiam/Best regards,

Rafał Hanzel
Programista systemowy, Dział Badań i Rozwoju Systemów Komputerowych
Tel. +48 32 3589246, Fax +48 32 3589277, email: hanz...@zetokatowice.pl
Tel. kom. +48 501677656
Zakład Elektronicznej Techniki Obliczeniowej w Katowicach Spółka z o.o.
40-158 Katowice, ul. Owocowa 1
Sąd Rejonowy Katowice - Wschód w Katowicach Wydział VIII Gospodarczy 
Krajowego Rejestru Sądowego, KRS 051477

Kapitał zakładowy: 264500 zł
NIP 634-013-11-06
http://www.zetokatowice.pl

Jesteśmy uczestnikiem Programu
RZETELNA Firma
Sprawdź naszą rzetelność na
http://www.rzetelnafirma.pl/8FGKEVFH

Treść tej informacji może być poufna, w związku z czym powinna trafić 
bezpośrednio do rąk adresata. Jakiekolwiek jej ujawnianie, 
rozpowszechnianie, bądź kopiowanie jest zabronione. W przypadku 
omyłkowego otrzymania niniejszej informacji prosimy o poinformowanie 
nadawcy i usunięcie jej z komputera.



W dniu 2010-11-19 17:10, Raymond Higgs pisze:

Rafal,

Recently, we got an EMC V-max here in Poughkeepsie for testing.  We had to
do special configuration on the V-max to get point-to-point to work. Maybe
one of the ports on the V-max is set up for point-to-point, and the other
port isn't?

Regards,

Ray Higgs
System z FCP Development
Bld. 706, B24
2455 South Road
Poughkeepsie, NY 12601
(845) 435-8666,  T/L 295-8666
rayhi...@us.ibm.com



Rafal Hanzel
Sent by: Linux on 390 Port
11/19/2010 03:27 AM
Please respond to
Linux on 390 Port


To
LINUX-390@vm.marist.edu
cc

Subject
Re: FCP port closed



--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP port closed

[Raymond Higgs]

Rafal,

Recently, we got an EMC V-max here in Poughkeepsie for testing.  We had to 
do special configuration on the V-max to get point-to-point to work. Maybe 
one of the ports on the V-max is set up for point-to-point, and the other 
port isn't?

Regards,

Ray Higgs
System z FCP Development
Bld. 706, B24
2455 South Road
Poughkeepsie, NY 12601
(845) 435-8666,  T/L 295-8666
rayhi...@us.ibm.com



Rafal Hanzel  
Sent by: Linux on 390 Port 
11/19/2010 03:27 AM
Please respond to
Linux on 390 Port 


To
LINUX-390@vm.marist.edu
cc

Subject
Re: FCP port closed






Thanks to all for your response

Today storage admin has day off .. we are going to try all your 
suggestions tomorrow.

Answering your questions:
RH:
 > Which machine?  z9?  z10?  z196?  etc
z9

 > Which channel hardware?  Ficon Express 8?  4?  2?
FICON Express4 4 KM LX (Feature codes 3324)

 > Which operating system?
z/VM 5.4 + zLinux (SUSE)

 > Which topology?  switched?  point-to-point?
point-to-point

 > Which storage?
EMC


JB:
Are the link lights on?
IIRC yes .. but to be sure I'm going to see it tomorrow.

Is the fiber known to be good?
IIRC yes, Storage admin tried different cables .. but of course I'm 
going to try it tomorrow again.

Do the ports at both ends pass a wrap test?
hmm.. I don't know about any tests ... we will do it tomorrow.


Thanks one more time.

Pozdrawiam/Best regards,

Rafał Hanzel
Programista systemowy, Dział Badań i Rozwoju Systemów Komputerowych
Tel. +48 32 3589246, Fax +48 32 3589277, email: hanz...@zetokatowice.pl
Tel. kom. +48 501677656
Zakład Elektronicznej Techniki Obliczeniowej w Katowicach Spółka z o.o.
40-158 Katowice, ul. Owocowa 1
Sąd Rejonowy Katowice - Wschód w Katowicach Wydział VIII Gospodarczy 
Krajowego Rejestru Sądowego, KRS 051477
Kapitał zakładowy: 264500 zł
NIP 634-013-11-06
http://www.zetokatowice.pl

Jesteśmy uczestnikiem Programu
RZETELNA Firma
Sprawdź naszą rzetelność na
http://www.rzetelnafirma.pl/8FGKEVFH

Treść tej informacji może być poufna, w związku z czym powinna trafić 
bezpośrednio do rąk adresata. Jakiekolwiek jej ujawnianie, 
rozpowszechnianie, bądź kopiowanie jest zabronione. W przypadku 
omyłkowego otrzymania niniejszej informacji prosimy o poinformowanie 
nadawcy i usunięcie jej z komputera.


W dniu 2010-11-18 18:25, Raymond Higgs pisze:
> Rafal,
>
> This is not a problem with mixing long wave and short wave components.
> Such problems would prevent word sync, speed negotiation and fabric 
login.
>   PLOGI is a fibre channel ELS that happens afterward.
>
> There still might be a physical issue.  Maybe some cable is plugged in 
the
> wrong spot?  Maybe the channel is plugged into the wrong switch?
>
> I agree with the others.  Your IOCDS is ok.  You would have had QDIO
> establish/activate problems if something was wrong there.
>
> I've been discussing your problem with another developer here.  We tried
> to come up with a list of things to check, and ended up generating more
> question then anything else... Could you tell us more about your
> configuration?
>
> Which machine?  z9?  z10?  z196?  etc
> Which channel hardware?  Ficon Express 8?  4?  2?
> Which operating system?
> Which topology?  switched?  point-to-point?
> Which storage?
>
> I think the operating system should generate an error message somewhere
> with details about open port failing.  Fibre channel defines 2 bytes 
worth
> of reason code and reason code explanations.  The channel gives this 
info
> to the operating system when PLOGI fails.  These bits would be very
> helpful.
>
> If you are switched, check zoning.  We're not too sure this is the 
problem
> because we don't know which switch you might be using.  If it is zoning,
> it seems odd that the name server in the switch would tell you the 
n-port
> id if the target port wasn't in your zone.
>
> We have the most experience with IBM storage.  When host connections are
> wrong, other steps after PLOGI fail.  So we aren't sure that it is a 
host
> connection/lun masking problem.
>
> Most switches and storage arrays have login limits.  Could those be set
> too low?
>
> I hope this helps.
>
> Ray Higgs
> System z FCP Development
> Bld. 706, B24
> 2455 South Road
> Poughkeepsie, NY 12601
> (845) 435-8666,  T/L 295-8666
> rayhi...@us.ibm.com

>
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or 
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/


-

Re: FCP port closed

[Rafal Hanzel]

Thanks to all for your response

Today storage admin has day off .. we are going to try all your 
suggestions tomorrow.


Answering your questions:
RH:
> Which machine?  z9?  z10?  z196?  etc
z9

> Which channel hardware?  Ficon Express 8?  4?  2?
FICON Express4 4 KM LX (Feature codes 3324)

> Which operating system?
z/VM 5.4 + zLinux (SUSE)

> Which topology?  switched?  point-to-point?
point-to-point

> Which storage?
EMC


JB:
Are the link lights on?
IIRC yes .. but to be sure I'm going to see it tomorrow.

Is the fiber known to be good?
IIRC yes, Storage admin tried different cables .. but of course I'm 
going to try it tomorrow again.


Do the ports at both ends pass a wrap test?
hmm.. I don't know about any tests ... we will do it tomorrow.


Thanks one more time.

Pozdrawiam/Best regards,

Rafał Hanzel
Programista systemowy, Dział Badań i Rozwoju Systemów Komputerowych
Tel. +48 32 3589246, Fax +48 32 3589277, email: hanz...@zetokatowice.pl
Tel. kom. +48 501677656
Zakład Elektronicznej Techniki Obliczeniowej w Katowicach Spółka z o.o.
40-158 Katowice, ul. Owocowa 1
Sąd Rejonowy Katowice - Wschód w Katowicach Wydział VIII Gospodarczy 
Krajowego Rejestru Sądowego, KRS 051477

Kapitał zakładowy: 264500 zł
NIP 634-013-11-06
http://www.zetokatowice.pl

Jesteśmy uczestnikiem Programu
RZETELNA Firma
Sprawdź naszą rzetelność na
http://www.rzetelnafirma.pl/8FGKEVFH

Treść tej informacji może być poufna, w związku z czym powinna trafić 
bezpośrednio do rąk adresata. Jakiekolwiek jej ujawnianie, 
rozpowszechnianie, bądź kopiowanie jest zabronione. W przypadku 
omyłkowego otrzymania niniejszej informacji prosimy o poinformowanie 
nadawcy i usunięcie jej z komputera.



W dniu 2010-11-18 18:25, Raymond Higgs pisze:

Rafal,

This is not a problem with mixing long wave and short wave components.
Such problems would prevent word sync, speed negotiation and fabric login.
  PLOGI is a fibre channel ELS that happens afterward.

There still might be a physical issue.  Maybe some cable is plugged in the
wrong spot?  Maybe the channel is plugged into the wrong switch?

I agree with the others.  Your IOCDS is ok.  You would have had QDIO
establish/activate problems if something was wrong there.

I've been discussing your problem with another developer here.  We tried
to come up with a list of things to check, and ended up generating more
question then anything else... Could you tell us more about your
configuration?

Which machine?  z9?  z10?  z196?  etc
Which channel hardware?  Ficon Express 8?  4?  2?
Which operating system?
Which topology?  switched?  point-to-point?
Which storage?

I think the operating system should generate an error message somewhere
with details about open port failing.  Fibre channel defines 2 bytes worth
of reason code and reason code explanations.  The channel gives this info
to the operating system when PLOGI fails.  These bits would be very
helpful.

If you are switched, check zoning.  We're not too sure this is the problem
because we don't know which switch you might be using.  If it is zoning,
it seems odd that the name server in the switch would tell you the n-port
id if the target port wasn't in your zone.

We have the most experience with IBM storage.  When host connections are
wrong, other steps after PLOGI fail.  So we aren't sure that it is a host
connection/lun masking problem.

Most switches and storage arrays have login limits.  Could those be set
too low?

I hope this helps.

Ray Higgs
System z FCP Development
Bld. 706, B24
2455 South Road
Poughkeepsie, NY 12601
(845) 435-8666,  T/L 295-8666
rayhi...@us.ibm.com







--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP port closed

[Raymond Higgs]

Rafal,

This is not a problem with mixing long wave and short wave components.
Such problems would prevent word sync, speed negotiation and fabric login.
 PLOGI is a fibre channel ELS that happens afterward.

There still might be a physical issue.  Maybe some cable is plugged in the
wrong spot?  Maybe the channel is plugged into the wrong switch?

I agree with the others.  Your IOCDS is ok.  You would have had QDIO
establish/activate problems if something was wrong there.

I've been discussing your problem with another developer here.  We tried
to come up with a list of things to check, and ended up generating more
question then anything else... Could you tell us more about your
configuration?

Which machine?  z9?  z10?  z196?  etc
Which channel hardware?  Ficon Express 8?  4?  2?
Which operating system?
Which topology?  switched?  point-to-point?
Which storage?

I think the operating system should generate an error message somewhere
with details about open port failing.  Fibre channel defines 2 bytes worth
of reason code and reason code explanations.  The channel gives this info
to the operating system when PLOGI fails.  These bits would be very
helpful.

If you are switched, check zoning.  We're not too sure this is the problem
because we don't know which switch you might be using.  If it is zoning,
it seems odd that the name server in the switch would tell you the n-port
id if the target port wasn't in your zone.

We have the most experience with IBM storage.  When host connections are
wrong, other steps after PLOGI fail.  So we aren't sure that it is a host
connection/lun masking problem.

Most switches and storage arrays have login limits.  Could those be set
too low?

I hope this helps.

Ray Higgs
System z FCP Development
Bld. 706, B24
2455 South Road
Poughkeepsie, NY 12601
(845) 435-8666,  T/L 295-8666
rayhi...@us.ibm.com

Linux on 390 Port  wrote on 11/18/2010 10:29:05
AM:

> John Schnitzler Jr/Endicott/i...@ibmus
> Sent by: Linux on 390 Port 
>
> 11/18/2010 10:29 AM
>
> Please respond to
> Linux on 390 Port 
>
> To
>
> LINUX-390@vm.marist.edu
>
> cc
>
> Subject
>
> Re: FCP port closed
>
> >Storage admin told me that both configurations (for PCHID 111 i 113)
are
> >exactly the same... so I must believe him 
> >
> >but maybe I convince him to check it again ...
> >Is it rather a storage problem or a system z box or OS ??
> >
>
> The IOCP entry looks fine.  I would suspect a connection problem between
> the FCP card and the switch GBIC or HBA on the storage device. First
make
> sure your cabling is correct. Make sure you are making and LX to LX
> connection or an SX to SX connection. Check the status lights on the
switch
> GBIC card or the HBA to make sure the connection is good.
>
> Regards,
> John
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP port closed

[John Schnitzler Jr]

>Storage admin told me that both configurations (for PCHID 111 i 113) are
>exactly the same... so I must believe him 
>
>but maybe I convince him to check it again ...
>Is it rather a storage problem or a system z box or OS ??
>

The IOCP entry looks fine.  I would suspect a connection problem between
the FCP card and the switch GBIC or HBA on the storage device. First make
sure your cabling is correct. Make sure you are making and LX to LX
connection or an SX to SX connection. Check the status lights on the switch
GBIC card or the HBA to make sure the connection is good.

Regards,
John

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP port closed

[Robert J Brenneman]

If the HMC Is reporting this connectivity issue it has nothing to do
with the OS.
Your IOCP deck looks fine, so the hardware is configured properly on
your end, unless you made a change to the IOCP and forgot to activate
/ POR that change.

Are the link lights on? Is the fiber known to be good? Do the ports at
both ends pass a wrap test?

--
Jay Brenneman

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP port closed

[Rafal Hanzel]

Storage admin told me that both configurations (for PCHID 111 i 113) are 
exactly the same... so I must believe him 


but maybe I convince him to check it again ...
Is it rather a storage problem or a system z box or OS ??

Pozdrawiam/Best regards,

Rafał Hanzel
Programista systemowy, Dział Badań i Rozwoju Systemów Komputerowych
Tel. +48 32 3589246, Fax +48 32 3589277, email: hanz...@zetokatowice.pl
Tel. kom. +48 501677656
Zakład Elektronicznej Techniki Obliczeniowej w Katowicach Spółka z o.o.
40-158 Katowice, ul. Owocowa 1
Sąd Rejonowy Katowice - Wschód w Katowicach Wydział VIII Gospodarczy 
Krajowego Rejestru Sądowego, KRS 051477

Kapitał zakładowy: 264500 zł
NIP 634-013-11-06
http://www.zetokatowice.pl

Jesteśmy uczestnikiem Programu
RZETELNA Firma
Sprawdź naszą rzetelność na
http://www.rzetelnafirma.pl/8FGKEVFH

Treść tej informacji może być poufna, w związku z czym powinna trafić 
bezpośrednio do rąk adresata. Jakiekolwiek jej ujawnianie, 
rozpowszechnianie, bądź kopiowanie jest zabronione. W przypadku 
omyłkowego otrzymania niniejszej informacji prosimy o poinformowanie 
nadawcy i usunięcie jej z komputera.



W dniu 2010-11-18 15:01, Robert J Brenneman pisze:

Obviously something is not the same...


Ask the storage admin to verify that the subsystem port that pchid 113
is connected to is configured as an open systems or FCP port - not a
FICON port.


--
Jay Brenneman

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/




--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: FCP port closed

[Robert J Brenneman]

Obviously something is not the same...


Ask the storage admin to verify that the subsystem port that pchid 113
is connected to is configured as an open systems or FCP port - not a
FICON port.


--
Jay Brenneman

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

FCP port closed

[Rafal Hanzel]

Hi all

I tried on the IBM-MAIN and I thought try here and IBMVM.

I don't know if I explain it correctly but I will try.

I have 2 of 4 Ficon card ports configured as a FCP
It's part of my IOCP:
CHPID PATH=(CSS(0),23),SHARED,PARTITION=((LPARVM),(=)),   *
   PCHID=111,TYPE=FCP
.
CHPID PATH=(CSS(0),25),SHARED,PARTITION=((LPARVM),(=)),   *
   PCHID=113,TYPE=FCP

CNTLUNIT CUNUMBR=0054,PATH=((CSS(0),23)),UNIT=FCP
IODEVICE ADDRESS=(5100,240),CUNUMBR=(0054),UNIT=FCP
CNTLUNIT CUNUMBR=0055,PATH=((CSS(0),25)),UNIT=FCP
IODEVICE ADDRESS=(5200,240),CUNUMBR=(0055),UNIT=FCP


To PCHID 111 we have attached storage system and to PCHID 113 another
storage system. Storage administrator tell me that configurations on
both are the same.

But we have some problems with one of them. On HMC panels CHANNELS -->
ANALYZE SERIAL LINK STATUS I see:
for PCHID 111
 Device   Subchannel
 Number  SetS_ID  Target WWPND_IDStatus
  51000   0062562B  5006048c49ae549c 00AE549C   Port Open
and It works perfectly

for PCHID 113
Device   Subchannel
 Number  SetS_ID  Target WWPN   D_ID  Status
  52000   00E2562B    00AE548C   PLOGI Error
  *   00E2562B  500604844a37390c  00AE548C   Port Closed

(The first line appears when I try to get WWPN from VM. Second line is 
still visible.)


and it doesn't work :-(


Any ideas ?
Where can I find any explanation/documentation ?

Thank you very much in advance

--
Pozdrawiam/Best regards,

Rafał Hanzel
Programista systemowy, Dział Badań i Rozwoju Systemów Komputerowych
Tel. +48 32 3589246, Fax +48 32 3589277, email: hanz...@zetokatowice.pl
Tel. kom. +48 501677656
Zakład Elektronicznej Techniki Obliczeniowej w Katowicach Spółka z o.o.
40-158 Katowice, ul. Owocowa 1
Sąd Rejonowy Katowice - Wschód w Katowicach Wydział VIII Gospodarczy 
Krajowego Rejestru Sądowego, KRS 051477

Kapitał zakładowy: 264500 zł
NIP 634-013-11-06
http://www.zetokatowice.pl

Jesteśmy uczestnikiem Programu
RZETELNA Firma
Sprawdź naszą rzetelność na
http://www.rzetelnafirma.pl/8FGKEVFH

Treść tej informacji może być poufna, w związku z czym powinna trafić 
bezpośrednio do rąk adresata. Jakiekolwiek jej ujawnianie, 
rozpowszechnianie, bądź kopiowanie jest zabronione. W przypadku 
omyłkowego otrzymania niniejszej informacji prosimy o poinformowanie 
nadawcy i usunięcie jej z komputera.


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Oct.20 Webcast - Intro to SCSI over FCP for Linux on System z - 9:00 AM EDT or 2:00 PM EDT

[Pamela Christina in rainy Endicott NY]

Cross-posted to IBMVM, LINUX390, and IBMMAIN for
the Linux on the mainframe enthusiasts out there.

The next LVC / IBM Linux WebCast is scheduled for Wed. Oct. 20

Register to listen
http://www.vm.ibm.com/education/lvc/

   Date:Wednesday October 20, 2010

   Times:   Choice of two live calls: 9:00AM or 2:00PM EDT
(A replay is planned to be available a few days after the live call.)

   Title:   Introduction to SCSI over FCP for Linux on System z

 Speaker:   Christof Schmitt, IBM Boeblingen Programming Laboratory

Duration:  75 minutes

Abstract:   SCSI over FCP is an open, standard-based alternative and
supplement to existing ESCON or FICON I/O connections.
This session will provide an introduction to the storage
attachment via the SCSI over FCP protocol. It includes setup
considerations,related features in Linux on System z and
troubleshooting basics.

Registration info and Details are in the PDF invitation:
  http://www.vm.ibm.com/education/lvc/lvc1020l.pdf
or just visit this web page.
http://www.vm.ibm.com/education/lvc/


Please direct any questions to Julie Liesenfelt at jul...@us.ibm.com.


Regards, Pam C

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Question on FCP and IODF devices

[Alan Altmark]

On Monday, 10/04/2010 at 11:01 EDT, Jean Nelson/Jacksonville/i...@ibmus
wrote:
> So we have 64 devices on Channel 98 and 64 devices on Channel 99.
>
> My question is this?
>
> If on the SAN side , if they define a disk on PATH 98 , do ALL of the 64
> address defined in the IODF see the WWPN/LUN?
> (Or on a specific Port that equates to PATH/CHPID 98 on the on the
> mainframe side)
>
> If we have 8 zLinux servers defined under z/VM , we would need   8
> separate device address (one for each z/LINUX) to gain access to that
SAN /
> Path.
>
> I haven't been able to find a lot of documentation on this...
> They just give you the IODF definition and not a whole lot of
explination
> on the details on HOW (if any) the device addresses work with the SAN
> (WWPN/LUN)
>
> We have been able to get a WWPN/LUN combination working on the SAN (PATH
98
> and PATH 99).
>
> Just looking for some help from some people that have setup SAN disks
> attached to z/LINUX (not  EDEV's).

If you activate N-port ID virtualization (NPIV) on the FCP adapter, each
of the subchannels on each FCP adapter will have its own WWPN.   Your
zoning and LUN masking will determine which subchannel (guest) has access
to which LUNs.

Alan Altmark

z/VM and Linux on System z Consultant
IBM System Lab Services and Training
ibm.com/systems/services/labservices
office: 607.429.3323
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Question on FCP and IODF devices

[Brian France]

 We used to use FCP connected disks to an SVC. EACH and every linux
image could see each others data.  We've now moved off the SVC to native
shark disks. On the HMC, we turned on NPIV for each of the 4 chpids that
connect us to the SAN. NPIV assigns an individual WWPN for EACH address
so that you can assign (zone) on your shark each linux image connection.

On 10/4/2010 11:00 AM, Jean Nelson wrote:

Question:

We have gotten SAN (FCP) working with a couple of z/Linux Servers.

Right now we have:

CHPID PATH=(98),PARTITION=((VMP1)),TYPE=FCP
CNTLUNIT CUNUMBER=A800,PATH=(98),UNIT=FCP
IODEVICE ADDRESS=(A800),CUNUMBR=(A800),UNIT=FCP

CHPID PATH=(99),PARTITION=((VMP1)),TYPE=FCP
CNTLUNIT CUNUMBER=A801,PATH=(99),UNIT=FCP
IODEVICE ADDRESS=(A801),CUNUMBR=(A801),UNIT=FCP

We attach A800 and A801  to the z/LINUX machine and everything works just
fine.

Now we want to have multiple SAN disks on the same Channels  (CHPID 98 and
CHPID 99)

CHPID PATH=(98),PARTITION=((VMP1)),TYPE=FCP
CNTLUNIT CUNUMBER=A800,PATH=(98),UNIT=FCP
IODEVICE ADDRESS=(A800,64),CUNUMBR=(A800),UNIT=FCP

CHPID PATH=(99),PARTITION=((VMP1)),TYPE=FCP
CNTLUNIT CUNUMBER=A900,PATH=(99),UNIT=FCP
IODEVICE ADDRESS=(A900,64),CUNUMBR=(A900),UNIT=FCP


So we have 64 devices on Channel 98 and 64 devices on Channel 99.

My question is this?

If on the SAN side , if they define a disk on PATH 98 , do ALL of the 64
address defined in the IODF see the WWPN/LUN?
(Or on a specific Port that equates to PATH/CHPID 98 on the on the
mainframe side)

If we have 8 zLinux servers defined under z/VM , we would need   8
separate device address (one for each z/LINUX) to gain access to that SAN /
Path.

I haven't been able to find a lot of documentation on this...
They just give you the IODF definition and not a whole lot of explination
on the details on HOW (if any) the device addresses work with the SAN
(WWPN/LUN)

We have been able to get a WWPN/LUN combination working on the SAN (PATH 98
and PATH 99).

Just looking for some help from some people that have setup SAN disks
attached to z/LINUX (not  EDEV's).

Thanks again in advance!
Jean


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/




--

--

Brian W. France
Systems Administrator (Mainframe)
Pennsylvania State University
Administrative Information Services - Infrastructure/SYSARC
Rm 25 Shields Bldg., University Park, Pa. 16802
814-863-4739
b...@psu.edu <mailto:b...@psu.edu>

"To make an apple pie from scratch, you must first invent the universe."

Carl Sagan

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Question on FCP and IODF devices

[Jean Nelson]

Question:

We have gotten SAN (FCP) working with a couple of z/Linux Servers.

Right now we have:

CHPID PATH=(98),PARTITION=((VMP1)),TYPE=FCP
CNTLUNIT CUNUMBER=A800,PATH=(98),UNIT=FCP
IODEVICE ADDRESS=(A800),CUNUMBR=(A800),UNIT=FCP

CHPID PATH=(99),PARTITION=((VMP1)),TYPE=FCP
CNTLUNIT CUNUMBER=A801,PATH=(99),UNIT=FCP
IODEVICE ADDRESS=(A801),CUNUMBR=(A801),UNIT=FCP

We attach A800 and A801  to the z/LINUX machine and everything works just
fine.

Now we want to have multiple SAN disks on the same Channels  (CHPID 98 and
CHPID 99)

CHPID PATH=(98),PARTITION=((VMP1)),TYPE=FCP
CNTLUNIT CUNUMBER=A800,PATH=(98),UNIT=FCP
IODEVICE ADDRESS=(A800,64),CUNUMBR=(A800),UNIT=FCP

CHPID PATH=(99),PARTITION=((VMP1)),TYPE=FCP
CNTLUNIT CUNUMBER=A900,PATH=(99),UNIT=FCP
IODEVICE ADDRESS=(A900,64),CUNUMBR=(A900),UNIT=FCP


So we have 64 devices on Channel 98 and 64 devices on Channel 99.

My question is this?

If on the SAN side , if they define a disk on PATH 98 , do ALL of the 64
address defined in the IODF see the WWPN/LUN?
(Or on a specific Port that equates to PATH/CHPID 98 on the on the
mainframe side)

If we have 8 zLinux servers defined under z/VM , we would need   8
separate device address (one for each z/LINUX) to gain access to that SAN /
Path.

I haven't been able to find a lot of documentation on this...
They just give you the IODF definition and not a whole lot of explination
on the details on HOW (if any) the device addresses work with the SAN
(WWPN/LUN)

We have been able to get a WWPN/LUN combination working on the SAN (PATH 98
and PATH 99).

Just looking for some help from some people that have setup SAN disks
attached to z/LINUX (not  EDEV's).

Thanks again in advance!
Jean


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

FCP and z10

[Carlos Bodra - Pessoal]

 Hi listners,

Just to tell that we get out 2105-F20 connected to z10 BC via FCP this
weekend via switch 2005-B16.

Seems that we have a problem with 2105-800 that we was trying to
connected earlier. We will check ESS800
to find what is wrong.

Thanks for all that send tips and hints.

--
Carlos Bodra
IBM zSeries Certified Specialist
Sao Paulo - Brazil


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: z10 shark 800 and FCP

[John Schnitzler Jr]

Hello Carlos,

Make sure the chipid is defined as type FCP.
Connect the chipid to the switch.
Looking in the switch do you see the 50050764 wwpn of the chipid
logging into the switch?
If not, check to make sure you are using the same type of adapters.
If the chipid is a long wave LX adapter your switch much have a matching LX
GBIC card.
If your using a short wave SX chipid, make sure the switch GBIC card is
also SX.
Do the lights on the switch's GBIC adapter say there is a good connection
with the switch?
If your switch guy can see the 50050764 wwpn logged into the
switch, does he also see the 50050763 wwpn of the shark in the
same zone?
If not, again make sure that both ends of the switch and shark are LX or
SX. In my F20 sharks the WWPN of the chipids do not display, I had to
manually configure and enter the 16 digit 50050764. wwpn of each
chipid that was going to access the shark. I don't have any experience with
the ESS800 so can't help you much there. Here's a presentation that may
help.

Regards,
John




From:   Carlos Bodra - Pessoal <>
To: LINUX-390@vm.marist.edu
Date:   09/27/2010 11:11 AM
Subject:    Re: z10 shark 800 and FCP
Sent by:Linux on 390 Port 



  John

Even wwpn list shows as empty, we select in ESS zSeries(linux) as host
and enter manually wwpn of z10 port in shark configuration,
It´s start a configuration, but ends with error.

Carlos Bodra
IBM zSeries Certified Specialist
Sao Paulo - Brazil


Em 27/09/2010 12:01, John Schnitzler Jr escreveu:
> Carlos,
>
> Have you tried to manually enter the 5005... wwpn of the z10 into the
> sharks configuration.
>
> Regards,
>
> John
>
>
>
> From:   Carlos Bodra - Pessoal
> To: LINUX-390@vm.marist.edu
> Date:   09/27/2010 10:54 AM
> Subject:z10 shark 800 and FCP
> Sent by:Linux on 390 Port
>
>
>
>Hi
>
> I´m trying to connect a z10 to a shark using FCP protocol, but without
> success.
> We did some tests that are describe below:
>
> 1 - Connect z10 to shark 800 sw2 adapter with both sides defined as
> Ficon. Works fine
>
> 2 - Connect z10 to shark 800 sw2 adapter with both sides defined as FCP.
> Direct connection, no switch envolved. Didn´t work. We aren´t able to
> define open system storage in ESS Specialist. Seems that shark doesn´t
> "see" FCP adapter in z10. After reading some material (redbooks) we
> found information about that a direct connection didn´t work. Is
> mandatory use a switch.
>
> 3 - Connect z10 to shark 800 sw2 adapter with both sides defined as FCP
> and using an old IBM 2109-S16. Didn´t work. We aren´t able to define
> open system storage in ESS Specialist. Seems that shark doesn´t "see"
> FCP adapter in z10.
>
> 4 - Connect z10 to shark 800 sw2 adapter with both sides defined as FCP
> and using an IBM 2005-B16. Didn´t work. We aren´t able to define open
> system storage in ESS Specialist. Seems that shark doesn´t "see" FCP
> adapter in z10.
>
> 5 - Connect z10 to DS4700 via FC adapter with z10 define as FCP and
> storage defined as FC and using an old IBM 2109-S16. Didn´t work. We
> aren´t able to access open dasd.
>
> We checked microcode level for shark and seems to be ok. If I connect
> shark or DS4700 to an intel server, we are able to access open dasd
> without problems.
>
> Since z10 with FCP access is new for us, any hint or tip about are
welcome.
>
> Thanks in advance.
> --
>
> Carlos Bodra
> IBM zSeries Certified Specialist
> Sao Paulo - Brazil
>
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more informati

Re: z10 shark 800 and FCP

[Carlos Bodra - Pessoal]

 John

Even wwpn list shows as empty, we select in ESS zSeries(linux) as host 
and enter manually wwpn of z10 port in shark configuration,

It´s start a configuration, but ends with error.

Carlos Bodra
IBM zSeries Certified Specialist
Sao Paulo - Brazil


Em 27/09/2010 12:01, John Schnitzler Jr escreveu:

Carlos,

Have you tried to manually enter the 5005... wwpn of the z10 into the
sharks configuration.

Regards,

John



From:   Carlos Bodra - Pessoal
To: LINUX-390@vm.marist.edu
Date:   09/27/2010 10:54 AM
Subject:z10 shark 800 and FCP
Sent by:Linux on 390 Port



   Hi

I´m trying to connect a z10 to a shark using FCP protocol, but without
success.
We did some tests that are describe below:

1 - Connect z10 to shark 800 sw2 adapter with both sides defined as
Ficon. Works fine

2 - Connect z10 to shark 800 sw2 adapter with both sides defined as FCP.
Direct connection, no switch envolved. Didn´t work. We aren´t able to
define open system storage in ESS Specialist. Seems that shark doesn´t
"see" FCP adapter in z10. After reading some material (redbooks) we
found information about that a direct connection didn´t work. Is
mandatory use a switch.

3 - Connect z10 to shark 800 sw2 adapter with both sides defined as FCP
and using an old IBM 2109-S16. Didn´t work. We aren´t able to define
open system storage in ESS Specialist. Seems that shark doesn´t "see"
FCP adapter in z10.

4 - Connect z10 to shark 800 sw2 adapter with both sides defined as FCP
and using an IBM 2005-B16. Didn´t work. We aren´t able to define open
system storage in ESS Specialist. Seems that shark doesn´t "see" FCP
adapter in z10.

5 - Connect z10 to DS4700 via FC adapter with z10 define as FCP and
storage defined as FC and using an old IBM 2109-S16. Didn´t work. We
aren´t able to access open dasd.

We checked microcode level for shark and seems to be ok. If I connect
shark or DS4700 to an intel server, we are able to access open dasd
without problems.

Since z10 with FCP access is new for us, any hint or tip about are welcome.

Thanks in advance.
--

Carlos Bodra
IBM zSeries Certified Specialist
Sao Paulo - Brazil


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/




--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: z10 shark 800 and FCP

[John Schnitzler Jr]

Carlos,

Have you tried to manually enter the 5005... wwpn of the z10 into the
sharks configuration.

Regards,

John



From:   Carlos Bodra - Pessoal 
To: LINUX-390@vm.marist.edu
Date:   09/27/2010 10:54 AM
Subject:z10 shark 800 and FCP
Sent by:Linux on 390 Port 



  Hi

I´m trying to connect a z10 to a shark using FCP protocol, but without
success.
We did some tests that are describe below:

1 - Connect z10 to shark 800 sw2 adapter with both sides defined as
Ficon. Works fine

2 - Connect z10 to shark 800 sw2 adapter with both sides defined as FCP.
Direct connection, no switch envolved. Didn´t work. We aren´t able to
define open system storage in ESS Specialist. Seems that shark doesn´t
"see" FCP adapter in z10. After reading some material (redbooks) we
found information about that a direct connection didn´t work. Is
mandatory use a switch.

3 - Connect z10 to shark 800 sw2 adapter with both sides defined as FCP
and using an old IBM 2109-S16. Didn´t work. We aren´t able to define
open system storage in ESS Specialist. Seems that shark doesn´t "see"
FCP adapter in z10.

4 - Connect z10 to shark 800 sw2 adapter with both sides defined as FCP
and using an IBM 2005-B16. Didn´t work. We aren´t able to define open
system storage in ESS Specialist. Seems that shark doesn´t "see" FCP
adapter in z10.

5 - Connect z10 to DS4700 via FC adapter with z10 define as FCP and
storage defined as FC and using an old IBM 2109-S16. Didn´t work. We
aren´t able to access open dasd.

We checked microcode level for shark and seems to be ok. If I connect
shark or DS4700 to an intel server, we are able to access open dasd
without problems.

Since z10 with FCP access is new for us, any hint or tip about are welcome.

Thanks in advance.
--

Carlos Bodra
IBM zSeries Certified Specialist
Sao Paulo - Brazil


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

<>

z10 shark 800 and FCP

[Carlos Bodra - Pessoal]

 Hi

I´m trying to connect a z10 to a shark using FCP protocol, but without 
success.

We did some tests that are describe below:

1 - Connect z10 to shark 800 sw2 adapter with both sides defined as 
Ficon. Works fine


2 - Connect z10 to shark 800 sw2 adapter with both sides defined as FCP. 
Direct connection, no switch envolved. Didn´t work. We aren´t able to 
define open system storage in ESS Specialist. Seems that shark doesn´t 
"see" FCP adapter in z10. After reading some material (redbooks) we 
found information about that a direct connection didn´t work. Is 
mandatory use a switch.


3 - Connect z10 to shark 800 sw2 adapter with both sides defined as FCP 
and using an old IBM 2109-S16. Didn´t work. We aren´t able to define 
open system storage in ESS Specialist. Seems that shark doesn´t "see" 
FCP adapter in z10.


4 - Connect z10 to shark 800 sw2 adapter with both sides defined as FCP 
and using an IBM 2005-B16. Didn´t work. We aren´t able to define open 
system storage in ESS Specialist. Seems that shark doesn´t "see" FCP 
adapter in z10.


5 - Connect z10 to DS4700 via FC adapter with z10 define as FCP and 
storage defined as FC and using an old IBM 2109-S16. Didn´t work. We 
aren´t able to access open dasd.


We checked microcode level for shark and seems to be ok. If I connect 
shark or DS4700 to an intel server, we are able to access open dasd 
without problems.


Since z10 with FCP access is new for us, any hint or tip about are welcome.

Thanks in advance.
--

Carlos Bodra
IBM zSeries Certified Specialist
Sao Paulo - Brazil


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Shared fcp between two linux guests

[Christof Schmitt]

On Thu, Jun 24, 2010 at 02:53:23PM -0400, Bob McCarthy wrote:
> Ray,
>The following displays are for the 2nd RedHat guest that is brought
> up. It fails.
>
>
> kernel: zfcp: The adapter 0.0.2107 reported the following
> characteristics:
>  tdcrac02 kernel: WWNN 0x5005076400cb90c4, WWPN 0x500507640162c859, S_ID
> 0x00014c00,
>  tdcrac02 kernel: adapter version 0x4, LIC version 0xb02, FC link speed
> 4 Gb/s
>  tdcrac02 kernel: zfcp: Switched fabric fibrechannel network detected at
> adapter 0.0.2107.
>  tdcrac02 kernel: zfcp: unit erp failed on unit 0x40114026 on
> port 0x50050763030384ac  on adapter 0.0.2107
>  tdcrac02 kernel: zfcp: unit erp failed on unit 0x40114026 on
> port 0x500507630308c4ac  on adapter 0.0.2107
>
> [r...@tdcrac02 log]# lscss | grep 2107
> 0.0.2107 0.0.0008  1732/03 1731/03 yes  80  80  ff   50000000 
>
> [r...@tdcrac02 ~]# lszfcp -D
> Error: No fcp devices found.

lszfcp -D looks at the available SCSI devices in the Linux kernel. If
there are no SCSI devices, then the output will be empty, as it is
here. One possibility would be that the LUN on the storage is not
mapped for the WWPN of the FCP channel.

If this is the case, then the Linux kernel tries to add a SCSI device
for the LUN, queries the LUN on the storage, the storage returns
something along "the LUN is invalid" and the Linux kernel stops the
creating this SCSI device.

You could try to increase the Linux kernel messages for the SCSI
device discovery:
scsi_logging_level -s --scan 3
It should show more information and i would expect some sort of
"invalid LUN" message before the "unit erp failed".

> [r...@tdcrac02 ~]# lszfcp -a -b 0.0.2107
> 0.0.2107 host0
[...]
> port_name   = "0x500507640162c859"

This is the WWPN of the FCP channel that the storage server will see.
I would suggest checking the configuration on the storage server to
check if the LUN 0x40114026 is mapped and visible for this
WWPN.

> > On Thu, Jun 24, 2010 at 10:43, Bob McCarthy
> > 
> wrote:
[...]
> > > Jun 21 15:36:52 tdcrac02 kernel: zfcp: Switched fabric fibrechannel
> > > network detected at adapter 0.0.8001.
> > > Jun 21 15:36:52 tdcrac02 kernel: zfcp: unit erp failed on unit
> > > 0x0020 on port 0x5001738001820141  on adapter 0.0.8001
> > > We are running zVM V5.4 and RedHat linux v5.5. The fcp disk is
> > > assigned separate channel paths in each guest through DEDICATE
> > > statements in vm.

If i remember correctly, the lsluns is also part of the s390-tools
package in RHEL 5.5. You can run lsluns to query the LUNs that are
available on the storage server for the Linux system where you run
lsluns.

With the above information, i would guess that this LUN is not
available for the RHEL5.5 system.

I hope this helps in tracking down the problem,

Christof

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Shared fcp between two linux guests

[Bob McCarthy]

Ray,
   The following displays are for the 2nd RedHat guest that is brought
up. It fails.


kernel: zfcp: The adapter 0.0.2107 reported the following
characteristics:
 tdcrac02 kernel: WWNN 0x5005076400cb90c4, WWPN 0x500507640162c859, S_ID
0x00014c00,
 tdcrac02 kernel: adapter version 0x4, LIC version 0xb02, FC link speed
4 Gb/s
 tdcrac02 kernel: zfcp: Switched fabric fibrechannel network detected at
adapter 0.0.2107.
 tdcrac02 kernel: zfcp: unit erp failed on unit 0x40114026 on
port 0x50050763030384ac  on adapter 0.0.2107
 tdcrac02 kernel: zfcp: unit erp failed on unit 0x40114026 on
port 0x500507630308c4ac  on adapter 0.0.2107  

[r...@tdcrac02 log]# lscss | grep 2107
0.0.2107 0.0.0008  1732/03 1731/03 yes  80  80  ff   5000 

[r...@tdcrac02 ~]# lszfcp -D
Error: No fcp devices found.

[r...@tdcrac02 ~]# lszfcp -a -b 0.0.2107
0.0.2107 host0
Bus = "ccw"
availability= "good"
card_version= "0x0004"
cmb_enable  = "0"
cutype  = "1731/03"
devtype = "1732/03"
failed  = "0"
hardware_version= "0x"
in_recovery = "0"
lic_version = "0x0b02"
modalias= "ccw:t1731m03dt1732dm03"
online  = "1"
peer_d_id   = "0x00"
peer_wwnn   = "0x"
peer_wwpn   = "0x"
status  = "0x5400082e"
Class = "fc_host"
maxframe_size   = "2112 bytes"
node_name   = "0x5005076400cb90c4"
permanent_port_name = "0x500507640162c859"
port_id = "0x014c00"
port_name   = "0x500507640162c859"
port_type   = "NPort (fabric via point-to-point)"
serial_number   = "IBM02000B90C4"
speed   = "4 Gbit"
supported_classes   = "Class 2, Class 3"
supported_speeds= "4 Gbit"
tgtid_bind_type = "wwpn (World Wide Port Name)"
Class = "scsi_host"
cmd_per_lun = "1"
host_busy   = "0"
proc_name   = "zfcp"
queue_full  = "0 91001920976"
sg_tablesize= "538"
state   = "running"
unchecked_isa_dma   = "0"
unique_id   = "0"
utilization = "0 2 0"
 


The following displays are for the 1st Redhat guest that is brought up.
It sees the wwpn and luns 


[r...@tdcrac01 ~]# lscss | grep 2107
0.0.2107 0.0.000b  1732/03 1731/03 yes  80  80  ff   4900 


[r...@tdcrac01 ~]# lszfcp -D
0.0.2107/0x50050763030384ac/0x40114026 0:0:0:1
0.0.2107/0x500507630308c4ac/0x40114026 0:0:1:1


[r...@tdcrac01 ~]# lszfcp -a -b 0.0.2107
0.0.2107 host0
Bus = "ccw"
availability= "good"
card_version= "0x0004"
cmb_enable  = "0"
cutype  = "1731/03"
devtype = "1732/03"
failed  = "0"
hardware_version= "0x"
in_recovery = "0"
lic_version = "0x0b02"
modalias= "ccw:t1731m03dt1732dm03"
online  = "1"
peer_d_id   = "0x00"
peer_wwnn   = "0x"
peer_wwpn   = "0x"
status  = "0x5400082e"
Class = "fc_host"
maxframe_size   = "2112 bytes"
node_name   = "0x5005076400cb90c4"
permanent_port_name = "0x5005076401e2c929"
port_id = "0x01a400"
port_name   = "0x5005076401e2c929"
port_type   = "NPort (fabric via point-to-point)"
serial_number   = "IBM02000B90C4"
speed   = "2 Gbit"
supported_classes   = "Class 2, Class 3"
supported_speeds= "4 Gbit"
tgtid_bind_type = "wwpn (World Wide Port Name)"
Class = "scsi_host"
cmd_per_lun = "1"
host_busy   = "0"
proc_name   = "zfcp"
queue_full  = "0 103530559246"
sg_tablesize= "538"
state       = "running"
unchecked_isa_dma   = "0"
unique_id   = "0"
utilization = "1 12 0"


Thank you,   Bob  

-Original Message-
From: Linux on 390 Port [mailto:linux-...@vm.marist.edu] On Behalf Of
Raymond Higgs

Re: Shared fcp between two linux guests

[Raymond Higgs]

Linux on 390 Port  wrote on 06/24/2010 01:02:06
PM:

> Bob McCarthy 
> Sent by: Linux on 390 Port 
>
> 06/24/2010 01:02 PM
>
> Please respond to
> Linux on 390 Port 
>
> To
>
> LINUX-390@vm.marist.edu
>
> cc
>
> Subject
>
> Re: Shared fcp between two linux guests
>
> Rick,
>   Are FCP adapters that you are referring to the same as the
> physical fcp channels (or CHPIDs)  that are defined to the vm lpar
> through HCD? If so, I have (4) physical FCP channels defined on the
> lpar, and I am using a different channel for each linux guest. This
> is the same configuration for both RedHat and SUSE. When I used the
> same FCP channel for both guests, I received an additional "path in
> use" error message, which I should have expected. We have reversed
> the order in which the guests are brought up with the RedHat guests,
> and the 1st guest that comes up will successfully access the lun,
> and the 2nd guest will receive the error.
> Thank you, Bob
>
> -Original Message-
> From: Linux on 390 Port [mailto:linux-...@vm.marist.edu] On Behalf
> Of Richard Troth
> Sent: Thursday, June 24, 2010 10:56 AM
> To: LINUX-390@VM.MARIST.EDU
> Subject: Re: Shared fcp between two linux guests
>
> Bob --
>
> You said:
>
> > NPIV is not turned on at the lpar level, but it is turned on at the
> > SAN level.
>
> If you're sharing the same physical FCP adapter(s) between the two
> Linux guests, then you probably need NPIV (at the LPAR level).
>
> I cannot imagine how your other Linux guests are able to share a LUN
> if the storage frame sees the same Linux-side WWPN (FCP WWPN).
> Since you indicated that your SuSE guests *can* share a LUN between
> them, I have to ask if they are using different physical FCP
> adapters or if there is NPIV in play for the FCP adapter(s) they are
> using.  What's the deal there?
>
> The only time I have been able to share a LUN, the storage frame has
> always seen different "HBA-side" WWPNs, whether physically different
> FCP adapters or with NPIV.
>
> Try reversing the order in qhich the guests are brought up to test
things.
>
> -- Rick;   <><
>
>
>
>
>
> On Thu, Jun 24, 2010 at 10:43, Bob McCarthy 
wrote:
> > We are planning to setup an Oracle RAC environment with two linux
> > guests on the same vm lpar. They will be accessing fcp disk. The
> > problem is as
> > follows:
> > 1. Guest 1 is brought up. "lszfcp -D" command will show the wwpn and
> > lun as expected 2. Guest 2 is brought up. "lszfcp -D" command shows
> > nothing. "lszfcp -P"
> > command will show the wwpn
> > Looking at the error log in linux we see the following messages :
> > Jun 21 15:36:52 tdcrac02 kernel: zfcp: The adapter 0.0.8001 reported
> > the following characteristics:
> > Jun 21 15:36:52 tdcrac02 kernel: zfcp: Switched fabric fibrechannel
> > network detected at adapter 0.0.8001.
> > Jun 21 15:36:52 tdcrac02 kernel: zfcp: unit erp failed on unit
> > 0x0020 on port 0x5001738001820141  on adapter 0.0.8001 We
> > are running zVM V5.4 and RedHat linux v5.5. The fcp disk is assigned
> > separate channel paths in each guest through DEDICATE statements in
vm.
> > NPIV is not turned on at the lpar level, but it is turned on at the
> > SAN level. Additionally, we have set up two guests with the exact same

> > configuration, but using SUSE 10SP2. This configuration has no
> > problems sharing the SAN disk. Is there configuration in RedHat that
> > must be done that would be different from SUSE ?
> >  Thank you,Bob
> >
> > --
> > For LINUX-390 subscribe / signoff / archive access instructions, send
> > email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> > visit http://www.marist.edu/htbin/wlvindex?LINUX-390
> > --
> > For more information on Linux on System z, visit
> > http://wiki.linuxvm.org/
> >
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO
> LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
http://wiki.linuxvm.org/
>
> --
> For LINUX-390 subs

Re: Shared fcp between two linux guests

[Raymond Higgs]

Linux on 390 Port  wrote on 06/24/2010 12:24:38
PM:

> Alan Altmark/Endicott/i...@ibmus
> Sent by: Linux on 390 Port 
>
> 06/24/2010 12:24 PM
>
> Please respond to
> Linux on 390 Port 
>
> To
>
> LINUX-390@vm.marist.edu
>
> cc
>
> Subject
>
> Re: Shared fcp between two linux guests
>
> On Thursday, 06/24/2010 at 10:59 EDT, Richard Troth 
> wrote:
>
> > > NPIV is not turned on at the lpar level, but it is turned on at the
> SAN
> > > level.
> >
> > If you're sharing the same physical FCP adapter(s) between the two
> > Linux guests, then you probably need NPIV (at the LPAR level).
> >
> > I cannot imagine how your other Linux guests are able to share a LUN
> > if the storage frame sees the same Linux-side WWPN (FCP WWPN).  Since
> > you indicated that your SuSE guests *can* share a LUN between them, I
> > have to ask if they are using different physical FCP adapters or if
> > there is NPIV in play for the FCP adapter(s) they are using.  What's
> > the deal there?
> >
> > The only time I have been able to share a LUN, the storage frame has
> > always seen different "HBA-side" WWPNs, whether physically different
> > FCP adapters or with NPIV.
> >
> > Try reversing the order in qhich the guests are brought up to test
> things.
>
>
> > > Jun 21 15:36:52 tdcrac02 kernel: zfcp: The adapter 0.0.8001 reported
> the
> > > following characteristics:
> > > Jun 21 15:36:52 tdcrac02 kernel: zfcp: Switched fabric fibrechannel
> > > network detected at adapter 0.0.8001.
> > > Jun 21 15:36:52 tdcrac02 kernel: zfcp: unit erp failed on unit
> > > 0x0020 on port 0x5001738001820141  on adapter 0.0.8001
>
> If this is an older machine without NPIV, that "unit erp" error has been
> seen when the guest was not authorized using the LUN Access Control
> program.
>
> 'erp' is 'error recovery program'.  The device driver gets failures at
the
> LUN, port, and unit levels (I think).  Perhaps Ray Higgs can shed some
> light on it.
>
> Alan Altmark
> z/VM Development
> IBM Endicott
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/

LUN Access Control is not available for z10.

I'd use san_disc to investigate.  Go through the example in the manpage.
If you do not get what you expect from port_list, there is probably a
zoning problem.  Use the brocade switch console to address this.  If you
do not get what you expect from report_luns, then there is a lun masking
problem.  Use the XIV management tools to address.

Sometimes getting LUN numbers correct can be difficult because of all of
the zeros.  Did you mean 0x0002 instead of 0x0020?

Regards,

Ray Higgs
System z FCP Development
Bld. 706, B24
2455 South Road
Poughkeepsie, NY 12601
(845) 435-8666,  T/L 295-8666
rayhi...@us.ibm.com

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Shared fcp between two linux guests

[Mark Post]

>>> On 6/24/2010 at 10:43 AM, Bob McCarthy  wrote: 
> Additionally, we have set up two guests with the exact same
> configuration, but using SUSE 10SP2. This configuration has no problems
> sharing the SAN disk. Is there configuration in RedHat that must be done
> that would be different from SUSE ?

I've been trying to stay out of this because I didn't want to be perceived as 
bashing a competitor.  However, what I've been told by numerous people is that 
the SCSI over FCP support in RHEL has not been at the same level as what is in 
SLES.  I had thought most of that was corrected in RHEL 5.5, but I can't be 
certain of that.  You really need to be working with Red Hat support on this to 
find out what's going on.  I know Justin and his co-workers would be very 
interested in getting this resolved for you.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Shared fcp between two linux guests

[Bob McCarthy]

Alan,
   The machine is z10 (EC), but NPIV has not been turned on. It attaches
to IBM XIV SAN storage through a Brocade switch. The switch has NPIV set
on.
 Thanks,Bob  

-Original Message-
From: Linux on 390 Port [mailto:linux-...@vm.marist.edu] On Behalf Of
Alan Altmark
Sent: Thursday, June 24, 2010 12:25 PM
To: LINUX-390@VM.MARIST.EDU
Subject: Re: Shared fcp between two linux guests

On Thursday, 06/24/2010 at 10:59 EDT, Richard Troth 
wrote:

> > NPIV is not turned on at the lpar level, but it is turned on at the
SAN
> > level.
>
> If you're sharing the same physical FCP adapter(s) between the two 
> Linux guests, then you probably need NPIV (at the LPAR level).
>
> I cannot imagine how your other Linux guests are able to share a LUN 
> if the storage frame sees the same Linux-side WWPN (FCP WWPN).  Since 
> you indicated that your SuSE guests *can* share a LUN between them, I 
> have to ask if they are using different physical FCP adapters or if 
> there is NPIV in play for the FCP adapter(s) they are using.  What's 
> the deal there?
>
> The only time I have been able to share a LUN, the storage frame has 
> always seen different "HBA-side" WWPNs, whether physically different 
> FCP adapters or with NPIV.
>
> Try reversing the order in qhich the guests are brought up to test
things.


> > Jun 21 15:36:52 tdcrac02 kernel: zfcp: The adapter 0.0.8001 reported
the
> > following characteristics:
> > Jun 21 15:36:52 tdcrac02 kernel: zfcp: Switched fabric fibrechannel 
> > network detected at adapter 0.0.8001.
> > Jun 21 15:36:52 tdcrac02 kernel: zfcp: unit erp failed on unit 
> > 0x0020 on port 0x5001738001820141  on adapter 0.0.8001

If this is an older machine without NPIV, that "unit erp" error has been
seen when the guest was not authorized using the LUN Access Control
program.

'erp' is 'error recovery program'.  The device driver gets failures at
the LUN, port, and unit levels (I think).  Perhaps Ray Higgs can shed
some light on it.

Alan Altmark
z/VM Development
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions, send
email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Shared fcp between two linux guests

[Bob McCarthy]

Rick,
  Are FCP adapters that you are referring to the same as the physical fcp 
channels (or CHPIDs)  that are defined to the vm lpar through HCD? If so, I 
have (4) physical FCP channels defined on the lpar, and I am using a different 
channel for each linux guest. This is the same configuration for both RedHat 
and SUSE. When I used the same FCP channel for both guests, I received an 
additional "path in use" error message, which I should have expected. We have 
reversed the order in which the guests are brought up with the RedHat guests, 
and the 1st guest that comes up will successfully access the lun, and the 2nd 
guest will receive the error. 
Thank you, Bob  

-Original Message-
From: Linux on 390 Port [mailto:linux-...@vm.marist.edu] On Behalf Of Richard 
Troth
Sent: Thursday, June 24, 2010 10:56 AM
To: LINUX-390@VM.MARIST.EDU
Subject: Re: Shared fcp between two linux guests

Bob --

You said:

> NPIV is not turned on at the lpar level, but it is turned on at the 
> SAN level.

If you're sharing the same physical FCP adapter(s) between the two Linux 
guests, then you probably need NPIV (at the LPAR level).

I cannot imagine how your other Linux guests are able to share a LUN if the 
storage frame sees the same Linux-side WWPN (FCP WWPN).  Since you indicated 
that your SuSE guests *can* share a LUN between them, I have to ask if they are 
using different physical FCP adapters or if there is NPIV in play for the FCP 
adapter(s) they are using.  What's the deal there?

The only time I have been able to share a LUN, the storage frame has always 
seen different "HBA-side" WWPNs, whether physically different FCP adapters or 
with NPIV.

Try reversing the order in qhich the guests are brought up to test things.

-- Rick;   <><





On Thu, Jun 24, 2010 at 10:43, Bob McCarthy  wrote:
> We are planning to setup an Oracle RAC environment with two linux 
> guests on the same vm lpar. They will be accessing fcp disk. The 
> problem is as
> follows:
> 1. Guest 1 is brought up. "lszfcp -D" command will show the wwpn and 
> lun as expected 2. Guest 2 is brought up. "lszfcp -D" command shows 
> nothing. "lszfcp -P"
> command will show the wwpn
> Looking at the error log in linux we see the following messages :
> Jun 21 15:36:52 tdcrac02 kernel: zfcp: The adapter 0.0.8001 reported 
> the following characteristics:
> Jun 21 15:36:52 tdcrac02 kernel: zfcp: Switched fabric fibrechannel 
> network detected at adapter 0.0.8001.
> Jun 21 15:36:52 tdcrac02 kernel: zfcp: unit erp failed on unit 
> 0x0020 on port 0x5001738001820141  on adapter 0.0.8001 We 
> are running zVM V5.4 and RedHat linux v5.5. The fcp disk is assigned 
> separate channel paths in each guest through DEDICATE statements in vm.
> NPIV is not turned on at the lpar level, but it is turned on at the 
> SAN level. Additionally, we have set up two guests with the exact same 
> configuration, but using SUSE 10SP2. This configuration has no 
> problems sharing the SAN disk. Is there configuration in RedHat that 
> must be done that would be different from SUSE ?
>                                              Thank you,    Bob
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions, send 
> email to lists...@vm.marist.edu with the message: INFO LINUX-390 or 
> visit http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit 
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions, send email to 
lists...@vm.marist.edu with the message: INFO LINUX-390 or visit 
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/