Re: VSwitch and IPv6
On Friday, 09/07/2007 at 01:40 EDT, Marcy Cortes [EMAIL PROTECTED] wrote: This is all confusing to me - this layer 2/3 stuff. It was said to be not allowed on the same card. Then not on the same port? Now OK on the same port (given I have the latest and greatest z9 stuff, that's easy - someone else gets to do that)? In the vernacular, card means port. Feature means card. Oh, wait. A single feature has two daughter cards in it. wah It is time to standardize our terminology, everyone. Architecturally: - Feature: The thing that you order and plug into the I/O cage. It has one or more chpids, each with one or ports. - Chpid: The channel that gives host access to one or more ports. - Port: The thing you plug the cable into. They are numbered, within each chpid, from 0 to n. For all of the OSAs, you have been using Port 0. Exceedingly convenient since all of the host interface definitions default to port 0. [To keep you on your toes, the Token Ring feature exhibits two physical connectors per port (DB-9 and RJ-45), but only one can be used. But that's TR. Who cares? All of the Ethernet features exhibit one physical connector per port.] See Appendix A of your OSA-Express Customer's Guide and Reference. Scenario - I have 2 OSA cards. 2 ports on each (duh). One port on each going to a diff switch. There's no duh. Not all OSA features have two ports. The 10 GbE feature has a single chpid with a single port. :-) OSA1 port A switch a port B switch b OSA2 port A switch a port B switch b VM TCPIP stack has OSA1 portA and OSA2 portB - layer 3 (and some guest LAN, but that doesn't matter). VSWITCH Layer 3 has OSA1 portB and OSA2 portA Can I add a Layer 2 vswitch here without disturbing what's there? That is, define 1 more vswitch using addresses on OSA1 portB and OSA2 port A? Yes. Existing VSWITCH Layer 3 def looks like: Define VSWITCH td6vsw1 rdev f400 f600 vlan 797 porttype trunk Can I just add the layer 2 one with something like: Define vswitch td6vsw2 rdev f410 f610 ETHERNET vlan 797 porttype trunk And then update Linux with *for SLES: QETH_LAYER2_SUPPORT=1 in /etc/sysconfig/hardware/hwcfg-qeth-bus-ccw-0.0. And that'll work?? Yes. The only issue with L2/L3 on the same port is that, up to now, hosts using L2 could not talk to hosts using L3 on the same OSA port. That is, you couldn't have them on the same subnet and expect them to talk to each other. If the servers don't talk to each other Or must I come up with some more ports. Only if you - want to use Link Aggregation - have network security issues to deal with And, of course, you can't (technically can, but really really shouldn't) mix VLAN-tagged traffic and non-VLAN-tagged traffic on the same OSA port. Bonus points for explaining how I can get link aggregation involved :) Use SET PORT GROUP to create a named list of OSA RDEVs. Use the GROUP name instead of RDEVs on DEFINE VSWITCH. Create a similar group on the physical switch. (Both ends of the cable have to be configured to operate in a link agg. group.) Et voila. z/VM 5.3 only. Because the link aggregation protocol requires [v]switch-switch control flows, not just data, you must dedicate the entire OSA port to the VSWITCH. No sharing. At some point the OSAs will be updated to detect this and will not allow active sharing of an OSA port that is using link aggregation. Alan Altmark z/VM Development IBM Endicott -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Betr.: Re: VSwitch and IPv6
I will leave the more complex L2/L3 question to someone more enlightened than me. But I am in the clear about Link Aggregation (I wasn't until recently, but support straightened me out) LA requires exlusivity, meaning ONE triplet in ONE lpar having access, an nothing else. Which is not as bad as it sounds at first. If you need LA for bandwidth, it sure makes no sense to put more load on the ports involved from other triplets. So for LA you need to hunt for free ports. Best regards, Pieter Harder [EMAIL PROTECTED] tel +31-73-6837133 / +31-6-47272537 Marcy Cortes [EMAIL PROTECTED] 09/07/07 7:39 This is all confusing to me - this layer 2/3 stuff. It was said to be not allowed on the same card. Then not on the same port? Now OK on the same port (given I have the latest and greatest z9 stuff, that's easy - someone else gets to do that)? Example needed please. I'll give you a little picture but I'm a little graphically challenged. Scenario - I have 2 OSA cards. 2 ports on each (duh). One port on each going to a diff switch. OSA1 port A switch a port B switch b OSA2 port A switch a port B switch b VM TCPIP stack has OSA1 portA and OSA2 portB - layer 3 (and some guest LAN, but that doesn't matter). VSWITCH Layer 3 has OSA1 portB and OSA2 portA Can I add a Layer 2 vswitch here without disturbing what's there? That is, define 1 more vswitch using addresses on OSA1 portB and OSA2 port A? Existing VSWITCH Layer 3 def looks like: Define VSWITCH td6vsw1 rdev f400 f600 vlan 797 porttype trunk Can I just add the layer 2 one with something like: Define vswitch td6vsw2 rdev f410 f610 ETHERNET vlan 797 porttype trunk And then update Linux with *for SLES: QETH_LAYER2_SUPPORT=1 in /etc/sysconfig/hardware/hwcfg-qeth-bus-ccw-0.0. And that'll work?? Or must I come up with some more ports. Bonus points for explaining how I can get link aggregation involved :) Marcy Cortes This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. -Original Message- From: Linux on 390 Port [mailto:[EMAIL PROTECTED] On Behalf Of Alan Altmark Sent: Thursday, September 06, 2007 9:36 PM To: LINUX-390@VM.MARIST.EDU Subject: Re: [LINUX-390] VSwitch and IPv6 On Thursday, 09/06/2007 at 10:08 EDT, Robert J Brenneman [EMAIL PROTECTED] wrote: Yep - you need Layer2 - Setup the vswitch like Alan mentioned. BUT: the osa port can **not** support layer2 layer3 at the same time. To the best of my knowledge, all L2-capable OSAs support mixed L2 and L3. HOWEVER, only z9 with recent microcode allows L2 and L3 to crosstalk. Alan Altmark z/VM Development IBM Endicott -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 Brabant Water N.V. Postbus 1068 5200 BC 's-Hertogenbosch http://www.brabantwater.nl Handelsregister: 16005077 -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: Layer 2 - (was Re: VSwitch and IPv6)
Yay!! It works it works it works! Thank you Alan!!! I'll buy you and Jay a beer in San Antonio! Your bonus points are in the mail too. Marcy This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. -Original Message- From: Linux on 390 Port [mailto:[EMAIL PROTECTED] On Behalf Of Alan Altmark Sent: Friday, September 07, 2007 6:24 AM To: LINUX-390@VM.MARIST.EDU Subject: Re: [LINUX-390] VSwitch and IPv6 On Friday, 09/07/2007 at 01:40 EDT, Marcy Cortes [EMAIL PROTECTED] wrote: This is all confusing to me - this layer 2/3 stuff. It was said to be not allowed on the same card. Then not on the same port? Now OK on the same port (given I have the latest and greatest z9 stuff, that's easy - someone else gets to do that)? In the vernacular, card means port. Feature means card. Oh, wait. A single feature has two daughter cards in it. wah It is time to standardize our terminology, everyone. Architecturally: - Feature: The thing that you order and plug into the I/O cage. It has one or more chpids, each with one or ports. - Chpid: The channel that gives host access to one or more ports. - Port: The thing you plug the cable into. They are numbered, within each chpid, from 0 to n. For all of the OSAs, you have been using Port 0. Exceedingly convenient since all of the host interface definitions default to port 0. [To keep you on your toes, the Token Ring feature exhibits two physical connectors per port (DB-9 and RJ-45), but only one can be used. But that's TR. Who cares? All of the Ethernet features exhibit one physical connector per port.] See Appendix A of your OSA-Express Customer's Guide and Reference. Scenario - I have 2 OSA cards. 2 ports on each (duh). One port on each going to a diff switch. There's no duh. Not all OSA features have two ports. The 10 GbE feature has a single chpid with a single port. :-) OSA1 port A switch a port B switch b OSA2 port A switch a port B switch b VM TCPIP stack has OSA1 portA and OSA2 portB - layer 3 (and some guest LAN, but that doesn't matter). VSWITCH Layer 3 has OSA1 portB and OSA2 portA Can I add a Layer 2 vswitch here without disturbing what's there? That is, define 1 more vswitch using addresses on OSA1 portB and OSA2 port A? Yes. Existing VSWITCH Layer 3 def looks like: Define VSWITCH td6vsw1 rdev f400 f600 vlan 797 porttype trunk Can I just add the layer 2 one with something like: Define vswitch td6vsw2 rdev f410 f610 ETHERNET vlan 797 porttype trunk And then update Linux with *for SLES: QETH_LAYER2_SUPPORT=1 in /etc/sysconfig/hardware/hwcfg-qeth-bus-ccw-0.0. And that'll work?? Yes. The only issue with L2/L3 on the same port is that, up to now, hosts using L2 could not talk to hosts using L3 on the same OSA port. That is, you couldn't have them on the same subnet and expect them to talk to each other. If the servers don't talk to each other Or must I come up with some more ports. Only if you - want to use Link Aggregation - have network security issues to deal with And, of course, you can't (technically can, but really really shouldn't) mix VLAN-tagged traffic and non-VLAN-tagged traffic on the same OSA port. Bonus points for explaining how I can get link aggregation involved :) Use SET PORT GROUP to create a named list of OSA RDEVs. Use the GROUP name instead of RDEVs on DEFINE VSWITCH. Create a similar group on the physical switch. (Both ends of the cable have to be configured to operate in a link agg. group.) Et voila. z/VM 5.3 only. Because the link aggregation protocol requires [v]switch-switch control flows, not just data, you must dedicate the entire OSA port to the VSWITCH. No sharing. At some point the OSAs will be updated to detect this and will not allow active sharing of an OSA port that is using link aggregation. Alan Altmark z/VM Development IBM Endicott -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: VSwitch and IPv6
On Thursday, 09/06/2007 at 08:36 EDT, Scully, William P [EMAIL PROTECTED] wrote: This leads me to believe that I need a VSwitch similar to: Define VSwitch INTRAV6 RDev 6C00 Connect Controller TCPIPOSA Ethernet Correct. However, in doing so I'm not clear how, in Linux for zSeries, to configure the IP stack. The IBM Device Drivers doc seems to imply that I should be setting the layer2 attribute of the qeth driver. Is that so? Or am I walking down a path with really isn't ideal? Yes, you need to set the layer2 attribute, but you do that by setting (e.g. LAYER2=) in the device configuration file. Alan Altmark z/VM Development IBM Endicott -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: VSwitch and IPv6
Yep - you need Layer2 - Setup the vswitch like Alan mentioned. BUT: the osa port can **not** support layer2 layer3 at the same time. If all your ports are in use as layer3 at the moment, you'll have to free one up to initialize it in layer2. The first connection to the osa sets the mode, and everyone else who shares it has to follow suit. To get the Linux driver in Layer 2 mode you have to stick a value a file, but that varies depending on which distribution youre using. for RHEL5: *OPTIONS=layer2=1* in /etc/sysconfig/network-scripts/ifcfg-ethX * *for SLES: QETH_LAYER2_SUPPORT=1 in /etc/sysconfig/hardware/hwcfg- qeth-bus-ccw-0.0. I pulled that from Mark Ver's RHEL4 / SLES9 configuration cross reference at http://linuxvm.org/present/SHARE107/S9211mv.pdf RHEL 5 SLES 10 should be close enough to try the same things on. -- Jay Brenneman -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: VSwitch and IPv6
This is all confusing to me - this layer 2/3 stuff. It was said to be not allowed on the same card. Then not on the same port? Now OK on the same port (given I have the latest and greatest z9 stuff, that's easy - someone else gets to do that)? Example needed please. I'll give you a little picture but I'm a little graphically challenged. Scenario - I have 2 OSA cards. 2 ports on each (duh). One port on each going to a diff switch. OSA1 port A switch a port B switch b OSA2 port A switch a port B switch b VM TCPIP stack has OSA1 portA and OSA2 portB - layer 3 (and some guest LAN, but that doesn't matter). VSWITCH Layer 3 has OSA1 portB and OSA2 portA Can I add a Layer 2 vswitch here without disturbing what's there? That is, define 1 more vswitch using addresses on OSA1 portB and OSA2 port A? Existing VSWITCH Layer 3 def looks like: Define VSWITCH td6vsw1 rdev f400 f600 vlan 797 porttype trunk Can I just add the layer 2 one with something like: Define vswitch td6vsw2 rdev f410 f610 ETHERNET vlan 797 porttype trunk And then update Linux with *for SLES: QETH_LAYER2_SUPPORT=1 in /etc/sysconfig/hardware/hwcfg-qeth-bus-ccw-0.0. And that'll work?? Or must I come up with some more ports. Bonus points for explaining how I can get link aggregation involved :) Marcy Cortes This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. -Original Message- From: Linux on 390 Port [mailto:[EMAIL PROTECTED] On Behalf Of Alan Altmark Sent: Thursday, September 06, 2007 9:36 PM To: LINUX-390@VM.MARIST.EDU Subject: Re: [LINUX-390] VSwitch and IPv6 On Thursday, 09/06/2007 at 10:08 EDT, Robert J Brenneman [EMAIL PROTECTED] wrote: Yep - you need Layer2 - Setup the vswitch like Alan mentioned. BUT: the osa port can **not** support layer2 layer3 at the same time. To the best of my knowledge, all L2-capable OSAs support mixed L2 and L3. HOWEVER, only z9 with recent microcode allows L2 and L3 to crosstalk. Alan Altmark z/VM Development IBM Endicott -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 smime.p7s Description: S/MIME cryptographic signature
Re: VSwitch and IPv6
On Thursday, 09/06/2007 at 10:08 EDT, Robert J Brenneman [EMAIL PROTECTED] wrote: Yep - you need Layer2 - Setup the vswitch like Alan mentioned. BUT: the osa port can **not** support layer2 layer3 at the same time. To the best of my knowledge, all L2-capable OSAs support mixed L2 and L3. HOWEVER, only z9 with recent microcode allows L2 and L3 to crosstalk. Alan Altmark z/VM Development IBM Endicott -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
