Re: OSA Adapter TCP/IP stack association limit?
On Thursday, 08/07/2008 at 11:00 EDT, Robert J Brenneman <[EMAIL PROTECTED]> wrote: > The vswitch counts as 1 stack on the OSA, no matter how many systems are behind > it. So yes - it gets you around the 640 stacks per OSA issue. That limitation > came from the number of subchannels you could generate on the OSA chpid itself. > I think it's even higher than 640 for the recent z10 and the newest OSA > adapters. In each edition of the OSA Express Customer Guide and Reference (current is -09) from ResourceLink, appendix B contains the stack/address limits and hardware-specific things such as the MTU size (1492 or 8992). OSA Express has a limit of 480 devices and 160 stacks. OSA Express2 and 3 have a limit of 1920 devices and 640 stacks. The OSA Express3 is available only on the z10. It has the same general configuration characteristics as the OSA Express2, but (a) It has only GbE and 10 GbE (no 1000-baseT) (b) The number of ports per card (feature) is doubled (c) It has better performance Alan Altmark z/VM Development IBM Endicott
Re: OSA Adapter TCP/IP stack association limit?
I see this update in the online version of http://publibz.boulder.ibm.com/epubs/pdf/ioa2z150.pdf (see p. 12): "Beginning with the May 2006 version of Licensed Internal Code on z890, z990 and z9 systems only, this restriction has been lifted. With this version of Licensed Internal Code, a layer 2 host can directly communicate with a layer 3 host sharing the same OSA-Express port." So - it is even lifted for z990. I don't know what the release number is for the May 2006 version of the code. On Thu, Aug 7, 2008 at 1:20 PM, r.stricklin <[EMAIL PROTECTED]> wrote: > On Aug 7, 2008, at 10:10 AM, Bruce Hayden wrote: > >> The z9, with updated OSA microcode, can also run the OSA port in >> "shared" layer 2 and layer 3 mode.. > > Bruce; > > This would be an important feature for us. Do you know the specific OSA > microcode level which is known to support it? > > Thanks. > > ok > bear > -- Bruce Hayden Linux on System z Advanced Technical Support IBM, Endicott, NY
Re: OSA Adapter TCP/IP stack association limit?
> The z9, with updated OSA microcode, can also run the OSA port in > "shared" layer 2 and layer 3 mode.. Nifty. Thanks -- one of these days I'll get my brain upgraded for more short term storage. 8-) --db
Re: OSA Adapter TCP/IP stack association limit?
On Aug 7, 2008, at 10:10 AM, Bruce Hayden wrote: The z9, with updated OSA microcode, can also run the OSA port in "shared" layer 2 and layer 3 mode.. Bruce; This would be an important feature for us. Do you know the specific OSA microcode level which is known to support it? Thanks. ok bear
Re: OSA Adapter TCP/IP stack association limit?
The z9, with updated OSA microcode, can also run the OSA port in "shared" layer 2 and layer 3 mode.. On Thu, Aug 7, 2008 at 11:16 AM, David Boyes <[EMAIL PROTECTED]> wrote: > Downside is that unless you have a z10, I don't think you can have a OSA > card in layer 3 and layer 2 mode simultaneously, so you can't share a > card with a LPAR that requires layer 3 mode. Probably not a big deal for > you, as you're likely to push the limits of a card and not want to share > it anyway, but YMMV. > -- Bruce Hayden Linux on System z Advanced Technical Support IBM, Endicott, NY
Re: OSA Adapter TCP/IP stack association limit?
> Does this approach get around the stack limit some way? If we try to put > 800 > machines on a VLAN, will this blow the limit referenced below? The layer 2 TYPE ETHERNET VSWITCH gets the OSA limits out of the line of fire entirely -- the OSA(s) (note multiple) servicing the VSWITCH just forward frames, acting like a bridge to the VLAN. You can put lots more than 800 guests on a VLAN. I know of people with well over 2000 guests of various types on a single VLAN sharing a VSWITCH. Downside is that unless you have a z10, I don't think you can have a OSA card in layer 3 and layer 2 mode simultaneously, so you can't share a card with a LPAR that requires layer 3 mode. Probably not a big deal for you, as you're likely to push the limits of a card and not want to share it anyway, but YMMV. The limit applies only to device triples directly attached to individual guests, which is a not so hot idea under VM anyway given that VSWITCH exists. It'll also simplify the ability to move virtual machines if IBM ever gets us such a thing -- not tying the adapters to actual physical devices is a Good Thing.
Re: OSA Adapter TCP/IP stack association limit?
The vswitch counts as 1 stack on the OSA, no matter how many systems are behind it. So yes - it gets you around the 640 stacks per OSA issue. That limitation came from the number of subchannels you could generate on the OSA chpid itself. I think it's even higher than 640 for the recent z10 and the newest OSA adapters. I don't think there is an explicit limit to the number of guests you can connect to a vswitch, but I think you'll run into network performance issues with too many systems in the same broadcast domain before you hit any architectural limits of the vswitch itself. I don't know about you, but I've never actually seen 800 systems on the same layer2 network. It may be better to fence off those 800 machines into 4 separate logical VLANs of 200 systems each. They can all run on the same vswitch under VM, but you have to break it up so the broadcasts don't clog it up. This is only a concern if you're running systems that do lots of broadcasts, of course. -- Jay Brenneman
Re: OSA Adapter TCP/IP stack association limit?
On 8/6/08 3:48 PM, "Alan Altmark" <[EMAIL PROTECTED]> wrote: > Yes, there is (the PRIROUTER option), but who cares? Define the VSWITCH > as ETHERNET (layer 2), not IP. Does this approach get around the stack limit some way? If we try to put 800 machines on a VLAN, will this blow the limit referenced below? I finally found the text in a Share presentation made by Steffan Thoss, IBM Boeblingen. "Primary Network Device: OSA Express 'Integrated Power computer' with network daughter card Shared between up to 640 TCP/IP stacks" --. .- .-. -.-- Gary Dennis Mantissa Corporation
Re: OSA Adapter TCP/IP stack association limit?
On Wednesday, 08/06/2008 at 04:42 EDT, Rob van der Heij <[EMAIL PROTECTED]> wrote: > On Wed, Aug 6, 2008 at 8:35 PM, Mark Post <[EMAIL PROTECTED]> wrote: > > > >> Is that stack association limit correct? If this is the case, is there a > >> circumvention? Can multiple OSA adapters be associated with a single VLAN? > > > > Use a VSWITCH, since you've already decided this is going to be run on z/VM. > > The limit is model dependent, for z9 way more than 640 IIRC. > > The way to get around it (once) is to make the VM system primary > router so it will get all packets for which the OSA has no entry in > the OAT. I think the limit also applies to the number of stacks on the > VSWITCH. There isn't a way for VSWITCH to set primary router, is > there? Yes, there is (the PRIROUTER option), but who cares? Define the VSWITCH as ETHERNET (layer 2), not IP. Alan Altmark z/VM Development IBM Endicott
Re: OSA Adapter TCP/IP stack association limit?
On Wed, Aug 6, 2008 at 8:35 PM, Mark Post <[EMAIL PROTECTED]> wrote: >> Is that stack association limit correct? If this is the case, is there a >> circumvention? Can multiple OSA adapters be associated with a single VLAN? > > Use a VSWITCH, since you've already decided this is going to be run on z/VM. The limit is model dependent, for z9 way more than 640 IIRC. The way to get around it (once) is to make the VM system primary router so it will get all packets for which the OSA has no entry in the OAT. I think the limit also applies to the number of stacks on the VSWITCH. There isn't a way for VSWITCH to set primary router, is there? Rob
Re: OSA Adapter TCP/IP stack association limit?
Will using VSWITCH get us around the 640 limit per OSA adapter? --. .- .-. -.-- Gary Dennis On 8/6/08 1:35 PM, "Mark Post" <[EMAIL PROTECTED]> wrote: On 8/6/2008 at 2:24 PM, in message <[EMAIL PROTECTED]>, > "Gary M. Dennis" <[EMAIL PROTECTED]> wrote: >> Did I read somewhere (the "where" being a place I cannot at this point >> locate) that the number of IP stacks which could be associated with a single >> OSA adapter was 640? > > That sounds right. > > -snip- >> Is that stack association limit correct? If this is the case, is there a >> circumvention? Can multiple OSA adapters be associated with a single VLAN? > > Use a VSWITCH, since you've already decided this is going to be run on z/VM. > > > Mark Post >
Re: OSA Adapter TCP/IP stack association limit?
>>> On 8/6/2008 at 2:24 PM, in message <[EMAIL PROTECTED]>, "Gary M. Dennis" <[EMAIL PROTECTED]> wrote: > Did I read somewhere (the "where" being a place I cannot at this point > locate) that the number of IP stacks which could be associated with a single > OSA adapter was 640? That sounds right. -snip- > Is that stack association limit correct? If this is the case, is there a > circumvention? Can multiple OSA adapters be associated with a single VLAN? Use a VSWITCH, since you've already decided this is going to be run on z/VM. Mark Post
