Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-23 Thread Hal Rosenstock
On Thu, 2007-02-22 at 17:18, Sean Hefty wrote: > >>Can someone help my understanding here? Is ipoib joining a multicast group > >>using the full membership PKey, even if the node that it joins from only > >>has the > >>limited membership PKey configured? And the code in ib_find_cached_pkey > >>h

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-23 Thread Hal Rosenstock
On Thu, 2007-02-22 at 17:18, Sean Hefty wrote: > >>Can someone help my understanding here? Is ipoib joining a multicast group > >>using the full membership PKey, even if the node that it joins from only > >>has the > >>limited membership PKey configured? And the code in ib_find_cached_pkey > >>h

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-22 Thread Sean Hefty
>>Can someone help my understanding here? Is ipoib joining a multicast group >>using the full membership PKey, even if the node that it joins from only has >>the >>limited membership PKey configured? And the code in ib_find_cached_pkey helps >>enable this? > > Yep. The ipoib create_child functi

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-22 Thread Or Gerlitz
On 2/22/07, Sean Hefty <[EMAIL PROTECTED]> wrote: > >My understanding is that when an IPoIB broadcast domain contains both > >partial and full members (*) attempts to communicate between two partial > >members would silently fail, does this silence is something you think we > >should work to change

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-22 Thread Or Gerlitz
On 2/22/07, Sean Hefty <[EMAIL PROTECTED]> wrote: > >An IB multicast group _cannot_ have partial members so this never should > >get far enough to where two limited members would be unable to > >communicate. > Can someone help my understanding here? Is ipoib joining a multicast group > using the

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-22 Thread Sean Hefty
>An IB multicast group _cannot_ have partial members so this never should >get far enough to where two limited members would be unable to >communicate. Can someone help my understanding here? Is ipoib joining a multicast group using the full membership PKey, even if the node that it joins from on

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-22 Thread Sean Hefty
>My understanding is that when an IPoIB broadcast domain contains both >partial and full members (*) attempts to communicate between two partial >members would silently fail, does this silence is something you think we >should work to change? I'm looking at this from a different view than just ipo

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-22 Thread Hal Rosenstock
On Thu, 2007-02-22 at 03:04, Or Gerlitz wrote: > Sean Hefty wrote: > >> Note that since the HCA validates the pkey in the in coming packet, no > >> matter what the IB SW would do, partial members of a partition can't > >> talk to each other. So the approach taken by the core/ipoib code was > >> to

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-22 Thread Hal Rosenstock
On Thu, 2007-02-22 at 02:28, Or Gerlitz wrote: > Hal Rosenstock wrote: > > On Wed, 2007-02-21 at 15:45, Or Gerlitz wrote: > >> On 21 Feb 2007 08:20:23 -0500, Hal Rosenstock <[EMAIL PROTECTED]> wrote: > > >> If the IPoIB spec does not allow both partial and full members of a > >> partition to share

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-22 Thread Or Gerlitz
Sean Hefty wrote: >> Note that since the HCA validates the pkey in the in coming packet, no >> matter what the IB SW would do, partial members of a partition can't >> talk to each other. So the approach taken by the core/ipoib code was >> to just ignore the MSb in places where the code looks for th

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-21 Thread Or Gerlitz
Hal Rosenstock wrote: > On Wed, 2007-02-21 at 15:45, Or Gerlitz wrote: >> On 21 Feb 2007 08:20:23 -0500, Hal Rosenstock <[EMAIL PROTECTED]> wrote: >> If the IPoIB spec does not allow both partial and full members of a >> partition to share a broadcast domain (eg the IPv4 broadcast group >> associa

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-21 Thread Hal Rosenstock
On Wed, 2007-02-21 at 17:53, Hal Rosenstock wrote: > On Wed, 2007-02-21 at 17:36, Sean Hefty wrote: > > > It does this since its makes life simple and robust. > > > > Is an SM prevented from loading two PKeys into an HCA's PKey table that > > differ > > by only the membership bit? > > Nope. >

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-21 Thread Hal Rosenstock
On Wed, 2007-02-21 at 17:36, Sean Hefty wrote: > > It does this since its makes life simple and robust. > > Is an SM prevented from loading two PKeys into an HCA's PKey table that > differ > by only the membership bit? Nope. > I can't think of any reason to do such a thing, Me neither. It wou

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-21 Thread Sean Hefty
> It does this since its makes life simple and robust. Is an SM prevented from loading two PKeys into an HCA's PKey table that differ by only the membership bit? I can't think of any reason to do such a thing, but depending on which index was selected could limit which nodes you could communic

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-21 Thread Hal Rosenstock
On Wed, 2007-02-21 at 15:45, Or Gerlitz wrote: > On 21 Feb 2007 08:20:23 -0500, Hal Rosenstock <[EMAIL PROTECTED]> wrote: > > On Wed, 2007-02-21 at 07:35, Or Gerlitz wrote: > > > > > I believe it is a spec (compliance) violation for the port to be a > > > > partial member and join as a full member

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-21 Thread Or Gerlitz
On 2/21/07, Sean Hefty <[EMAIL PROTECTED]> wrote: > >There is no problem. As i have explained over this thread the ipoib > >and the core abstract away from the user the actual value of the MSb > >of the pkey, that is whether it is a full or partial membership pkey. > > But *why* does the kernel cod

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-21 Thread Or Gerlitz
On 21 Feb 2007 08:20:23 -0500, Hal Rosenstock <[EMAIL PROTECTED]> wrote: > On Wed, 2007-02-21 at 07:35, Or Gerlitz wrote: > > > I believe it is a spec (compliance) violation for the port to be a > > > partial member and join as a full member. > > Since partial members can't talk among themselves,

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-21 Thread Sean Hefty
>There is no problem. As i have explained over this thread the ipoib >and the core abstract away from the user the actual value of the MSb >of the pkey, that is whether it is a full or partial membership pkey. But *why* does the kernel code do this, and should it? - Sean

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-21 Thread Or Gerlitz
On 2/21/07, Sean Hefty <[EMAIL PROTECTED]> wrote: >>However, no matter what the SM configures, the core & ipoib code act as >>the full pkey is there. This is nice simplification and it works well. > Is the problem here really in the librdmacm or in the core/ipoib software? There is no problem. As

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-21 Thread Hal Rosenstock
On Wed, 2007-02-21 at 07:35, Or Gerlitz wrote: > >> However, no matter what the SM configures, the core & ipoib code act as > >> the full pkey is there. This is nice simplification and it works well. > > > I believe it is a spec (compliance) violation for the port to be a > > partial member and j

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-21 Thread Or Gerlitz
>> However, no matter what the SM configures, the core & ipoib code act as >> the full pkey is there. This is nice simplification and it works well. > I believe it is a spec (compliance) violation for the port to be a > partial member and join as a full member. Since partial members can't talk a

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-21 Thread Hal Rosenstock
On Wed, 2007-02-21 at 01:43, Or Gerlitz wrote: > Hal Rosenstock wrote: > > On Tue, 2007-02-20 at 10:38, Or Gerlitz wrote: > > >> Yes. Its a little bit confusing: partial and full members of an IPoIB IB > >> partition use the same MGID. When an IPoIB MGID is constructed, the pkey > >> placed by t

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-21 Thread Or Gerlitz
>> Yes. Its a little bit confusing: partial and full members of an IPoIB IB >> partition use the same MGID. When an IPoIB MGID is constructed, the pkey >> placed by the driver is --always-- the full membership one. However, on >> a node with partial membership, what's plugged into the QP is the

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-20 Thread Or Gerlitz
Hal Rosenstock wrote: > On Tue, 2007-02-20 at 10:38, Or Gerlitz wrote: >> Yes. Its a little bit confusing: partial and full members of an IPoIB IB >> partition use the same MGID. When an IPoIB MGID is constructed, the pkey >> placed by the driver is --always-- the full membership one. However, o

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-20 Thread Hal Rosenstock
On Tue, 2007-02-20 at 10:38, Or Gerlitz wrote: > Hal Rosenstock wrote: > > >> The pkey extracted by the RDMA CM from the IPoIB device hardware address > >> always > >> has the full membership bit set. However, when looking in the pkey table > >> the > >> search must mask out the full membership

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-20 Thread Or Gerlitz
Hal Rosenstock wrote: >> The pkey extracted by the RDMA CM from the IPoIB device hardware address >> always >> has the full membership bit set. However, when looking in the pkey table the >> search must mask out the full membership bit. > Is this true for both RC and UD QPs ? I thought that at l

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-20 Thread Hal Rosenstock
On Mon, 2007-02-19 at 01:40, Or Gerlitz wrote: > Hi Sean, > > this fixes a bug which did not allow to run librdmacm apps over a node > which is partial member of a partition. The patch takes the approach of the > kernel ib_find_cached_pkey implementation. > > If you approve this, i suggest pushin

Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-19 Thread Moni Levy
Or, On 2/19/07, Or Gerlitz <[EMAIL PROTECTED]> wrote: > Hi Sean, > > this fixes a bug which did not allow to run librdmacm apps over a node > which is partial member of a partition. The patch takes the approach of the > kernel ib_find_cached_pkey implementation. > > If you approve this, i suggest p

[openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey

2007-02-18 Thread Or Gerlitz
Hi Sean, this fixes a bug which did not allow to run librdmacm apps over a node which is partial member of a partition. The patch takes the approach of the kernel ib_find_cached_pkey implementation. If you approve this, i suggest pushing it also into OFED 1.2 as a bug fix. Or. -