Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On Mon, Apr 21, 2014 at 9:09 PM, Christoph Lameter wrote: > We need this option for many hosts to avoid backflow of multicast > packages. Could we get that merged? [...] > Currently, there's no way for user-space applications to specify > the IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK QP creation flags > defined by commit 47ee1b9 "IB/core: Add support for multicast loopback > blocking". > As a result, applications who send and recieve on the same QP over > the same multicast group get all their packets bouncded back to them, > which is bad.. Hi Christoph, I do see your P&F (Pain and Frustration), however, this patch was only meant to be bandaid (*) for the fact that whole ~six years elapsed since a basically proper patch that does the task was posted http://lists.openfabrics.org/pipermail/general/2008-July/053021.html and we still don't have uverbs support for QP creation flags!! Roland, how you prefer to move on here? we can come up with quick patch that just uses the reserved field of struct ib_uverbs_create_qp and would allow to support upto 8 creation flags, this would not require new uverbs command, OK? Roland, we must fix this lack of user space to provide creation flags. The bandaid I did after a whole production system was successfully rebased to the upstream kernel sans this PITA which I has to solve quickly in this uglish way. Or. (*) this is just bandaid and not for upstream since it touches just one HW driver, i.e a similar patch you would need for every other HW driver, and deals with only one QP creation flag. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On Mon, 21 Apr 2014, Or Gerlitz wrote: > > Currently, there's no way for user-space applications to specify > > the IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK QP creation flags > > defined by commit 47ee1b9 "IB/core: Add support for multicast loopback > > blocking". > > As a result, applications who send and recieve on the same QP over > > the same multicast group get all their packets bouncded back to them, > > which is bad.. > > Hi Christoph, I do see your P&F (Pain and Frustration), however, this > patch was only meant to be bandaid (*) for the fact that whole ~six > years elapsed since a basically proper patch that does the task was > posted http://lists.openfabrics.org/pipermail/general/2008-July/053021.html > and we still don't have uverbs support for QP creation flags!! > > Roland, how you prefer to move on here? we can come up with quick > patch that just uses the reserved field of struct ib_uverbs_create_qp > and would allow to support upto 8 creation flags, this would not > require new uverbs command, OK? Urgh. So we have all these flags and we cannot use them? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On Tue, Apr 22, 2014 at 12:06 AM, Christoph Lameter wrote: > On Mon, 21 Apr 2014, Or Gerlitz wrote: > >> > Currently, there's no way for user-space applications to specify >> > the IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK QP creation flags >> > defined by commit 47ee1b9 "IB/core: Add support for multicast loopback >> > blocking". >> > As a result, applications who send and recieve on the same QP over >> > the same multicast group get all their packets bouncded back to them, >> > which is bad.. >> >> Hi Christoph, I do see your P&F (Pain and Frustration), however, this >> patch was only meant to be bandaid (*) for the fact that whole ~six >> years elapsed since a basically proper patch that does the task was >> posted http://lists.openfabrics.org/pipermail/general/2008-July/053021.html >> and we still don't have uverbs support for QP creation flags!! >> >> Roland, how you prefer to move on here? we can come up with quick >> patch that just uses the reserved field of struct ib_uverbs_create_qp >> and would allow to support upto 8 creation flags, this would not >> require new uverbs command, OK? > > Urgh. So we have all these flags and we cannot use them? Christoph, the support (kernel IB core API and HW drivers e.g mlx4 and now also mlx5) is there for **kernel** consumers (e.g IPoIB). and missing for **user-space** consumers whose calls are tunneled through uverbs. The uverbs piece is the missing one. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On Tue, 22 Apr 2014, Or Gerlitz wrote: > > Urgh. So we have all these flags and we cannot use them? > > Christoph, the support (kernel IB core API and HW drivers e.g mlx4 and > now also mlx5) is there for **kernel** consumers (e.g IPoIB). and > missing for **user-space** consumers whose calls are tunneled through > uverbs. The uverbs piece is the missing one. Could you please post an updated patch? We will test. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
Note that I am pretty unhappy about Mellanox right now. 8 year old patches that have never been refreshed? If I submit patchsets then I will post these things regularly until they are accepted. You need to have a patch queue or a git tree with up to date patches that contains all the patches you intend to merge. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On 22/04/2014 16:48, Christoph Lameter wrote: Note that I am pretty unhappy about Mellanox right now. 8 year old patches that have never been refreshed? If I submit patchsets then I will post these things regularly until they are accepted. You need to have a patch queue or a git tree with up to date patches that contains all the patches you intend to merge. Chritoph, again I can hear your Pain and Frustration (BTW the patch is from 2008 so only six years old...), when patches are repeatedly ignored, there's no much point in reposting them again and again. We are making sure that large patche sets are posted in a public git tree, for example the T10 signature patches were on the floor for 5-6 months and eventually went into 3.15 and during that hang time, we have put them in public git tree, I will make sure we do it right away for the ODP patches and other user space patches (RoCE UD and Flow-steering) that were posted here and still not accepted. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On Tue, 22 Apr 2014, Or Gerlitz wrote: > Chritoph, again I can hear your Pain and Frustration (BTW the patch is from > 2008 so only six years old...), when patches are repeatedly ignored, there's > no much point in reposting them again and again. We are making sure that large > patche sets are posted in a public git tree, for example the T10 signature > patches were on the floor for 5-6 months and eventually went into 3.15 and > during that hang time, we have put them in public git tree, I will make sure > we do it right away for the ODP patches and other user space patches (RoCE UD > and Flow-steering) that were posted here and still not accepted. Ok I saw the ODP patches. But where are the multicast block patches? Is there any way to move this forward? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On Tue, Apr 29, 2014 at 8:32 PM, Christoph Lameter wrote: > Ok I saw the ODP patches. But where are the multicast block patches? Is > there any way to move this forward? We can try, earlier on this thread I wrote "Roland, how you prefer to move on here? we can come up with quick patch that just uses the reserved field of struct ib_uverbs_create_qp and would allow to support upto 8 creation flags, this would not require new uverbs command, OK?" and no reply... Or. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On Wed, 30 Apr 2014, Or Gerlitz wrote: > On Tue, Apr 29, 2014 at 8:32 PM, Christoph Lameter wrote: > > > Ok I saw the ODP patches. But where are the multicast block patches? Is > > there any way to move this forward? > > We can try, earlier on this thread I wrote > > "Roland, how you prefer to move on here? we can come up with quick > patch that just uses the reserved field of struct ib_uverbs_create_qp > and would allow to support upto 8 creation flags, this would not > require new uverbs command, OK?" Post the patch please? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
- Original Message - > We need this option for many hosts to avoid backflow of multicast > packages. Could we get that merged? > > > > From 18ceae090b02b3055382e11c305dcb334d938122 Mon Sep 17 00:00:00 > 2001 > From: Or Gerlitz > Date: Tue, 4 Mar 2014 17:20:00 +0200 > Subject: [PATCH] IB/mlx4: Allow to always block UD multicast loopback > > Currently, there's no way for user-space applications to specify > the IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK QP creation flags defined > by commit 47ee1b9 "IB/core: Add support for multicast loopback > blocking". > > As a result, applications who send and recieve on the same QP over > the same multicast group get all their packets bouncded back to them, > which is bad.. > > Add a module param to the mlx4_ib driver which allows to force all QP > attachements to be carried out with multicast loopback prevented. This seems like driving a thumb tack with a sledge hammer. No doubt the correct path here is to get the proper flag added to verbs multicast attach command. Or, do you still have a patch that does this the right way? Did the one from 2008 do it the right way or this way? > Signed-off-by: Or Gerlitz > --- > drivers/infiniband/hw/mlx4/main.c |9 +++-- > 1 files changed, 7 insertions(+), 2 deletions(-) > > diff --git a/drivers/infiniband/hw/mlx4/main.c > b/drivers/infiniband/hw/mlx4/main.c > index e81c554..fa01a9e 100644 > --- a/drivers/infiniband/hw/mlx4/main.c > +++ b/drivers/infiniband/hw/mlx4/main.c > @@ -68,6 +68,10 @@ int mlx4_ib_sm_guid_assign = 1; > module_param_named(sm_guid_assign, mlx4_ib_sm_guid_assign, int, > 0444); > MODULE_PARM_DESC(sm_guid_assign, "Enable SM alias_GUID assignment if > sm_guid_assign > 0 (Default: 1)"); > > +int mlx4_block_loopback = 0; > +module_param_named(block_loopback, mlx4_block_loopback, int, 0644); > +MODULE_PARM_DESC(block_loopback, "Always block UD multicast loopback > packets if > 0"); > + > static const char mlx4_ib_version[] = > DRV_NAME ": Mellanox ConnectX InfiniBand driver v" > DRV_VERSION " (" DRV_RELDATE ")\n"; > @@ -1163,6 +1167,8 @@ static int mlx4_ib_mcg_attach(struct ib_qp > *ibqp, union ib_gid *gid, u16 lid) > struct mlx4_ib_steering *ib_steering = NULL; > enum mlx4_protocol prot = (gid->raw[1] == 0x0e) ? > MLX4_PROT_IB_IPV4 : MLX4_PROT_IB_IPV6; > + int block_mcast_loopback = (!!(mqp->flags & > MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK)) | > +mlx4_block_loopback; > > if (mdev->dev->caps.steering_mode == > MLX4_STEERING_MODE_DEVICE_MANAGED) { > @@ -1172,8 +1178,7 @@ static int mlx4_ib_mcg_attach(struct ib_qp > *ibqp, union ib_gid *gid, u16 lid) > } > > err = mlx4_multicast_attach(mdev->dev, &mqp->mqp, gid->raw, > mqp->port, > - !!(mqp->flags & > -MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK), > + block_mcast_loopback, > prot, ®_id); > if (err) > goto err_malloc; > -- > 1.7.1 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" > in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Doug Ledford GPG KeyID: 0E572FDD http://people.redhat.com/dledford -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On 01/05/2014 02:28, Doug Ledford wrote: This seems like driving a thumb tack with a sledge hammer. No doubt the correct path here is to get the proper flag added to verbs multicast attach command. Or, do you still have a patch that does this the right way? Did the one from 2008 do it the right way or this way? Doug, as I commented earlier on this thread, the correct way here is to carry out the task is to repeat the QP creation flags which we have in the kernel verbs API for years (see IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK in ib_verbs.h and usage in IPoIB) for the user space verbs API. The patch I provided to Christophand CO was a band-aid to bring up a whole system on upstream and mlx4 where this (avoiding multicast loopback) was the only/major missing piece. The patch posted back in 2008 is proper, but it added a whole new uverb call/structure, where I think we can get along here with just re-using a reserved field in the existing uverbs API, see my other post on this thread. Or. Or. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On Wed, 30 Apr 2014, Doug Ledford wrote: > > Add a module param to the mlx4_ib driver which allows to force all QP > > attachements to be carried out with multicast loopback prevented. > > This seems like driving a thumb tack with a sledge hammer. No doubt > the correct path here is to get the proper flag added to verbs multicast > attach command. Or, do you still have a patch that does this the > right way? Did the one from 2008 do it the right way or this way? This is the least invasive approach to do what we need and this approach has been a part of OFED for long years. The 2008 approach would be cleaner but it depends on an infrastructure change. I really would like to see that patch brought up to date and discussed. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
- Original Message - > On Wed, 30 Apr 2014, Doug Ledford wrote: > > > > Add a module param to the mlx4_ib driver which allows to force > > > all QP > > > attachements to be carried out with multicast loopback prevented. > > > > This seems like driving a thumb tack with a sledge hammer. No > > doubt > > the correct path here is to get the proper flag added to verbs > > multicast > > attach command. Or, do you still have a patch that does this the > > right way? Did the one from 2008 do it the right way or this way? > > This is the least invasive approach to do what we need and this > approach > has been a part of OFED for long years. I get that it's non-invasive and when doing something to get around the fact that it doesn't exist upstream, that makes sense. > The 2008 approach would be cleaner but it depends on an > infrastructure > change. I really would like to see that patch brought up to date and > discussed. Indeed. The non-invasive approach is the wrong one for upstream, it's really only suited for being a bandaid out of tree. For upstream, it needs to be done right. -- Doug Ledford GPG KeyID: 0E572FDD http://people.redhat.com/dledford -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
- Original Message - > On 01/05/2014 02:28, Doug Ledford wrote: > > This seems like driving a thumb tack with a sledge hammer. No > > doubt > > the correct path here is to get the proper flag added to verbs > > multicast > > attach command. Or, do you still have a patch that does this the > > right way? Did the one from 2008 do it the right way or this way? > > Doug, as I commented earlier on this thread, the correct way here is > to > carry out the task is to repeat the QP creation flags which we have > in > the kernel verbs API for years (see > IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK in ib_verbs.h and usage in > IPoIB) I'm curious why this is done at QP creation time. Should this not be done at multicast join time? A single QP can be a member of more than one MC group, would it not seem more correct to you that the loopback should be specified on a per group basis and not a per QP basis? > for the user space verbs API. The patch I provided to Christophand CO > was a band-aid to bring up a whole system on upstream and mlx4 where > this (avoiding multicast loopback) was the only/major missing piece. > > The patch posted back in 2008 is proper, but it added a whole new > uverb > call/structure, where I think we can get along here with just > re-using a > reserved field in the existing uverbs API, see my other post on this > thread. > > Or. > > Or. > -- Doug Ledford GPG KeyID: 0E572FDD http://people.redhat.com/dledford -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On Thu, 1 May 2014, Doug Ledford wrote: > Indeed. The non-invasive approach is the wrong one for upstream, it's > really only suited for being a bandaid out of tree. For upstream, it > needs to be done right. Can we get the bandaid into RH7 for now? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
Tried to forwardport the 2008 patch but there is XRC stuff and other things in there. The main function ib_uverbs_create_qp_expanded needs some attention. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
> I'm curious why this is done at QP creation time. Should this not > be done at multicast join time? A single QP can be a member of more > than one MC group, would it not seem more correct to you that the > loopback should be specified on a per group basis and not a per QP > basis? I agree with Doug here. This makes more sense to specify on multicast attach time, not QP creation time.
RE: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
?? On Mon, 5 May 2014, Hefty, Sean wrote: > PiBJJ20gY3VyaW91cyB3aHkgdGhpcyBpcyBkb25lIGF0IFFQIGNyZWF0aW9uIHRpbWUuICBTaG91 > bGQgdGhpcyBub3QNCj4gYmUgZG9uZSBhdCBtdWx0aWNhc3Qgam9pbiB0aW1lPyAgQSBzaW5nbGUg > UVAgY2FuIGJlIGEgbWVtYmVyIG9mIG1vcmUNCj4gdGhhbiBvbmUgTUMgZ3JvdXAsIHdvdWxkIGl0 > IG5vdCBzZWVtIG1vcmUgY29ycmVjdCB0byB5b3UgdGhhdCB0aGUNCj4gbG9vcGJhY2sgc2hvdWxk > IGJlIHNwZWNpZmllZCBvbiBhIHBlciBncm91cCBiYXNpcyBhbmQgbm90IGEgcGVyIFFQDQo+IGJh > c2lzPw0KDQpJIGFncmVlIHdpdGggRG91ZyBoZXJlLiAgVGhpcyBtYWtlcyBtb3JlIHNlbnNlIHRv > IHNwZWNpZnkgb24gbXVsdGljYXN0IGF0dGFjaCB0aW1lLCBub3QgUVAgY3JlYXRpb24gdGltZS4g > DQo= > -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
I have no earthly idea what this is or where it came from. It doesn't show up in my sent items folder as this. > PiBJJ20gY3VyaW91cyB3aHkgdGhpcyBpcyBkb25lIGF0IFFQIGNyZWF0aW9uIHRpbWUuICBTaG9 > 1 > > > bGQgdGhpcyBub3QNCj4gYmUgZG9uZSBhdCBtdWx0aWNhc3Qgam9pbiB0aW1lPyAgQSBzaW5nbGU > g > > > UVAgY2FuIGJlIGEgbWVtYmVyIG9mIG1vcmUNCj4gdGhhbiBvbmUgTUMgZ3JvdXAsIHdvdWxkIGl > 0 > > > IG5vdCBzZWVtIG1vcmUgY29ycmVjdCB0byB5b3UgdGhhdCB0aGUNCj4gbG9vcGJhY2sgc2hvdWx > k > > > IGJlIHNwZWNpZmllZCBvbiBhIHBlciBncm91cCBiYXNpcyBhbmQgbm90IGEgcGVyIFFQDQo+IGJ > h > > > c2lzPw0KDQpJIGFncmVlIHdpdGggRG91ZyBoZXJlLiAgVGhpcyBtYWtlcyBtb3JlIHNlbnNlIHR > v > > > IHNwZWNpZnkgb24gbXVsdGljYXN0IGF0dGFjaCB0aW1lLCBub3QgUVAgY3JlYXRpb24gdGltZS4 > g > > DQo= > > What I said was: I agree with Doug here. This makes more sense to specify on multicast attach time, not QP creation time. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On Mon, 5 May 2014, Hefty, Sean wrote: > I agree with Doug here. This makes more sense to specify on multicast atta= > ch time, not QP creation time. I read somewhere that some android apps have a bug an do not properly set the fields in the header when sendiung a base 64 encoded message. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On Mon, 5 May 2014, Hefty, Sean wrote: > I agree with Doug here. This makes more sense to specify on multicast atta= > ch time, not QP creation time. On topic response: Do devices support multicast loopback blocking on a multicast group? From what I can tell the multicast support in NICs is based on having a table with MAC addresses of the multicast groups that the NIC is able to receive. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
- Original Message - > On Mon, 5 May 2014, Hefty, Sean wrote: > > > I agree with Doug here. This makes more sense to specify on > > multicast atta= > > ch time, not QP creation time. > > On topic response: Do devices support multicast loopback blocking on > a > multicast group? I would think this is very doubtful...see below... > From what I can tell the multicast support in NICs > is > based on having a table with MAC addresses of the multicast groups > that > the NIC is able to receive. That table only tells the NIC to listen to specific multicast addresses on the wire. This is roughly equivalent to telling the SM to subscribe the port to the multicast groups it needs subscribed too. In both cases, this merely gets the packets to the card. From there, the card (or the OS as is usually the case on NICs and Ethernet multicast) must redistribute the packet to all queue pairs that have subscribed to the group that the packet was received from. If you were to block it at the group level, then it universally effect all applications that subscribed to that group, and there might well be a number of applications that did not request this behavior and would be rightfully confused at the card/OS not sending them their multicast packets. So, I would suggest that the blocking of loopback multicast packets needs to be "opt in" for all applications. The big hammer of blocking all loopback on an entire card or an entire group, while possible, should be highly discouraged. It might work with limited applications that know it is being done, but it can also lead to hard to diagnose problems if you add a new application into the mix and it is unaware of this hammer being used and unable to handle the situation. -- Doug Ledford GPG KeyID: 0E572FDD http://people.redhat.com/dledford -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
On Tue, 6 May 2014, Doug Ledford wrote: > That table only tells the NIC to listen to specific multicast > addresses on the wire. This is roughly equivalent to telling > the SM to subscribe the port to the multicast groups it needs > subscribed too. In both cases, this merely gets the packets > to the card. From there, the card (or the OS as is usually the > case on NICs and Ethernet multicast) must redistribute the > packet to all queue pairs that have subscribed to the group > that the packet was received from. If you were to block it at > the group level, then it universally effect all applications > that subscribed to that group, and there might well be a number > of applications that did not request this behavior and would > be rightfully confused at the card/OS not sending them their > multicast packets. So, I would suggest that the blocking of > loopback multicast packets needs to be "opt in" for all > applications. The big hammer of blocking all loopback on an > entire card or an entire group, while possible, should be > highly discouraged. It might work with limited applications > that know it is being done, but it can also lead to hard to > diagnose problems if you add a new application into the mix > and it is unaware of this hammer being used and unable to > handle the situation. Right the multicast blocking occurs at the socket level for regular networking (see IP_MULTICAST_LOOP socket options). Socket are owned by the application. A QP is roughly the equivalent thing at the RDMA level. So it seems to me that blocking needs to occur at the QP level and not at the multicast group level as suggested by Sean. The 2008 patch does exactly that and allow setting the multicast loopback blocking from user space. Moreover the support is already there for in kernel users. Could we just get that 2008 patch updated and merged? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
> Right the multicast blocking occurs at the socket level for regular > networking (see IP_MULTICAST_LOOP socket options). Socket are owned > by the application. > > > A QP is roughly the equivalent thing at the RDMA level. So it seems to me > that blocking needs to occur at the QP level and not at the multicast > group level as suggested by Sean. I was unaware of the IP_MULTICAST_LOOP option. Given that it makes sense to apply this at the QP level. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] IB/mlx4: Allow to always block UD multicast loopback
- Original Message - > On Tue, 6 May 2014, Doug Ledford wrote: > > > That table only tells the NIC to listen to specific multicast > > addresses on the wire. This is roughly equivalent to telling > > the SM to subscribe the port to the multicast groups it needs > > subscribed too. In both cases, this merely gets the packets > > to the card. From there, the card (or the OS as is usually the > > case on NICs and Ethernet multicast) must redistribute the > > packet to all queue pairs that have subscribed to the group > > that the packet was received from. If you were to block it at > > the group level, then it universally effect all applications > > that subscribed to that group, and there might well be a number > > of applications that did not request this behavior and would > > be rightfully confused at the card/OS not sending them their > > multicast packets. So, I would suggest that the blocking of > > loopback multicast packets needs to be "opt in" for all > > applications. The big hammer of blocking all loopback on an > > entire card or an entire group, while possible, should be > > highly discouraged. It might work with limited applications > > that know it is being done, but it can also lead to hard to > > diagnose problems if you add a new application into the mix > > and it is unaware of this hammer being used and unable to > > handle the situation. > > Right the multicast blocking occurs at the socket level for regular > networking (see IP_MULTICAST_LOOP socket options). Socket are owned > by the application. > > > A QP is roughly the equivalent thing at the RDMA level. So it seems > to me > that blocking needs to occur at the QP level and not at the multicast > group level as suggested by Sean. Nobody that I know of suggested that this should occur at the multicast group level. I suggested, and Sean agreed with, the idea that this should happen at multicast join time. That means it would be on a per queue pair, per multicast join basis. Setting the IP_MULTICAST_LOOP on the socket effects all joins on that socket, so this would be a slightly different API. Right now, the IP stack is somewhat limited in that you would have to create two different sockets and set IP_MULTICAST_LOOP differently on the two sockets in order to have some joins reflect your data back and some not. You could sort of create what I'm talking about by using the source address block, but that would block all applications from your IP address, not just your own sent data, and so wouldn't really work. My original preference would have been to allow one queue pair to have joins in either blocked or unblocked state, that way you would only need one queue pair for both your reflected and non-reflected joins. But it would be easier to have a program capable of both sockets and RDMA connections if we make the queue pairs follow the sockets semantics here, so I'll withdraw my concerns over the current patch (but also wouldn't object to it being done on a per queue pair basis either, since we don't exactly follow socket semantics either way given that the socket semantics separate the join and the ioctl to control this behavior, so separating qp creation and the setting of this behavior in the RDMA API seems reasonable too). > The 2008 patch does exactly that and allow setting the multicast > loopback > blocking from user space. Moreover the support is already there for > in > kernel users. Could we just get that 2008 patch updated and merged? > > -- Doug Ledford GPG KeyID: 0E572FDD http://people.redhat.com/dledford -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html