Re: [PATCH 3/3] ibacm: check for special handling of loopback requests

2010-11-17 Thread Or Gerlitz
Ralph Campbell wrote: I guess what I'm objecting to is hard coding mlx4. I was trying to think of a way that would allow other HCAs to support the block loopback option in the future. It looks like ipoib sets IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK for kernel QPs but this isn't defined in

Re: [PATCH 3/3] ibacm: check for special handling of loopback requests

2010-11-17 Thread Or Gerlitz
Hefty, Sean wrote: One could argue that this change is reasonable regardless of the OFED kernel patch. It avoids sending multicast traffic when the destination is local. The main drawback beyond the extra code is that a node can't send a multicast message to itself, with the intent that

Re: [PATCH] mlx4: properly mask MGM entry members count

2010-11-17 Thread Eli Cohen
On Tue, Nov 16, 2010 at 02:43:46PM -0800, Roland Dreier wrote: Where is the change in handling in the members_count field? In the firmware? The firmware now distinguishes between differnet multicast addresses according to a protocol field located in the high MS byte which previously was

rdma_lat whos

2010-11-17 Thread Or Gerlitz
Hi Ido, We came into a situation where running rdma_lat with vs with out the -c flag, which means w. or w.o using the rdma-cm introduces a notable ~1us difference in latency for 1k messages, that is ~3us w.o using rdma-cm and 3.9us when using the rdma-cm. I have reproduced that now with the

Re: rdma_lat whos

2010-11-17 Thread Or Gerlitz
Or Gerlitz wrote: Now, Jack, using this patch, I could get rdma_lat which doesn't use the rdma-cm, which means setting all the low-level QP params by the hand to produce the SAME result of 3.9us as with the rdma-cm, as you can see its one liner patch which uses higher MTU of 2048 vs the

[PATCH] mlx4: Fix vlan insertion order

2010-11-17 Thread Eli Cohen
We must update the control segment before marking it as valid. Signed-off-by: Eli Cohen e...@mellanox.co.il --- drivers/infiniband/hw/mlx4/qp.c |6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index

RE: [PATCH 3/3] ibacm: check for special handling of loopback requests

2010-11-17 Thread Hefty, Sean
To be precise, the bit avoids recieving multicast packets by the QP that --sent-- it, not by other QPs subscribed to that group on the same node/hca, the patch change-log even states that Inter QP multicast packets on the relevant HCA will still be delivered. You can test that with two mckey

RE: [PATCH 3/3] ibacm: check for special handling of loopback requests

2010-11-17 Thread Hefty, Sean
I guess what I'm objecting to is hard coding mlx4. I was trying to think of a way that would allow other HCAs to support the block loopback option in the future. It looks like ipoib sets IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK for kernel QPs but this isn't defined in libibverbs yet. If there's

Re: [PATCH] mlx4: properly mask MGM entry members count

2010-11-17 Thread Roland Dreier
Where is the change in handling in the members_count field? In the firmware? The firmware now distinguishes between differnet multicast addresses according to a protocol field located in the high MS byte which previously was reserved. The patches that Alekseys will will probably

[PATCH 3/3 v2] ibacm: Introduce loopback resolution 'protocol'

2010-11-17 Thread Hefty, Sean
OFED 1.5.2 introduced a kernel patch that disables looping back multicast messages to the sending QP. This capability was enabled by default. The current ACM address resolution protocol depends on this feature of multicast. To handle the case where multicast loopback has been disabled, add an

Re: [PATCH 0/3] IB Netlink Interface and RDMA CM exports

2010-11-17 Thread Roland Dreier
I was under the impression that a different module is appropriate here based on the structure of the IB modules and similar modules like netfilter_netlink, and other architectures in the kernel. However, I'm naturally open to other views. If the other way around is the consensus

Re: mxl4 and rpcrdma: connection to 192.168.0.100:20049 on mlx4_0, memreg 5 slots 32 ird 16

2010-11-17 Thread Stephen Cousins
On Wed, Nov 17, 2010 at 12:43 PM, Steve Wise sw...@opengridcomputing.com wrote: I think NFSRDMA server will close the connection after 5 minutes of inactivity... Thanks Steve. I mistook the messages as being a problem that may have been related to nfs speed issues. I think it is something else

Re: mxl4 and rpcrdma: connection to 192.168.0.100:20049 on mlx4_0, memreg 5 slots 32 ird 16

2010-11-17 Thread Roland Dreier
Nov 15 09:39:00 node4 kernel: rpcrdma: connection to 192.168.0.100:20049 on mlx4_0, memreg 5 slots 32 ird 16 and then 5 minutes later: Nov 15 09:44:00 node4 kernel: rpcrdma: connection to 192.168.0.100:20049 closed (-103) I think NFSRDMA server will close the

building RDMA perftests with g++

2010-11-17 Thread ib
I have what is probably a silly question If I compile the rdma_bw example from perftests with g++, it doesn't work... granted I have to make a few changes wrt structure initialization, but I would think it should behave as when built with gcc... I am getting an error message in

Re: [PATCH 1/2] svcrdma: Change DMA mapping logic to avoid the page_address kernel API

2010-11-17 Thread Tom Tucker
On 11/16/10 1:39 PM, Or Gerlitz wrote: Tom Tuckert...@ogc.us wrote: This patch changes the bus mapping logic to avoid page_address() where necessary Hi Tom, Does when necessary comes to say that invocations of page_address which remained in the code after this patch was applied are safe

Re: [PATCH] mlx4: properly mask MGM entry members count

2010-11-17 Thread Eli Cohen
On Wed, Nov 17, 2010 at 09:38:47AM -0800, Roland Dreier wrote: Where is the change in handling in the members_count field? In the firmware? The firmware now distinguishes between differnet multicast addresses according to a protocol field located in the high MS byte which

Re: [PATCH 0/3] IB Netlink Interface and RDMA CM exports

2010-11-17 Thread Jason Gunthorpe
On Wed, Nov 17, 2010 at 01:34:29PM -0800, Roland Dreier wrote: In fact I'm not sure even for embedded it's worth making this something that can be configured out. We probably have too many config options that no one ever uses as it is. Are there any embedded systems that are both so small