RE: Fix IPoIB to conform to ethtool definitions

2010-11-08 Thread Eli Cohen
Will send those too. The idea is to let kernel consumers enjoy the improvements to latency that blue flame gives. And yes, SDP is motivating us but I am going to push to IPoIB too. I want to take the opportunity that you raised the issue to hear others opinion about changing the bitmap

Re: IBoE fixes/enhancements

2010-11-08 Thread Or Gerlitz
Eli Cohen wrote: I was going to send [...] upstream Also you had a fix to the port speed and something related to SL which I didn't understand, please send for review Or. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to

Re: kernel inline sending / UARs / etc

2010-11-08 Thread Or Gerlitz
Eli Cohen wrote: The idea is to let kernel consumers enjoy the improvements to latency that blue flame gives. And yes, SDP is motivating us but I am going to push to IPoIB too. From my recollection of numbers, for user space apps, using inline accounts for about 1us improvement in the

Re: kernel inline sending / UARs / etc

2010-11-08 Thread Eli Cohen
On Mon, Nov 08, 2010 at 10:27:15AM +0200, Or Gerlitz wrote: Is this correct that today all the kernels QPs use the same UAR, any problem with that? Yes, today all kernel consumers use the same UAR. And if you want to use blue flame then you need to use different UARs. -- To unsubscribe from

Re: kernel inline sending / UARs / etc

2010-11-08 Thread Or Gerlitz
Or Gerlitz wrote: Eli Cohen wrote: From my recollection of numbers, for user space apps, using inline accounts for about 1us improvement in the latency, if this is indeed the case, I'm sure if there's great value here for kernel consumers, do you have any numbers to support this patch? I

Re: IBoE fixes/enhancements

2010-11-08 Thread Eli Cohen
On Mon, Nov 08, 2010 at 10:24:26AM +0200, Or Gerlitz wrote: Eli Cohen wrote: I was going to send [...] upstream Also you had a fix to the port speed and something related to SL which I didn't understand, please send for review There have been a few minor fixes - I'll review and send. -- To

Re: kernel inline sending / UARs / etc

2010-11-08 Thread Eli Cohen
On Mon, Nov 08, 2010 at 10:45:42AM +0200, Or Gerlitz wrote: Or Gerlitz wrote: Eli Cohen wrote: From my recollection of numbers, for user space apps, using inline accounts for about 1us improvement in the latency, if this is indeed the case, I'm sure if there's great value here for

Re: kernel inline sending / UARs / etc

2010-11-08 Thread Or Gerlitz
Eli Cohen wrote: It indeed improves SDP's latency - I don't have exact numbers. the SDP number is very interesting (Amir, do you have it?) but irrelevant for upstream, any IPoIB numbers? Or. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to

Re: kernel inline sending / UARs / etc

2010-11-08 Thread Amir Vadai
I am working on it, will notify when have numbers. On 11/08/2010 10:51 AM, Or Gerlitz wrote: Eli Cohen wrote: It indeed improves SDP's latency - I don't have exact numbers. the SDP number is very interesting (Amir, do you have it?) but irrelevant for upstream, any IPoIB numbers? Or. -- To

Re: [PATCH 0/13] IB-mgmt: Port madeye to userspace

2010-11-08 Thread Or Gerlitz
Hefty, Sean wrote: CM mads aren't reliable, however they are retried. If a CM REQ does not receive a response after so many retries (usually 15), the REQ fails (status is timeout). The mad layer reports the timeout to the cm module. With snooping in place, a user will be notified that a

[PATCH] IPoIB: Fix IPoIB to conform to ethtool definitions

2010-11-08 Thread Eli Cohen
Ethtool documentation states that when one of the parameters, rx_coalesce_usecs or rx_max_coalesced_frames is set to zero while the other has a none zero value, the none zero parameter should still be operative. For example, if rx_max_coalesced_frames is set to zero while rx_coalesce_usecs is 0,

Re: kernel inline sending / UARs / etc

2010-11-08 Thread Eli Cohen
On Mon, Nov 08, 2010 at 10:51:30AM +0200, Or Gerlitz wrote: Eli Cohen wrote: It indeed improves SDP's latency - I don't have exact numbers. the SDP number is very interesting (Amir, do you have it?) but irrelevant for upstream, any IPoIB numbers? For IPoIB it gives ~1 usec for

Re: kernel inline sending / UARs / etc

2010-11-08 Thread Or Gerlitz
Eli Cohen wrote: For IPoIB it gives ~1 usec for improvement in latency. yep, this is what I expected, so over your testbed from what value to what value? also it would be important to note the change in the cpu utilization (e.g few vmstat 1 output lines before/after the change, while running

RE: [PATCH 0/13] IB-mgmt: Port madeye to userspace

2010-11-08 Thread Hefty, Sean
so the usage of mad snooping would be for cache invalidations, I wonder if registering on GID/MGID IN/OUT traps be sufficient for the same purpose? That requires registration with the SA. The intent is to avoid using a centralized service when possible. Otherwise, we end up with all nodes

Re: [PATCH] mlx4_core: Fix modify CQ

2010-11-08 Thread Eli Cohen
Roland, this patch has bad affect on mlx4_en. I sent another patch which does the same at the IPoIB layer. Please consider removing it. - thanks. On Mon, Oct 25, 2010 at 04:38:59PM +0200, Eli Cohen wrote: The modify CQ command disables the effect of both the count and period paramters if any

Re: [PATCH 0/13] IB-mgmt: Port madeye to userspace

2010-11-08 Thread Or Gerlitz
Hefty, Sean wrote: That requires registration with the SA. The intent is to avoid using a centralized service when possible. yep, makes sense, look like this design finally went the decentralized way... cool Or. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the

Re: [PATCH] mlx4_core: Fix modify CQ

2010-11-08 Thread Roland Dreier
this patch has bad affect on mlx4_en. I sent another patch which does the same at the IPoIB layer. Please consider removing it. I didn't merge this yet, did I? - R. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org

TCP CUBIC and IPoIB

2010-11-08 Thread Jason Gunthorpe
I wonder if anyone has ever looked at this? It looks like there is something wrong here. I was just looking at some results from a stock 2.6.35 kernel doing TCP over IPoIB (2k MTU) and CUBIC looks like it is going nutz, you can see in a bandwidth plot a fairly characteristic saw tooth pattern,

Re: [PATCH] mlx4_core: Fix modify CQ

2010-11-08 Thread Eli Cohen
On Tue, Nov 9, 2010 at 1:04 AM, Roland Dreier rdre...@cisco.com wrote: I didn't merge this yet, did I? No, you did not. I was just asking that you won't. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo