Re: [PATCH V4 for-next 1/5] IB/core: Add RSS and TSS QP groups - suggesting BOF during OFA conf to further discuss that

2013-04-30 Thread Shlomo Pongratz
On 4/29/2013 11:36 PM, Jason Gunthorpe wrote: On Mon, Apr 29, 2013 at 10:52:21PM +0300, Or Gerlitz wrote: On Fri, Apr 26, 2013 at 12:40 AM, Jason Gunthorpe wrote: But I don't follow why the send QPNs have to be sequential for IPoIB. It looks like this is being motivated by RSS and RSS QPNs are

Re: NFS over RDMA benchmark

2013-04-30 Thread Tom Talpey
On 4/30/2013 1:09 AM, Yan Burman wrote: -Original Message- From: J. Bruce Fields [mailto:bfie...@fieldses.org] Sent: Sunday, April 28, 2013 17:43 To: Yan Burman Cc: Wendy Cheng; Atchley, Scott; Tom Tucker; linux-rdma@vger.kernel.org; linux-...@vger.kernel.org; Or Gerlitz Subject: Re:

Re: NFS over RDMA benchmark

2013-04-30 Thread J. Bruce Fields
On Sun, Apr 28, 2013 at 10:42:48AM -0400, J. Bruce Fields wrote: On Sun, Apr 28, 2013 at 06:28:16AM +, Yan Burman wrote: On Wed, Apr 17, 2013 at 7:36 AM, Yan Burman y...@mellanox.com I've been trying to do some benchmarks for NFS over RDMA and I seem to

Re: NFS over RDMA benchmark

2013-04-30 Thread Tom Talpey
On 4/30/2013 1:09 AM, Yan Burman wrote: I now get up to ~95K IOPS and 4.1GB/sec bandwidth. ... ib_send_bw with intel iommu enabled did get up to 4.5GB/sec BTW, you may want to verify that these are the same GB. Many benchmarks say KB/MB/GB when they really mean KiB/MiB/GiB. At GB/GiB, the

RE: NFS over RDMA benchmark

2013-04-30 Thread Yan Burman
-Original Message- From: Tom Talpey [mailto:t...@talpey.com] Sent: Tuesday, April 30, 2013 16:05 To: Yan Burman Cc: J. Bruce Fields; Wendy Cheng; Atchley, Scott; Tom Tucker; linux- r...@vger.kernel.org; linux-...@vger.kernel.org; Or Gerlitz Subject: Re: NFS over RDMA benchmark

Re: [PATCH 1/2] libibverbs: Use autoreconf in autogen.sh

2013-04-30 Thread Jeff Squyres (jsquyres)
Bump bump. :-) On Apr 25, 2013, at 11:38 AM, Jeff Squyres (jsquyres) jsquy...@cisco.com wrote: Bump. On Apr 22, 2013, at 1:41 PM, Jeff Squyres jsquy...@cisco.com wrote: The old sequence of Autotools commands listed in autogen.sh is no longer correct. Instead, just use the single

RE: NFS over RDMA benchmark

2013-04-30 Thread Yan Burman
-Original Message- From: Tom Talpey [mailto:t...@talpey.com] Sent: Tuesday, April 30, 2013 17:20 To: Yan Burman Cc: J. Bruce Fields; Wendy Cheng; Atchley, Scott; Tom Tucker; linux- r...@vger.kernel.org; linux-...@vger.kernel.org; Or Gerlitz Subject: Re: NFS over RDMA benchmark

Re: NFS over RDMA benchmark

2013-04-30 Thread Tom Talpey
On 4/30/2013 10:23 AM, Yan Burman wrote: -Original Message- From: Tom Talpey [mailto:t...@talpey.com] On Sun, Apr 28, 2013 at 06:28:16AM +, Yan Burman wrote: I finally got up to 4.1GB/sec bandwidth with RDMA (ipoib-CM bandwidth is also way higher now). For some reason when I had

Re: NFS over RDMA benchmark

2013-04-30 Thread Wendy Cheng
On Mon, Apr 29, 2013 at 10:09 PM, Yan Burman y...@mellanox.com wrote: I finally got up to 4.1GB/sec bandwidth with RDMA (ipoib-CM bandwidth is also way higher now). For some reason when I had intel IOMMU enabled, the performance dropped significantly. I now get up to ~95K IOPS and

Re: [PATCH V4 for-next 1/5] IB/core: Add RSS and TSS QP groups - suggesting BOF during OFA conf to further discuss that

2013-04-30 Thread Jason Gunthorpe
On Tue, Apr 30, 2013 at 12:04:25PM +0300, Shlomo Pongratz wrote: And.. 'tss_qpn_mask_sz' seems unnecessarily limiting, using WC.srcQPN + ipoib_header.tss_qpn_offset == real QPN (ie use a signed offset, not a mask) Seems much better than Wc.srcQPN ~((1(ipoib_header.tss_qpn_mask_sz

Re: NFS over RDMA benchmark

2013-04-30 Thread Tom Tucker
On 4/30/13 9:38 AM, Yan Burman wrote: -Original Message- From: Tom Talpey [mailto:t...@talpey.com] Sent: Tuesday, April 30, 2013 17:20 To: Yan Burman Cc: J. Bruce Fields; Wendy Cheng; Atchley, Scott; Tom Tucker; linux- r...@vger.kernel.org; linux-...@vger.kernel.org; Or Gerlitz

Re: [PATCH V4 for-next 1/5] IB/core: Add RSS and TSS QP groups - suggesting BOF during OFA conf to further discuss that

2013-04-30 Thread Or Gerlitz
Jason Gunthorpe jguntho...@obsidianresearch.com wrote: For the TSS case, I'd say just allocate normal QPs and provide something like ibv_override_ud_src_qpn(). This is very general and broadly useful for any application using UD QPs. I've lost you, how you suggest to implement

Re: [PATCH V4 for-next 1/5] IB/core: Add RSS and TSS QP groups - suggesting BOF during OFA conf to further discuss that

2013-04-30 Thread Jason Gunthorpe
On Tue, Apr 30, 2013 at 11:08:19PM +0300, Or Gerlitz wrote: Jason Gunthorpe jguntho...@obsidianresearch.com wrote: For the TSS case, I'd say just allocate normal QPs and provide something like ibv_override_ud_src_qpn(). This is very general and broadly useful for any application

Re: [PATCH for-next 6/9] IB/core: Add receive Flow Steering support

2013-04-30 Thread Or Gerlitz
On Mon, Apr 29, 2013 at 10:40 PM, Christoph Lameter c...@linux.com wrote: On Mon, 29 Apr 2013, Steve Wise wrote: Hey Or, This looks good at first glance. I must confess I cannot tell yet if this will provide everything we need for chelsio's RAW packet requirements. But I think we should