Hi Bart,
On Thu, Jul 23, 2015 at 06:47:26AM -0700, Bart Van Assche wrote:
> This statement might need some clarification. Are you aware that this memory
> region is only used if the kernel module parameter register_always is zero ?
The way I read the driver it's also used if the driver doesn't su
> > I am unclear what happens sever side if the server starts issuing
> > SEND_WITH_INVALIDATE to a client that doesn't expect it. The net
> > result is a MR would be invalidated twice. I don't know if this is OK
> > or not.
>
> It is ok to invalidate an already-invalid MR.
Nice, ah but I forgot
On Fri, Jul 24, 2015 at 05:11:04PM -0500, Steve Wise wrote:
> By the way, just to be clear: If you use a FRWR, you by definition
> only have one SGE entry as the result of the registration. So
> regardless of what a device/protocol can do with the destination SGE
> of an RDMA READ operation, if y
> -Original Message-
> From: linux-rdma-ow...@vger.kernel.org
> [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Jason Gunthorpe
> Sent: Friday, July 24, 2015 3:25 PM
> To: Chuck Lever
> Cc: Sagi Grimberg; Christoph Hellwig; linux-rdma; Liran Liss; Oren Duer
> Subject: Re: [PATCH W
> -Original Message-
> From: linux-rdma-ow...@vger.kernel.org
> [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Steve Wise
> Sent: Friday, July 24, 2015 2:58 PM
> To: 'Jason Gunthorpe'
> Cc: dledf...@redhat.com; infinip...@intel.com; sa...@mellanox.com;
> ogerl...@mellanox.com; r
On Fri, Jul 24, 2015 at 04:26:00PM -0400, Chuck Lever wrote:
> Basically RPC work flow stopped because an RPC reply never
> arrived.
Oh, that is what I expect to see.. Remebmer the cq upcall is edge
triggered, so if you leave stuff in the cq then you don't get another
upcall until another CQE is a
During some other testing I found that when a completion upcall
returns to the provider leaving CQEs still on the completion queue,
there is a non-zero probability that a completion will be lost.
>>>
>>> What does lost mean?
>>
>> Lost means a WC in the CQ is skipped by ib_poll_cq
On Fri, Jul 24, 2015 at 03:59:06PM -0400, Chuck Lever wrote:
> And RPC-over-RDMA version 1 does not have any way to signal that
> the server has invalidated the MRs. Such signaling would be a
> pre-requisite to allow the Linux NFS/RDMA client to interoperate
> with non-Linux NFS/RDMA servers that d
On Jul 24, 2015, at 3:10 PM, Jason Gunthorpe
wrote:
> On Fri, Jul 24, 2015 at 01:46:05PM -0400, Chuck Lever wrote:
>>> I'm not surprised since invalidate is sync. I belive you need to
>>> incorporate SEND WITH INVALIDATE to substantially recover this
>>> overhead.
>>
>> I tried to find another
> -Original Message-
> From: Jason Gunthorpe [mailto:jguntho...@obsidianresearch.com]
> Sent: Friday, July 24, 2015 2:24 PM
> To: Steve Wise
> Cc: dledf...@redhat.com; infinip...@intel.com; sa...@mellanox.com;
> ogerl...@mellanox.com; r...@mellanox.com; linux-
> r...@vger.kernel.org; e..
On Fri, Jul 24, 2015 at 01:48:09PM -0500, Steve Wise wrote:
> > The use of FRWR for RDMA READ should be iWarp specific, IB shouldn't
> > pay that overhead. I am expecting to see a cap_rdma_read_rkey or
> > something in here ?
>
> Ok. But cap_rdma_read_rkey() doesn't really describe the
> requirem
On Fri, Jul 24, 2015 at 01:40:17PM -0500, Steve Wise wrote:
> > Huh. How does this relate to the max_page_list_len argument:
> >
> > struct ib_mr *ib_alloc_fast_reg_mr(struct ib_pd *pd, int max_page_list_len)
> >
> > Shouldn't max_fast_reg_page_list_len be checked during the above?
> >
> > Ie d
On Fri, Jul 24, 2015 at 01:46:05PM -0400, Chuck Lever wrote:
> > I'm not surprised since invalidate is sync. I belive you need to
> > incorporate SEND WITH INVALIDATE to substantially recover this
> > overhead.
>
> I tried to find another kernel ULP using SEND WITH INVALIDATE, but
> I didn’t see o
> -Original Message-
> From: linux-rdma-ow...@vger.kernel.org
> [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Steve Wise
> Sent: Friday, July 24, 2015 11:19 AM
> To: dledf...@redhat.com
> Cc: infinip...@intel.com; sa...@mellanox.com; ogerl...@mellanox.com;
> r...@mellanox.com;
> -Original Message-
> From: Jason Gunthorpe [mailto:jguntho...@obsidianresearch.com]
> Sent: Friday, July 24, 2015 11:57 AM
> To: Steve Wise
> Cc: dledf...@redhat.com; infinip...@intel.com; sa...@mellanox.com;
> ogerl...@mellanox.com; r...@mellanox.com; linux-
> r...@vger.kernel.org; e.
> -Original Message-
> From: Jason Gunthorpe [mailto:jguntho...@obsidianresearch.com]
> Sent: Friday, July 24, 2015 11:49 AM
> To: Steve Wise
> Cc: dledf...@redhat.com; infinip...@intel.com; sa...@mellanox.com;
> ogerl...@mellanox.com; r...@mellanox.com; linux-
> r...@vger.kernel.org; e.
> -Original Message-
> From: Jason Gunthorpe [mailto:jguntho...@obsidianresearch.com]
> Sent: Friday, July 24, 2015 11:41 AM
> To: Steve Wise
> Cc: dledf...@redhat.com; infinip...@intel.com; sa...@mellanox.com;
> ogerl...@mellanox.com; r...@mellanox.com; linux-
> r...@vger.kernel.org; e.
On Jul 24, 2015, at 12:26 PM, Jason Gunthorpe
wrote:
> On Fri, Jul 24, 2015 at 10:36:07AM -0400, Chuck Lever wrote:
>
>> Unfinished, but operational:
>>
>> http://git.linux-nfs.org/?p=cel/cel-2.6.git;a=shortlog;h=refs/heads/nfs-rdma-future
>
> Nice..
>
> Can you spend some time and reflect
On Fri, Jul 24, 2015 at 11:19:05AM -0500, Steve Wise wrote:
> Add new register and unregister functions to be used with devices that
> support FRMRs, provide a local dma lkey, yet do not support DIF/PI.
>
> isert_reg_frmr() only needs to use FRMRs for RDMA READ since RDMA WRITE
> can be handled en
On Fri, Jul 24, 2015 at 11:18:59AM -0500, Steve Wise wrote:
> No need to allocate a dma_mr if the device provides a local_dma_lkey.
It is probably safe to put your series on top of mine, which
incorporates this patch already.
https://github.com/jgunthorpe/linux/commits/remove-ib_get_dma_mr
Jason
On Fri, Jul 24, 2015 at 11:18:21AM -0500, Steve Wise wrote:
> Currently the sg tablesize, which dictates fast register page list
> depth to use, does not take into account the limits of the rdma device.
> So adjust it once we discover the device fastreg max depth limit. Also
> adjust the max_secto
> -Original Message-
> From: linux-rdma-ow...@vger.kernel.org
> [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Jason Gunthorpe
> Sent: Friday, July 24, 2015 11:27 AM
> To: Chuck Lever
> Cc: Sagi Grimberg; Christoph Hellwig; linux-rdma; Liran Liss; Oren Duer
> Subject: Re: [PATCH
On Fri, Jul 24, 2015 at 10:36:07AM -0400, Chuck Lever wrote:
> Unfinished, but operational:
>
> http://git.linux-nfs.org/?p=cel/cel-2.6.git;a=shortlog;h=refs/heads/nfs-rdma-future
Nice..
Can you spend some time and reflect on how some of this could be
lowered into the core code? The FMR and FRW
Add new register and unregister functions to be used with devices that
support FRMRs, provide a local dma lkey, yet do not support DIF/PI.
isert_reg_frmr() only needs to use FRMRs for RDMA READ since RDMA WRITE
can be handled entirely with the local dma lkey. So for RDMA READ,
it calls isert_reg_
Use the device's max_fast_reg_page_list_len attr to size the SQ
and FRMR pool.
Signed-off-by: Steve Wise
---
drivers/infiniband/ulp/isert/ib_isert.c | 28
drivers/infiniband/ulp/isert/ib_isert.h |1 +
2 files changed, 25 insertions(+), 4 deletions(-)
diff --g
No need to allocate a dma_mr if the device provides a local_dma_lkey.
Signed-off-by: Steve Wise
---
drivers/infiniband/ulp/isert/ib_isert.c | 47 ++-
drivers/infiniband/ulp/isert/ib_isert.h |1 +
2 files changed, 28 insertions(+), 20 deletions(-)
diff --git a/
Applications must not assume that max_sge and max_sge_rd are the same,
Hence expose max_sge_rd correctly as well.
Signed-off-by: Steve Wise
Acked-by: Mike Marciniszyn
---
drivers/infiniband/hw/ipath/ipath_verbs.c |1 +
drivers/infiniband/hw/qib/qib_verbs.c |1 +
2 files changed, 2
Use the device's max_sge_rd capability to compute the target's read sge
depth. Save both the read and write max_sge values in the isert_conn
struct, and use these when creating RDMA_WRITE/READ work requests.
Signed-off-by: Steve Wise
---
drivers/infiniband/ulp/isert/ib_isert.c | 24 +
Signed-off-by: Steve Wise
---
include/linux/sunrpc/svc_rdma.h |1 +
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 12 +---
net/sunrpc/xprtrdma/svc_rdma_transport.c |4
3 files changed, 6 insertions(+), 11 deletions(-)
diff --git a/include/linux/sunrpc/svc_rdma.h b/i
This is in preparation for adding new FRMR-only IO handlers
for devices that support FRMR and not PI.
Signed-off-by: Steve Wise
---
drivers/infiniband/ulp/isert/ib_isert.c | 28 ++--
1 files changed, 14 insertions(+), 14 deletions(-)
diff --git a/drivers/infiniband/ul
Currently the sg tablesize, which dictates fast register page list
depth to use, does not take into account the limits of the rdma device.
So adjust it once we discover the device fastreg max depth limit. Also
adjust the max_sectors based on the resulting sg tablesize.
Signed-off-by: Steve Wise
From: Sagi Grimberg
Applications must not assume that max_sge and max_sge_rd are the same,
Hence expose max_sge_rd correctly as well.
Reported-by: Steve Wise
Signed-off-by: Sagi Grimberg
---
drivers/infiniband/hw/mlx4/main.c|1 +
drivers/infiniband/hw/mlx5/main.c|
The following series implements support for iWARP transports in the iSER
initiator and target. This is based on v4.2-rc3.
I know we're in the middle of some API changes that will affect the isert
patches in this series, but I wanted to get these out for another round
of review. I can merge the i
On 07/24/2015 10:50 AM, Steve Wise wrote:
>
>
>> -Original Message-
>> From: linux-rdma-ow...@vger.kernel.org
>> [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Doug Ledford
>> Sent: Friday, July 24, 2015 9:45 AM
>> To: Steve Wise
>> Cc: linux-rdma@vger.kernel.org
>> Subject: Re:
> -Original Message-
> From: linux-rdma-ow...@vger.kernel.org
> [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Doug Ledford
> Sent: Friday, July 24, 2015 9:45 AM
> To: Steve Wise
> Cc: linux-rdma@vger.kernel.org
> Subject: Re: [PATCH] RDMA/cxgb3: fail get_dma_mr if the memory foo
On 07/23/2015 06:47 PM, Steve Wise wrote:
>> -Original Message- From: Doug Ledford
>> Should this be a static check of the pointer size versus installed
>> memory? Would it be possible to have this work for machines with
>> less than 4GB of physical memory even if they have 64bit pointers
On Jul 23, 2015, at 2:53 PM, Jason Gunthorpe
wrote:
> On Thu, Jul 23, 2015 at 07:59:48PM +0300, Sagi Grimberg wrote:
>> I don't mean to be negative about your ideas, I just don't think that
>> doing all the work in the drivers is going to get us to a better place.
>
> No worries, I'm hoping so
On 07/23/2015 05:48 PM, Jason Gunthorpe wrote:
> On Thu, Jul 23, 2015 at 05:41:38PM -0400, Doug Ledford wrote:
>
>> I assume this prevents the driver from working at all on certain arches
>> (like ppc with 64k page size)?
>
> Nothing uses page_size_cap correctly, so it has no impact.
OK, then I'
2015-07-23 22:46 GMT+03:00 Hefty, Sean :
> Ah, yes, the rdma_cm supports GIDs. It does not support GUIDs. Though it's
> usually trivial to convert a GUID into a GID.
in guid you mean node guid ? So using node guied bring something like
multipath (in case of able to via node guid get port guids
39 matches
Mail list logo
