[PATCH v2 02/11] xprtrdma: xprt_rdma_free() must not release backchannel reqs

Preserve any rpcrdma_req that is attached to rpc_rqst's allocated for the backchannel. Otherwise, after all the pre-allocated backchannel req's are consumed, incoming backward calls start writing on freed memory. Somehow this hunk got lost. Fixes: f531a5dbc451 ('xprtrdma: Pre-allocate backward

[PATCH v2 00/11] NFS/RDMA client patches for 4.5

For 4.5, I'd like to address the send queue accounting and invalidation/unmap ordering issues Jason brought up a couple of months ago. Thanks to all reviewers of v1. Also available in the "nfs-rdma-for-4.5" topic branch of this git repo: git://git.linux-nfs.org/projects/cel/cel-2.6.git Or for

Re: [PATCH v1 10/10] IB/iser: Support the remote invalidation exception

On Tue, Nov 24, 2015 at 6:23 PM, Sagi Grimberg wrote: > > From: Jenny Derzhavetz > > Declare that we support remote invalidation in case we are: > 1. using Fastreg method > 2. always registering memory. decide if you want or don't want to use periods @

[PATCH v2 04/11] xprtrdma: Move struct ib_send_wr off the stack

For FRWR FASTREG and LOCAL_INV, move the ib_*_wr structure off the stack. This allows frwr_op_map and frwr_op_unmap to chain WRs together without limit to register or invalidate a set of MRs with a single ib_post_send(). (This will be for chaining LOCAL_INV requests). Signed-off-by: Chuck Lever

[PATCH v2 05/11] xprtrdma: Introduce ro_unmap_sync method

In the current xprtrdma implementation, some memreg strategies implement ro_unmap synchronously (the MR is knocked down before the method returns) and some asynchonously (the MR will be knocked down and returned to the pool in the background). To guarantee the MR is truly invalid before the RPC

[PATCH v2 03/11] xprtrdma: Disable RPC/RDMA backchannel debugging messages

Clean up. Fixes: 63cae47005af ('xprtrdma: Handle incoming backward direction') Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/backchannel.c | 16 +--- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/net/sunrpc/xprtrdma/backchannel.c

[PATCH v2 07/11] xprtrdma: Add ro_unmap_sync method for FMR

FMR's ro_unmap method is already synchronous because ib_unmap_fmr() is a synchronous verb. However, some improvements can be made here. 1. Gather all the MRs for the RPC request onto a list, and invoke ib_unmap_fmr() once with that list. This reduces the number of doorbells when there is

[PATCH v2 10/11] xprtrdma: Invalidate in the RPC reply handler

There is a window between the time the RPC reply handler wakes the waiting RPC task and when xprt_release() invokes ops->buf_free. During this time, memory regions containing the data payload may still be accessed by a broken or malicious server, but the RPC application has already been allowed

[PATCH v2 11/11] xprtrdma: Revert commit e7104a2a9606 ('xprtrdma: Cap req_cqinit').

The root of the problem was that sends (especially unsignalled FASTREG and LOCAL_INV Work Requests) were not properly flow- controlled, which allowed a send queue overrun. Now that the RPC/RDMA reply handler waits for invalidation to complete, the send queue is properly flow-controlled. Thus this

[PATCH v2 01/11] xprtrdma: Fix additional uses of spin_lock_irqsave(rb_lock)

Clean up. rb_lock critical sections added in rpcrdma_ep_post_extra_recv() should have first been converted to use normal spin_lock now that the reply handler is a work queue. The backchannel set up code should use the appropriate helper instead of open-coding a rb_recv_bufs list add. Problem

[PATCH v2 06/11] xprtrdma: Add ro_unmap_sync method for FRWR

FRWR's ro_unmap is asynchronous. The new ro_unmap_sync posts LOCAL_INV Work Requests and waits for them to complete before returning. Note also, DMA unmapping is now done _after_ invalidation. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/frwr_ops.c | 137

[PATCH v2 09/11] SUNRPC: Introduce xprt_commit_rqst()

I'm about to add code in the RPC/RDMA reply handler between the xprt_lookup_rqst() and xprt_complete_rqst() call site that needs to execute outside of spinlock critical sections. Add a hook to remove an rpc_rqst from the pending list once the transport knows its going to invoke

[PATCH v2 08/11] xprtrdma: Add ro_unmap_sync method for all-physical registration

physical's ro_unmap is synchronous already. The new ro_unmap_sync method just has to DMA unmap all MRs associated with the RPC request. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/physical_ops.c | 13 + 1 file changed, 13 insertions(+) diff --git

RE: [PATCH for-next V1 5/9] IB/core: Add rdma_network_type to wc

> From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma- > > The abstraction at the gid cache is making it too easy to make this mistake. > It > is enabling callers to do direct gid lookups without a route lookup, which is > unconditionally wrong. Every call site into the gid cache I looked

Re: [PATCH 1/2] IB/core: correct issue with sge copyin corrupting wr

On Mon, Nov 30, 2015 at 4:34 PM, Mike Marciniszyn wrote: Being a non native English speaker I am not fully sure, but "copyin corrupting" sounds like slang / street talking... proper English please > Commit e622f2f4ad21 ("IB: split struct ib_send_wr") > introduced a

Re: [PATCH 0/2] Fix 4.4 IB merge window regressions

On Mon, Nov 30, 2015 at 01:05:19PM -0500, ira.weiny wrote: > On Mon, Nov 30, 2015 at 09:34:15AM -0500, Mike Marciniszyn wrote: > > This two patch series fixes regressions for qib and hfi1 > > introduced in the 4.4 merge window. > > > > These are critical for 4.4 and for rebasing the qib/hfi1 > >

Re: [PATCH 0/2] Fix 4.4 IB merge window regressions

On Mon, Nov 30, 2015 at 09:34:15AM -0500, Mike Marciniszyn wrote: > This two patch series fixes regressions for qib and hfi1 > introduced in the 4.4 merge window. > > These are critical for 4.4 and for rebasing the qib/hfi1 > refactoring. Doug, Greg, It should also be noted that without this

[PATCH V2 1/2] IB/core: Fix user mode post post wr corruption

Commit e622f2f4ad21 ("IB: split struct ib_send_wr") introduced a regression for HCAs whose user mode post sends go through ib_uverbs_post_send(). The code didn't account for the fact that the first sge is offset by an operation dependent length. The allocation did, but the pointer to the

[PATCH v2 0/8] NFS/RDMA server patches for 4.5

Here are patches to support server-side bi-directional RPC/RDMA operation (to enable NFSv4.1 on RPC/RDMA transports). Thanks to all who reviewed v1. Also available in the "nfsd-rdma-for-4.5" topic branch of this git repo: git://git.linux-nfs.org/projects/cel/cel-2.6.git Or for browsing:

[PATCH v2 2/7] svcrdma: Add svc_rdma_get_context() API that is allowed to fail

To support backward direction calls, I'm going to add an svc_rdma_get_context() call in the client RDMA transport. Called from ->buf_alloc(), we can't sleep waiting for memory. So add an API that can get a server op_ctxt but won't sleep. Signed-off-by: Chuck Lever ---

[PATCH v2 1/7] svcrdma: Do not send XDR roundup bytes for a write chunk

Minor optimization: when dealing with write chunk XDR roundup, do not post a Write WR for the zero bytes in the pad. Simply update the write segment in the RPC-over-RDMA header to reflect the extra pad bytes. The Reply chunk is also a write chunk, but the server does not use send_write_chunks()

[PATCH v2 4/7] svcrdma: Add infrastructure to send backwards direction RPC/RDMA calls

To support the NFSv4.1 backchannel on RDMA connections, add a mechanism for sending a backwards-direction RPC/RDMA call on a connection established by a client. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h |2 +

[PATCH v2 3/7] svcrdma: Define maximum number of backchannel requests

Extra resources for handling backchannel requests have to be pre-allocated when a transport instance is created. Set a limit. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h |2 ++ net/sunrpc/xprtrdma/svc_rdma_transport.c | 14 +-

[PATCH v2 6/7] xprtrdma: Add class for RDMA backwards direction transport

To support the server-side of an NFSv4.1 backchannel on RDMA connections, add a transport class that enables backward direction messages on an existing forward channel connection. Signed-off-by: Chuck Lever --- include/linux/sunrpc/xprt.h |1

RE: [PATCH 1/2] IB/core: correct issue with sge copyin corrupting wr

> Being a non native English speaker I am not fully sure, but "copyin > corrupting" sounds like slang / street talking... proper English please > > > Commit e622f2f4ad21 ("IB: split struct ib_send_wr") introduced a > > regression for HCAs whose user mode post sends go through > >

[PATCH v2 5/7] svcrdma: Add infrastructure to receive backwards direction RPC/RDMA replies

To support the NFSv4.1 backchannel on RDMA connections, add a capability for receiving an RPC/RDMA reply on a connection established by a client. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/rpc_rdma.c | 72 +++

[PATCH v2 7/7] svcrdma: No need to count WRs in svc_rdma_send()

Minor optimization: Instead of counting WRs in a chain, have callers pass in the number of WRs they've prepared. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h |2 +- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c |9 ++---

Re: [PATCH v1 10/10] IB/iser: Support the remote invalidation exception

On 24/11/2015 18:23, Sagi Grimberg wrote: > From: Jenny Derzhavetz > > Declare that we support remote invalidation in case we are: > 1. using Fastreg method > 2. always registering memory. > > Detect the invalidated rkey from the work completion info so we > won't

[PATCH 1/2] IB/core: correct issue with sge copyin corrupting wr

Commit e622f2f4ad21 ("IB: split struct ib_send_wr") introduced a regression for HCAs whose user mode post sends go through ib_uverbs_post_send(). The code didn't account for the fact that the first sge is offset by an operation dependent length. The allocation did, but the pointer to the sge

[PATCH 2/2] IB/qib: Fix qib_mr structure

From: Ira Weiny struct qib_mr requires the mr member be the last because struct qib_mregion contains a dynamic array at the end. The additions of members should have been placed before this structure as the comment noted. Failure to do so was causing random memory

[PATCH 0/2] Fix 4.4 IB merge window regressions

This two patch series fixes regressions for qib and hfi1 introduced in the 4.4 merge window. These are critical for 4.4 and for rebasing the qib/hfi1 refactoring. --- Ira Weiny (1): IB/qib: Fix qib_mr structure Mike Marciniszyn (1): IB/core: correct issue with sge copyin corrupting