Re: [PATCH 10/11] IB: only keep a single key in struct ib_mr

On Sun, Nov 22, 2015 at 06:46:48PM +0100, Christoph Hellwig wrote: > While IB supports the notion of returning separate local and remote keys > from a memory registration, the iWarp spec doesn't and neither does any > of our in-tree HCA drivers [1] nor consumers. Consolidate the in-kernel > API

Re: [PATCH 11/11] IB: provide better access flags for fast registrations

On Mon, Nov 23, 2015 at 11:58:29AM -0700, Jason Gunthorpe wrote: > > +#define IB_REG_LKEY(ib_reg_scope_t)0x > > +#define IB_REG_RKEY(ib_reg_scope_t)0x0001 > > Wrap in () just for convention? Ok. > Maybe > > unsigned int acc = ib_scope_to_access(scope); > if

Re: [PATCH for-next V1 0/9] Add RoCE v2 support

On 11/22/2015 04:28 PM, Or Gerlitz wrote: > On Mon, Nov 16, 2015, Matan Barak wrote: >> On Thu, Oct 15, 2015 , Matan Barak wrote: > >>> Hi Doug, >>> This series adds the support for RoCE v2. In order to support RoCE v2, >>> we add gid_type

Re: memory registration updates

On Mon, Nov 23, 2015 at 05:17:30PM +0200, Sagi Grimberg wrote: > >> I send 1-9 out separately earlier :) The other two sit on top of them >> and they are prep patches in a sense as they remove a lot of users >> of struct ib_mr that i don't have to modify in patches 10 and 11. > > Still, patches

Re: [PATCH 10/11] IB: only keep a single key in struct ib_mr

On Mon, Nov 23, 2015 at 12:41:24PM -0700, Jason Gunthorpe wrote: > I like this too, but, I'm a little worried this makes the API more > confusing - ideally, we'd get rid of all the IB_ACCESS stuff from > within the kernel completely. That's my plan - at least for MRs. The only place still using

Re: [PATCH for-next 03/10] IB/iser: Don't register memory for all immediatedata writes

On Tue, Nov 17, 2015 at 11:41:39AM +0200, Sagi Grimberg wrote: > > >On 11/16/2015 6:37 PM, Sagi Grimberg wrote: > >>+++ b/drivers/infiniband/ulp/iser/iser_memory.c > >>@@ -250,7 +250,7 @@ iser_reg_dma(struct iser_device *device, struct > >>iser_data_buf *mem, > >> struct scatterlist *sg =

Re: [PATCH] IB/ipoib: CSUM support in connected mode

On Wed, Nov 18, 2015 at 10:27:41PM +0200, Yuval Shaia wrote: > > You need private-data exchange to negotiate the feature. > > > > The feature should be a per-packet csum status header. > > > > When sending a skb that is already fully csumed the receiver sets > > CHECKSUM_UNNECESSARY. > > > >

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On Sat, Nov 14, 2015 at 08:08:49AM +0100, Christoph Hellwig wrote: > On Fri, Nov 13, 2015 at 11:25:13AM -0700, Jason Gunthorpe wrote: > > For instance, like this, not fulling draining the cq and then doing: > > > > > + completed = __ib_process_cq(cq, budget); > > > + if (completed < budget) { > >

Re: memory registration updates

Christoph, This series removes huge chunks of code related to old memory registration methods that we don't use anymore, and then simplifies the current memory registration API Let's split out patches 10,11 from this set because these patches are logically completely different from the rest

Re: [PATCH 3/9] IB: add a helper to safely drain a QP

So Maybe we should have: void ib_drain_qp(struct ib_qp *qp) Christoph suggested that this flushing would be taken care of by rdma_disconnect which sounds even better I think.. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to

Re: [PATCH 3/9] IB: add a helper to safely drain a QP

That won't work for iWARP. Is this code new? I didn't see any errors that would result from this code when I tested iSER over cxgb4 with the old iwarp support patches. Steve, I think I figured out why this works with iWARP. For iWARP, rdma_disconnect() calls iw_cm_disconnect() with

Re: memory registration updates

On Mon, Nov 23, 2015 at 11:03:42AM +0200, Sagi Grimberg wrote: > Christoph, > >> This series removes huge chunks of code related to old memory >> registration methods that we don't use anymore, and then simplifies the >> current memory registration API > > Let's split out patches 10,11 from this

Re: [PATCH 3/9] IB: add a helper to safely drain a QP

On Mon, Nov 23, 2015 at 12:35:44PM +0200, Sagi Grimberg wrote: > >> So Maybe we should have: >> void ib_drain_qp(struct ib_qp *qp) > > Christoph suggested that this flushing would be taken care > of by rdma_disconnect which sounds even better I think.. Note that will only work once we've

RE: [PATCH 3/9] IB: add a helper to safely drain a QP

> -Original Message- > From: Sagi Grimberg [mailto:sa...@dev.mellanox.co.il] > Sent: Monday, November 23, 2015 4:29 AM > To: Steve Wise; 'Christoph Hellwig'; linux-rdma@vger.kernel.org > Cc: bart.vanass...@sandisk.com; ax...@fb.com; linux-s...@vger.kernel.org; >

RE: [PATCH 3/9] IB: add a helper to safely drain a QP

> -Original Message- > From: linux-kernel-ow...@vger.kernel.org > [mailto:linux-kernel-ow...@vger.kernel.org] On Behalf Of Sagi Grimberg > Sent: Monday, November 23, 2015 4:36 AM > To: Steve Wise; 'Christoph Hellwig'; linux-rdma@vger.kernel.org > Cc: bart.vanass...@sandisk.com;

[PATCH v1 6/9] xprtrdma: Add ro_unmap_sync method for all-physical registration

physical's ro_unmap is synchronous already. The new ro_unmap_sync method just has to DMA unmap all MRs associated with the RPC request. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/physical_ops.c | 13 + 1 file changed, 13 insertions(+) diff --git

[PATCH v1 4/9] xprtrdma: Add ro_unmap_sync method for FRWR

FRWR's ro_unmap is asynchronous. The new ro_unmap_sync posts LOCAL_INV Work Requests and waits for them to complete before returning. Note also, DMA unmapping is now done _after_ invalidation. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/frwr_ops.c | 137

[PATCH v1 9/9] xprtrdma: Revert commit e7104a2a9606 ('xprtrdma: Cap req_cqinit').

The root of the problem was that sends (especially unsignalled FASTREG and LOCAL_INV Work Requests) were not properly flow- controlled, which allowed a send queue overrun. Now that the RPC/RDMA reply handler waits for invalidation to complete, the send queue is properly flow-controlled. Thus this

[PATCH v1 5/9] xprtrdma: Add ro_unmap_sync method for FMR

FMR's ro_unmap method is already synchronous because ib_unmap_fmr() is a synchronous verb. However, some improvements can be made here. 1. Gather all the MRs for the RPC request onto a list, and invoke ib_unmap_fmr() once with that list. This reduces the number of doorbells when there is

[PATCH v1 8/9] xprtrdma: Invalidate in the RPC reply handler

There is a window between the time the RPC reply handler wakes the waiting RPC task and when xprt_release() invokes ops->buf_free. During this time, memory regions containing the data payload may still be accessed by a broken or malicious server, but the RPC application has already been allowed

[PATCH v1 2/8] svcrdma: Define maximum number of backchannel requests

Extra resources for handling backchannel requests have to be pre-allocated when a transport instance is created. Set a limit. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h |5 + net/sunrpc/xprtrdma/svc_rdma_transport.c |6 +- 2

[PATCH v1 0/9] NFS/RDMA client patches for 4.5

For 4.5, I'd like to address the send queue accounting and invalidation/unmap ordering issues Jason brought up a couple of months ago. Here's a first shot at that. Also available in the "nfs-rdma-for-4.5" topic branch of this git repo: git://git.linux-nfs.org/projects/cel/cel-2.6.git Or for

[PATCH v1 6/8] xprtrdma: Add class for RDMA backwards direction transport

To support the server-side of an NFSv4.1 backchannel on RDMA connections, add a transport class for backwards direction operation. Signed-off-by: Chuck Lever --- include/linux/sunrpc/xprt.h |1 net/sunrpc/xprt.c|1

[PATCH v1 1/8] svcrdma: Do not send XDR roundup bytes for a write chunk

Minor optimization: when dealing with write chunk XDR roundup, do not post a Write WR for the zero bytes in the pad. Simply update the write segment in the RPC-over-RDMA header to reflect the extra pad bytes. The Reply chunk is also a write chunk, but the server does not use send_write_chunks()

[PATCH v1 4/8] svcrdma: Add infrastructure to send backwards direction RPC/RDMA calls

To support the NFSv4.1 backchannel on RDMA connections, add a mechanism for sending a backwards-direction RPC/RDMA call on a connection established by a client. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h |2 +

[PATCH v1 3/8] svcrdma: Add svc_rdma_get_context() API that is allowed to fail

To support backward direction calls, I'm going to add an svc_rdma_get_context() call in the client RDMA transport. Called from ->buf_alloc(), we can't sleep waiting for memory. So add an API that can get a server op_ctxt but won't sleep. Signed-off-by: Chuck Lever ---

[PATCH v1 0/8] NFS/RDMA server patches for 4.5

Here are patches to support server-side bi-directional RPC/RDMA operation (to enable NFSv4.1 on RPC/RDMA transports). These still need testing, but they are ready for initial review. Also available in the "nfsd-rdma-for-4.5" topic branch of this git repo:

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On 11/23/2015 02:18 PM, Jason Gunthorpe wrote: On Mon, Nov 23, 2015 at 01:54:05PM -0800, Bart Van Assche wrote: What I don't see is how SRP handles things when the sendq fills up, ie the case where __srp_get_tx_iu() == NULL. It looks like the driver starts to panic and generates printks. I can't

[PATCH v1 2/9] xprtrdma: Move struct ib_send_wr off the stack

For FRWR FASTREG and LOCAL_INV, move the ib_*_wr structure off the stack. This allows frwr_op_map and frwr_op_unmap to chain WRs together without limit to register or invalidate a set of MRs with a single ib_post_send(). (This will be for chaining LOCAL_INV requests). Signed-off-by: Chuck Lever

[PATCH v1 3/9] xprtrdma: Introduce ro_unmap_sync method

In the current xprtrdma implementation, some memreg strategies implement ro_unmap synchronously (the MR is knocked down before the method returns) and some asynchonously (the MR will be knocked down and returned to the pool in the background). To guarantee the MR is truly invalid before the RPC

[PATCH v1 7/9] SUNRPC: Introduct xprt_commit_rqst()

I'm about to add code in the RPC/RDMA reply handler between the xprt_lookup_rqst() and xprt_complete_rqst() call site that needs to execute outside of spinlock critical sections. Add a hook to remove an rpc_rqst from the pending list once the transport knows its going to invoke

[PATCH v1 1/9] xprtrdma: Add a safety margin for receive buffers

Rarely, senders post a Send that is larger than the client's inline threshold. That can be due to a bug, or the client and server may not have communicated about their inline limits. RPC-over-RDMA currently doesn't specify any particular limit on inline size, so peers have to guess what it is. It

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On Mon, Nov 23, 2015 at 01:54:05PM -0800, Bart Van Assche wrote: > Not really ... Please have a look at the SRP initiator source code. What the > SRP initiator does is to poll the send queue before sending a new > SCSI I see that. What I don't see is how SRP handles things when the sendq fills

[PATCH v1 7/8] svcrdma: No need to count WRs in svc_rdma_send()

Minor optimization: Instead of counting WRs in a chain, have callers pass in the number of WRs they've prepared. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h |2 +- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c |9 ++---

[PATCH v1 5/8] svcrdma: Add infrastructure to receive backwards direction RPC/RDMA replies

To support the NFSv4.1 backchannel on RDMA connections, add a capability for receiving an RPC/RDMA reply on a connection established by a client. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/rpc_rdma.c | 76 +++

[PATCH v1 8/8] svcrdma: Remove svc_rdma_fastreg_mr::access_flags field

Clean up: The access_flags field is not used outside of rdma_read_chunk_frmr() and is always set to the same value. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h |1 - net/sunrpc/xprtrdma/svc_rdma_recvfrom.c |3 +-- 2 files changed, 1

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On Mon, Nov 23, 2015 at 02:33:05PM -0800, Bart Van Assche wrote: > On 11/23/2015 02:18 PM, Jason Gunthorpe wrote: > >On Mon, Nov 23, 2015 at 01:54:05PM -0800, Bart Van Assche wrote: > >What I don't see is how SRP handles things when the > >sendq fills up, ie the case where __srp_get_tx_iu() ==

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On 11/23/2015 01:28 PM, Jason Gunthorpe wrote: On Mon, Nov 23, 2015 at 01:04:25PM -0800, Bart Van Assche wrote: Considerable time ago the send queue in the SRP initiator driver was modified from signaled to non-signaled to reduce the number of interrupts triggered by the SRP initiator driver.

Re: [PATCH v1 8/8] svcrdma: Remove svc_rdma_fastreg_mr::access_flags field

On Mon, Nov 23, 2015 at 07:53:04PM -0500, Chuck Lever wrote: > > Wait, the REMOTE_WRITE is there to support iWARP, but it isn't > > needed for IB or RoCE. Shouldn't this be updated to peek at those > > new attributes to decide, instead of remaining unconditional? > > That???s coming in another

Re: [PATCH v1 3/9] xprtrdma: Introduce ro_unmap_sync method

On Mon, Nov 23, 2015 at 05:14:14PM -0500, Chuck Lever wrote: > In the current xprtrdma implementation, some memreg strategies > implement ro_unmap synchronously (the MR is knocked down before the > method returns) and some asynchonously (the MR will be knocked down > and returned to the pool in

Future of FMR support, was: Re: [PATCH v1 5/9] xprtrdma: Add ro_unmap_sync method for FMR

On Mon, Nov 23, 2015 at 07:57:42PM -0500, Tom Talpey wrote: > On 11/23/2015 5:14 PM, Chuck Lever wrote: > >FMR's ro_unmap method is already synchronous because ib_unmap_fmr() > >is a synchronous verb. However, some improvements can be made here. > > I thought FMR support was about to be removed

Re: [PATCH v1 3/8] svcrdma: Add svc_rdma_get_context() API that is allowed to fail

> +struct svc_rdma_op_ctxt *svc_rdma_get_context_gfp(struct svcxprt_rdma *xprt, > + gfp_t flags) > +{ > + struct svc_rdma_op_ctxt *ctxt; > + > + ctxt = kmem_cache_alloc(svc_rdma_ctxt_cachep, flags); > + if (!ctxt) > + return

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On Mon, Nov 23, 2015 at 06:35:28PM -0800, Caitlin Bestler wrote: > Is it possible for an IB HCA to transmit a response on a QP and not > in that packet or a previous packet acknowledge something that it > has delivered to the user? AFAIK, the rules of ack coalescing do not interact with the send

Re: Future of FMR support, was: Re: [PATCH v1 5/9] xprtrdma: Add ro_unmap_sync method for FMR

On Mon, Nov 23, 2015 at 10:52:26PM -0800, Christoph Hellwig wrote: > > So at lest for 4.5 we're unlikely to be able to get rid of it alone > due to the RDS issue. We'll then need performance numbers for mlx4, > and figure out how much we care about mthca. mthca is unfortunately very popular in

Re: [PATCH v1 2/8] svcrdma: Define maximum number of backchannel requests

On 11/23/2015 5:20 PM, Chuck Lever wrote: Extra resources for handling backchannel requests have to be pre-allocated when a transport instance is created. Set a limit. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h |5 +

Re: [PATCH v1 2/8] svcrdma: Define maximum number of backchannel requests

> On Nov 23, 2015, at 7:39 PM, Tom Talpey wrote: > > On 11/23/2015 5:20 PM, Chuck Lever wrote: >> Extra resources for handling backchannel requests have to be >> pre-allocated when a transport instance is created. Set a limit. >> >> Signed-off-by: Chuck Lever

Re: [PATCH v1 2/8] svcrdma: Define maximum number of backchannel requests

On 11/23/2015 8:09 PM, Chuck Lever wrote: On Nov 23, 2015, at 7:39 PM, Tom Talpey wrote: On 11/23/2015 5:20 PM, Chuck Lever wrote: Extra resources for handling backchannel requests have to be pre-allocated when a transport instance is created. Set a limit. Signed-off-by:

Re: [PATCH v1 2/8] svcrdma: Define maximum number of backchannel requests

> On Nov 23, 2015, at 8:19 PM, Tom Talpey wrote: > > On 11/23/2015 8:09 PM, Chuck Lever wrote: >> >>> On Nov 23, 2015, at 7:39 PM, Tom Talpey wrote: >>> >>> On 11/23/2015 5:20 PM, Chuck Lever wrote: Extra resources for handling backchannel requests have

Re: [PATCH v1 2/8] svcrdma: Define maximum number of backchannel requests

On Mon, Nov 23, 2015 at 8:19 PM, Tom Talpey wrote: > On 11/23/2015 8:09 PM, Chuck Lever wrote: >> >> >>> On Nov 23, 2015, at 7:39 PM, Tom Talpey wrote: >>> >>> On 11/23/2015 5:20 PM, Chuck Lever wrote: Extra resources for handling backchannel requests

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On Mon, Nov 23, 2015 at 07:34:53PM -0500, Tom Talpey wrote: > Been there, seen that. Bluescreened on it, mysteriously. Yes, me too :( Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: [PATCH v1 6/8] xprtrdma: Add class for RDMA backwards direction transport

On 11/23/2015 5:21 PM, Chuck Lever wrote: To support the server-side of an NFSv4.1 backchannel on RDMA connections, add a transport class for backwards direction operation. So, what's special here is that it re-uses an existing forward channel's connection? If not, it would seem unnecessary to

Re: [PATCH v1 1/9] xprtrdma: Add a safety margin for receive buffers

On 11/23/2015 5:13 PM, Chuck Lever wrote: Rarely, senders post a Send that is larger than the client's inline threshold. That can be due to a bug, or the client and server may not have communicated about their inline limits. RPC-over-RDMA currently doesn't specify any particular limit on inline

Re: [PATCH v1 1/9] xprtrdma: Add a safety margin for receive buffers

On 11/23/2015 8:16 PM, Chuck Lever wrote: On Nov 23, 2015, at 7:55 PM, Tom Talpey wrote: On 11/23/2015 5:13 PM, Chuck Lever wrote: Rarely, senders post a Send that is larger than the client's inline threshold. That can be due to a bug, or the client and server may not have

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On Mon, Nov 23, 2015 at 03:30:42PM -0800, Caitlin Bestler wrote: >The receive completion can be safely assumed to indicate transmit >completion over a reliable connection unless your peer has gone >completely bonkers and is replying to a command that it did not >receive. Perhaps

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On 11/23/2015 7:00 PM, Jason Gunthorpe wrote: On Mon, Nov 23, 2015 at 03:30:42PM -0800, Caitlin Bestler wrote: The receive completion can be safely assumed to indicate transmit completion over a reliable connection unless your peer has gone completely bonkers and is replying to a

Re: [PATCH v1 8/8] svcrdma: Remove svc_rdma_fastreg_mr::access_flags field

> On Nov 23, 2015, at 7:52 PM, Tom Talpey wrote: > > On 11/23/2015 5:21 PM, Chuck Lever wrote: >> Clean up: The access_flags field is not used outside of >> rdma_read_chunk_frmr() and is always set to the same value. >> >> Signed-off-by: Chuck Lever >>

Re: [PATCH v1 8/9] xprtrdma: Invalidate in the RPC reply handler

On 11/23/2015 5:14 PM, Chuck Lever wrote: There is a window between the time the RPC reply handler wakes the waiting RPC task and when xprt_release() invokes ops->buf_free. During this time, memory regions containing the data payload may still be accessed by a broken or malicious server, but the

Re: [PATCH] IB/srp: Fix possible send queue overflow

On 11/22/2015 05:37 AM, Christoph Hellwig wrote: On Tue, Nov 10, 2015 at 12:35:05PM +0200, Sagi Grimberg wrote: Are you planning to pick this up? Note that this patch is stable material as well. Doug? any plans for this patch? We should really get this in an into -stable. Bart, can you

Re: [PATCH v1 1/9] xprtrdma: Add a safety margin for receive buffers

> On Nov 23, 2015, at 7:55 PM, Tom Talpey wrote: > > On 11/23/2015 5:13 PM, Chuck Lever wrote: >> Rarely, senders post a Send that is larger than the client's inline >> threshold. That can be due to a bug, or the client and server may >> not have communicated about their inline

Re: [PATCH v1 5/8] svcrdma: Add infrastructure to receive backwards direction RPC/RDMA replies

On 11/23/2015 5:20 PM, Chuck Lever wrote: To support the NFSv4.1 backchannel on RDMA connections, add a capability for receiving an RPC/RDMA reply on a connection established by a client. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/rpc_rdma.c | 76

Re: [PATCH v1 8/8] svcrdma: Remove svc_rdma_fastreg_mr::access_flags field

On 11/23/2015 5:21 PM, Chuck Lever wrote: Clean up: The access_flags field is not used outside of rdma_read_chunk_frmr() and is always set to the same value. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h |1 -

Re: [PATCH v1 5/9] xprtrdma: Add ro_unmap_sync method for FMR

On 11/23/2015 5:14 PM, Chuck Lever wrote: FMR's ro_unmap method is already synchronous because ib_unmap_fmr() is a synchronous verb. However, some improvements can be made here. I thought FMR support was about to be removed in the core. 1. Gather all the MRs for the RPC request onto a

Re: srp state in current mainline

On 11/22/2015 07:31 AM, Christoph Hellwig wrote: On Sun, Nov 22, 2015 at 05:26:28PM +0200, Sagi Grimberg wrote: No. register_always=Y is already broken in 4.3, but register_always=N is now also broken in 4.4. OK, I'm confused so please let me understand slowly :) Your patch "ib_srp:

Re: [PATCH v1 1/9] xprtrdma: Add a safety margin for receive buffers

> On Nov 23, 2015, at 8:22 PM, Tom Talpey wrote: > > On 11/23/2015 8:16 PM, Chuck Lever wrote: >> >>> On Nov 23, 2015, at 7:55 PM, Tom Talpey wrote: >>> >>> On 11/23/2015 5:13 PM, Chuck Lever wrote: Rarely, senders post a Send that is larger than the

Re: [PATCH v1 5/8] svcrdma: Add infrastructure to receive backwards direction RPC/RDMA replies

I'll have to think about whether I agree with that as a protocol statement. Chunks in a reply are there to account for the data that is handled in the chunk of a request. So it kind of comes down to whether RDMA is allowed (or used) on the backchannel. I still think that is fundamentally an

Re: [PATCH v1 5/8] svcrdma: Add infrastructure to receive backwards direction RPC/RDMA replies

> On Nov 23, 2015, at 7:44 PM, Tom Talpey wrote: > > On 11/23/2015 5:20 PM, Chuck Lever wrote: >> To support the NFSv4.1 backchannel on RDMA connections, add a >> capability for receiving an RPC/RDMA reply on a connection >> established by a client. >> >> Signed-off-by: Chuck

Re: [PATCH v1 2/8] svcrdma: Define maximum number of backchannel requests

On 11/23/2015 8:36 PM, Chuck Lever wrote: On Nov 23, 2015, at 8:19 PM, Tom Talpey wrote: On 11/23/2015 8:09 PM, Chuck Lever wrote: On Nov 23, 2015, at 7:39 PM, Tom Talpey wrote: On 11/23/2015 5:20 PM, Chuck Lever wrote: Extra resources for handling

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On 11/23/2015 4:00 PM, Jason Gunthorpe wrote: On Mon, Nov 23, 2015 at 03:30:42PM -0800, Caitlin Bestler wrote: The receive completion can be safely assumed to indicate transmit completion over a reliable connection unless your peer has gone completely bonkers and is replying to a

Re: [PATCH v1 3/9] xprtrdma: Introduce ro_unmap_sync method

On Mon, Nov 23, 2015 at 10:45:56PM -0800, Christoph Hellwig wrote: > On Mon, Nov 23, 2015 at 05:14:14PM -0500, Chuck Lever wrote: > > In the current xprtrdma implementation, some memreg strategies > > implement ro_unmap synchronously (the MR is knocked down before the > > method returns) and some

Re: memory registration updates

I send 1-9 out separately earlier :) The other two sit on top of them and they are prep patches in a sense as they remove a lot of users of struct ib_mr that i don't have to modify in patches 10 and 11. Still, patches 10,11 are not really a part of this patchset. I think they need to stand

Re: [PATCH 11/11] IB: provide better access flags for fast registrations

On Mon, Nov 23, 2015 at 10:09:05AM -0500, Chuck Lever wrote: > Out of curiosity, why are you keeping the IB_ACCESS flags? We'll still need them for all kinds of other use cases (ib_get_dma_mr, userspace MRs, qp_access_flags). > It would be more efficient for providers to convert the > scope

Re: memory registration updates

On 11/22/2015 11:46 AM, Christoph Hellwig wrote: This series removes huge chunks of code related to old memory registration methods that we don't use anymore, and then simplifies the current memory registration API This expects my "IB: merge struct ib_device_attr into struct ib_device" patch to

Re: [PATCH 11/11] IB: provide better access flags for fast registrations

> On Nov 22, 2015, at 12:46 PM, Christoph Hellwig wrote: > > Instead of the confusing IB spec values provide a flags argument that > describes: > > a) the operation we perform the memory registration for, and > b) if we want to access it for read or write purposes. > > This

Re: [PATCH] IB/cma: Add a missing rcu_read_unlock()

On Fri, Nov 20, 2015 at 11:04:12AM -0800, Bart Van Assche wrote: > Ensure that validate_ipv4_net_dev() calls rcu_read_unlock() if > fib_lookup() fails. Detected by sparse. Compile-tested only. > > Fixes: "IB/cma: Validate routing of incoming requests" (commit f887f2ac87c2). > Cc: Haggai Eran

Re: [PATCH 11/11] IB: provide better access flags for fast registrations

On Sun, Nov 22, 2015 at 06:46:49PM +0100, Christoph Hellwig wrote: > Instead of the confusing IB spec values provide a flags argument that > describes: > > a) the operation we perform the memory registration for, and > b) if we want to access it for read or write purposes. > > This helps to

Re: garbage collect old memory registration code

On Sun, Nov 15, 2015 at 07:05:53PM +0100, Christoph Hellwig wrote: > This series removes huge chunks of code related to old memory > registration methods that we don't use anymore. > > This expects my "IB: merge struct ib_device_attr into struct ib_device" > patch to be already applied. > > Also

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On 11/23/2015 12:37 PM, Jason Gunthorpe wrote: On Sat, Nov 14, 2015 at 08:13:44AM +0100, Christoph Hellwig wrote: On Fri, Nov 13, 2015 at 03:06:36PM -0700, Jason Gunthorpe wrote: Looking at that thread and then at the patch a bit more.. +void ib_process_cq_direct(struct ib_cq *cq) [..] +

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On Mon, Nov 23, 2015 at 01:01:36PM -0700, Jason Gunthorpe wrote: > Okay, having now read the whole thing, I think I see the flow now. I don't > see any holes in the above, other than it is doing a bit more work > than it needs in some edges cases because it doesn't know if the CQ is > actually

Re: [PATCH for-next V1 5/9] IB/core: Add rdma_network_type to wc

> + /* Use the hint from IP Stack to select GID Type */ > + network_gid_type = ib_network_to_gid_type(addr->dev_addr.network); > + if (addr->dev_addr.network != RDMA_NETWORK_IB) { > + route->path_rec->gid_type = network_gid_type; > + /* TODO: get the hoplimit

Re: [PATCH for-next V1 3/9] IB/core: Add gid attributes to sysfs

On Thu, Oct 15, 2015 at 07:07:06PM +0300, Matan Barak wrote: > This patch set adds attributes of net device and gid type to each GID > in the GID table. Users that use verbs directly need to specify > the GID index. Since the same GID could have different types or > associated net devices, users

Re: [PATCH for-next V1 7/9] IB/cma: Add configfs for rdma_cm

On Thu, Oct 15, 2015 at 07:07:10PM +0300, Matan Barak wrote: > Users would like to control the behaviour of rdma_cm. > For example, old applications which don't set the > required RoCE gid type could be executed on RoCE V2 > network types. In order to support this configuration, > we implement a

Re: [PATCH for-next V1 9/9] IB/cma: Join and leave multicast groups with IGMP

On Thu, Oct 15, 2015 at 07:07:12PM +0300, Matan Barak wrote: > diff --git a/include/rdma/ib_sa.h b/include/rdma/ib_sa.h > index 0a40ed2..5bea0e8 100644 > +++ b/include/rdma/ib_sa.h > @@ -206,6 +206,9 @@ struct ib_sa_mcmember_rec { > u8 scope; > u8 join_state; >

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On Mon, Nov 23, 2015 at 01:04:25PM -0800, Bart Van Assche wrote: > Considerable time ago the send queue in the SRP initiator driver was > modified from signaled to non-signaled to reduce the number of interrupts > triggered by the SRP initiator driver. The SRP initiator driver polls the > send