> -----Original Message----- > From: linux-rdma-ow...@vger.kernel.org > [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Jason Gunthorpe > Sent: Friday, July 24, 2015 3:25 PM > To: Chuck Lever > Cc: Sagi Grimberg; Christoph Hellwig; linux-rdma; Liran Liss; Oren Duer > Subject: Re: [PATCH WIP 38/43] iser-target: Port to new memory registration > API > > On Fri, Jul 24, 2015 at 03:59:06PM -0400, Chuck Lever wrote: > > And RPC-over-RDMA version 1 does not have any way to signal that > > the server has invalidated the MRs. Such signaling would be a > > pre-requisite to allow the Linux NFS/RDMA client to interoperate > > with non-Linux NFS/RDMA servers that do not have such support. > > You can implement client support immediately, nothing special is > required. > > When processing a SEND WC check ex.invalidate_rkey and > IB_WC_WITH_INVALIDATE. If that rkey matches the MR associated with > that RPC slot then skip the invalidate. > > No protocol negotiation is required at that point. > > I am unclear what happens sever side if the server starts issuing > SEND_WITH_INVALIDATE to a client that doesn't expect it. The net > result is a MR would be invalidated twice. I don't know if this is OK > or not. >
It is ok to invalidate an already-invalid MR. > If it is OK, then the server can probably just start using it as > well without negotiation. > > Otherwise the client has to signal the server it supports it once at > connection setup. > > > For FRWR, you could post LINV from the receive completion upcall > > handler, and handle the rest of the invalidation from the send > > completion upcall, then poke the RPC reply handler. > > Yes > > > But this wouldn’t work at all for FMR, whose unmap verb is > > synchronous, would it? > > It could run the FMR unmap in a thread/workqueue/tasklet and then > complete the RPC side from that context. Same basic idea, using your > taslket not the driver's sendq context. > > > I’m not sure we’d buy more than a few microseconds here, and > > the receive upcall is single-threaded. > > Not sure on how that matches your performance goals, just remarking > that lauching the invalidate in the recv upcall and completing > processing from the sendq upcall is the very best performance you can > expect from this API. > > Jason > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
