On Wed, Oct 28, 2015 at 03:56:08PM -0400, Chuck Lever wrote: > A key question is whether connection loss guarantees that the > server is fenced, for all device types, from existing > registered MRs. After reconnect, each MR must be registered > again before it can be accessed remotely. Is this true for the > Linux IB core, and all kernel providers, when using FRWR?
MR validation is not linked to a QP in any way. The memory is not fully fenced until the invalidate completes, or the MR unregister completes. Nothing else is good enough. > After a connection loss, the Linux kernel RPC/RDMA client > creates a new QP as it reconnects, thus I’d expect the QPN to > be different on the new connection. That should be enough to > prevent access to MRs that were registered with the previous > QP and PD, right? No, the NFS implementation creates a single PD for everything and any QP in the PD can access all the MRs. This is another security issue of a different sort. If there was one PD per QP then the above would be true, since the MR is linked to the PD. Even so, moving a QP out of RTR is not a synchronous operation, and until the CQ is drained, the disoposition of ongoing RDMA is not defined. Basically: You can't avoid actually doing a blocking invalidate operation. The core layer must allow for this if it is going to async cancel RPCs. FWIW, the same is true on the send side too, if the RPC had send buffers and gets canceled, you have to block until a CQ linked to that send is seen. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
