> We do have unregister on finalization. But this code doesn't introduce any > synchronization across processes on the same node, since kernel manages the > receive qp. If the reference counter will be moved to app responsibility, it > will enforce the app to mange the reference counter on app level , in other > words , it will require some process to be responsible for the QP. In context > of MPI-2 dynamics, such approach will make MPI community live much more > complicated.
I'm not sure we're fully communicating here. I understand the usage model for xrc. The basic problem is determining the lifetime of the xrc target qp. One process creates the xrc target qp. A process also owns connecting the target qp. Any process can create the xrc domain. The OFED patches introduce new user and kernel interfaces to create, modify, and query the xrc target qp. The proposed patches use the existing interfaces. The OFED patches did not address connection establishment, and the proposed patches do. In both patch sets, the xrc target qp is shared across multiple processes based on the xrc domain. The final question is when does an xrc target qp get destroyed. The proposal is to destroy the qp when the xrc domain is destroyed. No OOB communication between local processes or reg/unreg is necessary. (The main reason for the OFED patches requiring OOB communication and reg/unreg is to support sharing an xrc domain across *jobs*. The proposed patches would require code changes to apps currently written to the OFED patches that want to share an xrc domain across jobs. Other apps are likely to work unchanged.) The trade-offs between the two patch sets are an easier usage model, versus slightly more flexibility. In the past, MPI developers have pushed hard for easier RDMA connection models. And from recent email exchanges with the developers, the additional flexibility is not used. - Sean -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
