Hi Jack, Please see my comments below > From Pavel Shamis: >>> We do have unregister on finalization. But this code doesn't introduce any >>> synchronization across processes on the same node, since kernel manages the >>> receive qp. If the reference counter will be moved to app responsibility, it >>> will enforce the app to mange the reference counter on app level , in other >>> words , it will require some process to be responsible for the QP. In >>> context >>> of MPI-2 dynamics, such approach will make MPI community live much more >>> complicated. >> > Why can't the server allocate a new domain per job? Who creates the target > QP? -- can't the > target QP creator create the domain (instead of the server), and provide the > domain handle to the server?
XRC domain is created by process that starts first. All the rest processes, that belong to the same mpi session and reside on the same node, join the domain. TGT QP is created by process that receive inbound connection first and it is not necessary the same process that created the domain. Even so we assume that both processes belong to the same domain, and belong to the same mpi session. > Once the calculation gets started (with other clients opening that domain and > creating XRC SRQs to receive messages via the TGT QP, > the TGT QP creator can dealloc the xrc domain and exit (without destroying > the TGT QP). > The xrc domain will not actually be deallocated in the low-level driver until > all XRC SRQ clients > also dealloc the domain (reducing its reference count to zero). > At that point, all the domain's TGT QPs will be destroyed as well. Does it mean that TGT QP as well is allocated on low level driver ? If so, I have no problem with this approach. > > We would, in fact, use opening/allocating the XRC domain, which is done > anyway, instead of registering > "interest" in a specific TGT QP, as the means of controlling target QP > lifetime. > > Is this an option? Sure, sounds good. Regards, Pasha-- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
