I don't think that really represents how lots of apps actually use
RDMA.
RDMA is often buried down in the software stack (eg in a MPI), and by
the time a mapping gets used for RDMA transfer the link between the
FD, mmap and the MR is totally opaque.
Having a MR specific notification means the low level RDMA libraries
have a chance to deal with everything for the app.
Eg consider a HPC app using MPI that uses some DAX aware library to
get DAX backed mmap's. It then passes memory in those mmaps to the
MPI library to do transfers. The MPI creates the MR on demand.
I suspect one of the more interesting use cases might be a file server,
for which that's not the case. But otherwise I agree with the above,
and also thing that notifying the MR handle is the only way to go for
another very important reason: fencing. What if the application/library
does not react on the notification? With a per-MR notification we
can unregister the MR in kernel space and have a rock solid fencing
mechanism. And that is the most important bit here.
I agree we must deregister the MR in kernel space. As said, I think
its perfectly reasonable to let user-space see error completions and
provide query mechanism for MR granularity (unfortunately this will
probably need drivers assistance as they know how their device reports
in MR granularity access violations).
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
