On Wed, Jun 05, 2013 at 06:10:11PM +0000, Jeff Squyres (jsquyres) wrote: > On Jun 5, 2013, at 10:14 AM, Jason Gunthorpe > <jguntho...@obsidianresearch.com> wrote: > > >> a = malloc(x); // a gets (va=0x100, pa=0x12345) back from malloc > >> MPI_Send(a, ...); // MPI registers 0x100 for len=x, and saves (0x100,x) in > >> reg cache > >> free(a); > >> a = malloc(x); // a gets (va=0x100, pa=0x98765) back from malloc > >> MPI_Send(a, ...); // MPI sees a=0x100 and things that it is already > >> registered > >> // ...kaboom > > > > ODP is supposed to completely solve this problem. The HCA's view and > > Kernels view of virtual to physical mapping becomes 100% synchronized, > > and there is no 'kaboom'. The kernel updates the HCA after the free, > > and after the 2nd malloc to 100% match the current virtual memory map > > in the process. > > Are you saying that the 2nd malloc will magically be registered > (with the new physical address)?
Yes, that is the whole point. ODP fundamentally fixes the *bug* where the HCA's view of process memory can become inconsistent with the kernel's view. 'magically be registered' is the wrong way to think about it - the registration of VA=0x100 is simply kept, and any change to the underlying physical mapping of the VA is synchronized with the HCA. > 'zactly. Roland has this ummunot branch in his git tree, where he > is in the middle of incorporating this functionality from the > original ummunotify standalone kernel module into libibverbs and > ibcore. Right, this was discussed at the Enterprise Summit a few weeks ago. I'm sure Roland would welcome patches... Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
