On Jul 6, 2008, at 1:28 PM, Patrick Geoffray wrote:
WHAT: make mpi_leave_pinned=1 by default when a BTL is used that
would benefit from it (when possible; 0 when not, obviously)
The probable reason registration cache (aka leave_pinned) is
disabled by default is that it may be unsafe. Even if you use
mallocopt to never return memory to the OS, how do you guarantee that:
* malloc always enforce the mallocopt *hints*.
* pinned memory can safely be fork()ed (system() for example).
* pinned memory can safely be unmmap()ed (Direct I/O or file mapping
for example).
If you can't, one solution may be to write a simple MPI code that
corrupts MVAPICH and make some noise about it.
I can appreciate that, but it still remains that we are the *only*
OpenFabrics MPI that doesn't do this by default. So an app would
break *all* of them. To date, there hasn't been an app that caused
anyone to change their defaults with regard to this issue (AFAIK).
- I agree that we don't know that the mallopt hint will be obeyed; I
don't know how to test for that.
- pinned memory is already unsafe in OpenFabrics when forking.
- we can catch munmap (it's in the code base already) in most cases,
so I'm not too worried about that.
--
Jeff Squyres
Cisco Systems
