Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-11 Thread Christopher Samuel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/06/10 18:43, Paul H. Hargrove wrote: > When the file is on a real (not tmpfs or other ramdisk) I am 95% certain > that this is an artifact of the Linux swapper/pager behavior which is > thinking it is being smart by "swapping ahead". Even when

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-11 Thread Paul H. Hargrove
Chris, I think that "reclaiming pages from the page cache" is the PROBLEM, not the solution. If I understand you correctly a lower value of "swappiness" means that the ANONYMOUS pages of an application's stack and heap are less likely to be subject to swap I/O. However, the concern here is

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-11 Thread Paul H. Hargrove
Sylvain Jeaugey wrote: On Thu, 10 Jun 2010, Paul H. Hargrove wrote: [snip] As for why mmap is slower. When the file is on a real (not tmpfs or other ramdisk) I am 95% certain that this is an artifact of the Linux swapper/pager behavior which is thinking it is being smart by "swapping ahea

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-11 Thread Jeff Squyres
On Jun 11, 2010, at 5:43 AM, Paul H. Hargrove wrote: > > Interesting. Do you think this behavior of the linux kernel would > > change if the file was unlink()ed after attach ? > > As Jeff pointed out, the file IS unlinked by Open MPI, presumably to > ensure it is not left behind in case of abnorm

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-11 Thread Sylvain Jeaugey
On Fri, 11 Jun 2010, Jeff Squyres wrote: On Jun 11, 2010, at 5:43 AM, Paul H. Hargrove wrote: Interesting. Do you think this behavior of the linux kernel would change if the file was unlink()ed after attach ? After a little talk with kernel guys, it seems that unlinking wouldn't change anythi

[OMPI devel] hwloc

2010-06-11 Thread Jeff Squyres
Just FYI: We fixed some Solaris issues in the hwloc paffinity the other day; it appears to be working properly on all platforms how. We'll let it soak a little longer, but I think we're looking good for the first step of removing all other paffinity components and just leaving hwloc and test.

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-11 Thread Barrett, Brian W
On Jun 11, 2010, at 5:10 AM, Jeff Squyres wrote: > On Jun 11, 2010, at 5:43 AM, Paul H. Hargrove wrote: > >>> Interesting. Do you think this behavior of the linux kernel would >>> change if the file was unlink()ed after attach ? >> >> As Jeff pointed out, the file IS unlinked by Open MPI, presum

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-11 Thread Jeff Squyres
On Jun 11, 2010, at 12:53 PM, Barrett, Brian W wrote: > The idea was one large memory segment for all processes and it wasn't > unlinked after complete attach so that we could have spawned procs also use > shmem (which never worked, of course). So I think we could unlink during > init at this