As for the performance, my 4-node (64-processes) 3-hour job indicates Intel MPI and OpenMPI have close benchmarks. Intel MPI takes 2:53 while Open MPI takes 3:10.
It is interesting that all my MPI_Wtime calls show OpenMPI is faster (up to twice or even more) than Intel MPI in communication for a single loop, but in overall wall time Open MPI is 10% slower for like 500K loops. The computing times are nearly the same. This is a little confusing. I may set up and run a new test. Beichuan -----Original Message----- From: users [mailto:users-boun...@open-mpi.org] On Behalf Of Jeff Squyres (jsquyres) Sent: Thursday, March 20, 2014 11:15 To: Open MPI Users Subject: Re: [OMPI users] OpenMPI job initializing problem On Mar 20, 2014, at 12:48 PM, Beichuan Yan <beichuan....@colorado.edu> wrote: > 2. http://www.open-mpi.org/community/lists/users/2011/11/17684.php > In the upcoming OMPI v1.7, we revamped the shared memory setup code such that > it'll actually use /dev/shm properly, or use some other mechanism other than > a mmap file backed in a real filesystem. So the issue goes away. Woo hoo! > my comment: up to OMPI v1.7.4, this shmem issue is still there. However, it > is resolved in OMPI v1.7.5rc5. This is surprising. > > Anyway, OMPI v1.7.5rc5 works well for multi-processes-on-one-node (shmem) > mode on Spirit. There is no need to tune TCP or IB parameters to use it. My > code just runs well: Great! > My test data takes 20 minutes to run with OMPI v1.7.4, but needs less than 1 > minute with OMPI v1.7.5rc5. I don't know what the magic is. I am wondering > when OMPI v1.7.5 final will be released. Wow -- that sounds like a fundamental difference there. Could be something to do with the NFS tmp directory...? I could see how that could cause oodles of unnecessary network traffic. 1.7.5 should be released ...immanently... -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ _______________________________________________ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users
