Later I change another machine and set the TMPDIR to default /tmp, but the problem (low CPU utilization under 20%) still occur :<
Vincent On Thu, Oct 23, 2014 at 10:38 PM, Jeff Squyres (jsquyres) < jsquy...@cisco.com> wrote: > If normal users can't write to /tmp (or if /tmp is an NFS-mounted > filesystem), that's the underlying problem. > > @Vinson -- you should probably try to get that fixed. > > > > On Oct 23, 2014, at 10:35 AM, Joshua Ladd <jladd.m...@gmail.com> wrote: > > > It's not coming from OSHMEM but from the OPAL "shmem" framework. You are > going to get terrible performance - possibly slowing to a crawl having all > processes open their backing files for mmap on NSF. I think that's the > error that he's getting. > > > > > > Josh > > > > On Thu, Oct 23, 2014 at 6:06 AM, Vinson Leung <lwhvinson1...@gmail.com> > wrote: > > HI, Thanks for your reply:) > > I really run an MPI program (compile with OpenMPI and run with "mpirun > -n 8 ......"). My OpenMPI version is 1.8.3 and my program is Gromacs. BTW, > what is OSHMEM ? > > > > Best > > Vincent > > > > On Thu, Oct 23, 2014 at 12:21 PM, Ralph Castain <r...@open-mpi.org> > wrote: > > From your error message, I gather you are not running an MPI program, > but rather an OSHMEM one? Otherwise, I find the message strange as it only > would be emitted from an OSHMEM program. > > > > What version of OMPI are you trying to use? > > > >> On Oct 22, 2014, at 7:12 PM, Vinson Leung <lwhvinson1...@gmail.com> > wrote: > >> > >> Thanks for your reply:) > >> Follow your advice I tried to set the TMPDIR to /var/tmp and /dev/shm > and even reset to /tmp (I get the system permission), the problem still > occur (CPU utilization still lower than 20%). I have no idea why and ready > to give up OpenMPI instead of using other MPI library. > >> > >> --------Old Message------------- > >> > >> Date: Tue, 21 Oct 2014 22:21:31 -0400 > >> From: Brock Palen <bro...@umich.edu> > >> To: Open MPI Users <us...@open-mpi.org> > >> Subject: Re: [OMPI users] low CPU utilization with OpenMPI > >> Message-ID: <cc54135d-0cfe-440a-8df2-06b587e17...@umich.edu> > >> Content-Type: text/plain; charset=us-ascii > >> > >> Doing special files on NFS can be weird, try the other /tmp/ locations: > >> > >> /var/tmp/ > >> /dev/shm (ram disk careful!) > >> > >> Brock Palen > >> www.umich.edu/~brockp > >> CAEN Advanced Computing > >> XSEDE Campus Champion > >> bro...@umich.edu > >> (734)936-1985 > >> > >> > >> > >> > On Oct 21, 2014, at 10:18 PM, Vinson Leung <lwhvinson1...@gmail.com> > wrote: > >> > > >> > Because of permission reason (OpenMPI can not write temporary file to > the default /tmp directory), I change the TMPDIR to my local directory > (export TMPDIR=/home/user/tmp ) and then the MPI program can run. But the > CPU utilization is very low under 20% (8 MPI rank running in Intel Xeon > 8-core CPU). > >> > > >> > And I also got some message when I run with OpenMPI: > >> > [cn3:28072] 9 more processes have sent help message > help-opal-shmem-mmap.txt / mmap on nfs > >> > [cn3:28072] Set MCA parameter "orte_base_help_aggregate" to 0 to see > all help / error messages > >> > > >> > Any idea? > >> > Thanks > >> > > >> > VIncent > >> > _______________________________________________ > >> > users mailing list > >> > us...@open-mpi.org > >> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > >> > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/10/25548.php > >> _______________________________________________ > >> users mailing list > >> us...@open-mpi.org > >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > >> Link to this post: > http://www.open-mpi.org/community/lists/users/2014/10/25555.php > > > > > > _______________________________________________ > > users mailing list > > us...@open-mpi.org > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/10/25556.php > > > > > > _______________________________________________ > > users mailing list > > us...@open-mpi.org > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/10/25558.php > > > > _______________________________________________ > > users mailing list > > us...@open-mpi.org > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/10/25560.php > > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/10/25561.php >
