[O-MPI devel] Current svn build issues..
Is someone in the process of moving around the ptl/sm code? My build of SVN revision 7005 fails there. Otherwise, any tips on testing the OpenIB BTL? with any luck, I'll have results for OpenMPI and mviapich later today.. -- -- Troy Benjegerdes'da hozer'ho...@hozed.org Somone asked me why I work on this free (http://www.fsf.org/philosophy/) software stuff and not get a real job. Charles Shultz had the best answer: "Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life." -- Charles Shultz
Re: [O-MPI devel] Current svn build issues..
Yes, this happened a week or three ago. Two ways to fix: 1. cd ompi/mca/ptl/sm/.deps foreach file (`ls`) echo >! $file end (assuming csh-flavored shell) Then you can make with no problems. 2. cd ompi/mca/ptl rm -rf sm svn up Then you'll need to re-autogen / configure / etc. On Aug 24, 2005, at 12:24 PM, Troy Benjegerdes wrote: Is someone in the process of moving around the ptl/sm code? My build of SVN revision 7005 fails there. Otherwise, any tips on testing the OpenIB BTL? with any luck, I'll have results for OpenMPI and mviapich later today.. -- -- Troy Benjegerdes'da hozer' ho...@hozed.org Somone asked me why I work on this free (http://www.fsf.org/ philosophy/) software stuff and not get a real job. Charles Shultz had the best answer: "Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life." -- Charles Shultz ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
[O-MPI devel] Open MPI over IB in action
Interesting news... Jim Barker installed Open MPI on one of our visualization teams' InfiniBand clusters. They successfully built ParaView and ran it to drive visualization on 3x3 "power wall" tiled display. ParaView has a history of breaking MPI's so I'm very happy that this went so smoothly. David -- David Daniel +1-505-667-0883 Advanced Computing Laboratory, LANL, MS-B287, Los Alamos NM 87545, USA
[O-MPI devel] OpenIB results
I have some Netpipe graphs of OpenMPI and Mviapich on OpenIB gen2 on Opteron systems, one with PCI-X IB cards, and the other with PCI-Express DDR IB cards. I'd like to chat with someone who fill me in a bit on what's going on with performance, and how the BTL for IB is implemented. One thing I'd like to try is modifying the kernel to allow 95% of physical memory to be registered for IB, and see how much of a difference this makes. I am also toying with the idea of writing an OpenIB BTL module for Netpipe, so I can evaluate the BTL layer performance directly without any MPI overhead. I also get the following messages on startup: libibverbs: Warning: no userspace device-specific driver found for uverbs0 driver search path: /usr/lib/infiniband [0,1,1][btl_openib_component.c:267:mca_btl_openib_component_init] No hca's found on this host! Something strange is going on, because it is finding at least 1 IB port, given the performance numbers. -- -- Troy Benjegerdes'da hozer'ho...@hozed.org Somone asked me why I work on this free (http://www.fsf.org/philosophy/) software stuff and not get a real job. Charles Shultz had the best answer: "Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life." -- Charles Shultz
Re: [O-MPI devel] processor affinity
On Tue, Aug 16, 2005 at 12:25:32PM -0400, Jeff Squyres wrote: > Processor affinity is now implemented. You must ask for it via the MCA > param "mpi_paffinity_alone". If this parameter is set to a nonzero > value, OMPI will assume that its job is alone on the nodes that it is > running on, and, if you have not oversubscribed the node, will bind MPI > processes to processors, starting with processor ID 0 (i.e., > effectively bindings MPI processes to the processor number equivalent > to their relative VPID on that node). > > Please let me know how this works out for everyone; thanks. Any thoughts on how to support NUMA with something like this? On the dual opteron w/DDR IB systems I've got, I'm seeing a big perfomance difference that primarily depends on which node the memory is on.
Re: [O-MPI devel] OpenIB results
Hello Troy, Can you forward the graphs? From the error output - doesn't look like it's actually using IB - may be using TCP instead. I won't be back in the office until friday - but could give you a call then if you'd like. Regards, Tim > I have some Netpipe graphs of OpenMPI and Mviapich on OpenIB gen2 on > Opteron systems, one with PCI-X IB cards, and the other with PCI-Express > DDR IB cards. > > I'd like to chat with someone who fill me in a bit on what's going on > with performance, and how the BTL for IB is implemented. One thing I'd > like to try is modifying the kernel to allow 95% of physical memory to > be registered for IB, and see how much of a difference this makes. > > I am also toying with the idea of writing an OpenIB BTL module for > Netpipe, so I can evaluate the BTL layer performance directly without > any MPI overhead. > > I also get the following messages on startup: > > libibverbs: Warning: no userspace device-specific driver found for > uverbs0 driver search path: /usr/lib/infiniband > [0,1,1][btl_openib_component.c:267:mca_btl_openib_component_init] > No hca's found on this host! > > Something strange is going on, because it is finding at least 1 IB > port, given the performance numbers. > > -- > -- > Troy Benjegerdes'da hozer'ho...@hozed.org > > Somone asked me why I work on this free (http://www.fsf.org/philosophy/) > software stuff and not get a real job. Charles Shultz had the best answer: > > "Why do musicians compose symphonies and poets write poems? They do it > because life wouldn't have any meaning for them if they didn't. That's why > I draw cartoons. It's my life." -- Charles Shultz > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel >
Re: [O-MPI devel] OpenIB results
Hi Troy, Tim and I would like to discuss this with you as well. One thing I would ask, are you using the btl_mvapi_leave_pinned=1 option? otherwise it is not a apples to apples comparison. - Galen On Aug 24, 2005, at 8:21 PM, Troy Benjegerdes wrote: I have some Netpipe graphs of OpenMPI and Mviapich on OpenIB gen2 on Opteron systems, one with PCI-X IB cards, and the other with PCI- Express DDR IB cards. I'd like to chat with someone who fill me in a bit on what's going on with performance, and how the BTL for IB is implemented. One thing I'd like to try is modifying the kernel to allow 95% of physical memory to be registered for IB, and see how much of a difference this makes. I am also toying with the idea of writing an OpenIB BTL module for Netpipe, so I can evaluate the BTL layer performance directly without any MPI overhead. I also get the following messages on startup: libibverbs: Warning: no userspace device-specific driver found for uverbs0 driver search path: /usr/lib/infiniband [0,1,1][btl_openib_component.c:267:mca_btl_openib_component_init] No hca's found on this host! Something strange is going on, because it is finding at least 1 IB port, given the performance numbers. -- -- Troy Benjegerdes'da hozer' ho...@hozed.org Somone asked me why I work on this free (http://www.fsf.org/ philosophy/) software stuff and not get a real job. Charles Shultz had the best answer: "Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life." -- Charles Shultz ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
