Hello Troy, Can you forward the graphs? From the error output - doesn't look like it's actually using IB - may be using TCP instead.
I won't be back in the office until friday - but could give you a call then if you'd like. Regards, Tim > I have some Netpipe graphs of OpenMPI and Mviapich on OpenIB gen2 on > Opteron systems, one with PCI-X IB cards, and the other with PCI-Express > DDR IB cards. > > I'd like to chat with someone who fill me in a bit on what's going on > with performance, and how the BTL for IB is implemented. One thing I'd > like to try is modifying the kernel to allow 95% of physical memory to > be registered for IB, and see how much of a difference this makes. > > I am also toying with the idea of writing an OpenIB BTL module for > Netpipe, so I can evaluate the BTL layer performance directly without > any MPI overhead. > > I also get the following messages on startup: > > libibverbs: Warning: no userspace device-specific driver found for > uverbs0 driver search path: /usr/lib/infiniband > [0,1,1][btl_openib_component.c:267:mca_btl_openib_component_init] > No hca's found on this host! > > Something strange is going on, because it is finding at least 1 IB > port, given the performance numbers. > > -- > -------------------------------------------------------------------------- > Troy Benjegerdes 'da hozer' ho...@hozed.org > > Somone asked me why I work on this free (http://www.fsf.org/philosophy/) > software stuff and not get a real job. Charles Shultz had the best answer: > > "Why do musicians compose symphonies and poets write poems? They do it > because life wouldn't have any meaning for them if they didn't. That's why > I draw cartoons. It's my life." -- Charles Shultz > _______________________________________________ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel >
