On Dec 20, 2006, at 7:04 PM, Jeff Squyres wrote:
I've been asked by the owner of the cluster "How can you prove to me
that this openmpi job is using the Infiniband network?"
At first I thought a simple netstat -an on the compute nodes might
tell
me, however I don't see the Infiniband IP's in the list so I'm
thinking
maybe I need to be looking elsewhere.
That's correct.
I completely forgot to explain this comment, sorry.
Open MPI does not use TCP over IB -- it uses the native "verbs"
interface (i.e., a lower lever than TCP over IB), and therefore a)
doesn't show up in tcpdump/netstat/etc., and b) gets much better
performance (TCP demands much processing on the host CPU, whereas
verbs can be offloaded to the processor on the HCA).
--
Jeff Squyres
Server Virtualization Business Unit
Cisco Systems
