Dear Open MPI developer, in this post: http://www.open-mpi.org/community/lists/users/2012/10/20416.phpI already reported about a case when Open MPI silently (without any word of caution!) changed the transport from InfiniBand to IPoIB, thus loosing the performance.
Another case of 'secret' disabling of InfiniBand is the use of 'multiple' threading level (assume the threading support is enabled by --enable-mpi-thread-multiple). Please have a look at ompi/mca/btl/openib/btl_openib_component.c (v.1.6.1, ll.2504-2508). In this lines a message about disabling InfiniBand transport is compounded, but normally it did not came out because is seem to be intended to be debug info only.
The problem is not the fallback itself but the muted way it is done. The user has hardly a possibility to get it out about the application is creeping over TCP, unless the performance loss will be noticed and analysed.
Well, we believe that disabling of high performance network _without any word a caution_ is a bad thing, because it may lead to huge waste of resources (because an actual problem may not be noticed for years - the program seem to work!). We will probably forbid any fallback to workaround this scenarios in future.
Maybe a bit more verbosity at this place is a good idea? Best, Paul Kapinos -- Dipl.-Inform. Paul Kapinos - High Performance Computing, RWTH Aachen University, Center for Computing and Communication Seffenter Weg 23, D 52074 Aachen (Germany) Tel: +49 241/80-24915
smime.p7s
Description: S/MIME Cryptographic Signature
