On 07/21/2015 07:59 PM, Jeff Squyres (jsquyres) wrote: > On Jul 21, 2015, at 4:38 PM, Orion Poplawski <or...@cora.nwra.com> wrote: >> >> I see that 1.10.0 can make use of libfabric, but I don't see much >> documentation on it and certainly nothing in the README file. > > Oops; shame on us. We'll update README. > >> Since libfabric >> makes use of libibverbs and librdmacm, does it replace that functionality in >> openmpi, or should we continue to build openmpi with direct verbs/rdmacm >> support? > > The current status of Mellanox IB support in libfabric is rudimentary, at > best (it's layered over verbs, as you noted). As such, OMPI doesn't use > libfabric with verbs -- there wouldn't be much point. libfabric is better > with other networks, such as Cisco usNIC and Intel PSM. There's a Cray uGNI > libfabric provider coming along, too -- but it's not in the released > libfabric distribution yet. > > Hence: > > 1. you should continue building the openib BTL with native verbs support > 2. you should use --with-libfabric to compile in support for Cisco usNIC- and > Intel PSM-based fabrics > > Make sense? >
Thanks for the explanation. -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA, Boulder/CoRA Office FAX: 303-415-9702 3380 Mitchell Lane or...@nwra.com Boulder, CO 80301 http://www.nwra.com
