On Thu, Oct 16, 2014 at 05:27:54PM -0400, Gus Correa wrote: > Thank you, Aurelien! > > Aha, "vader btl", that is new to me! > I tought Vader was that man dressed in black in Star Wars, > Obi-Wan Kenobi's nemesis. > That was a while ago, my kids were children, > and Alec Guiness younger than Harrison Ford is today. > Oh, how nostalgic code developers can get when it comes > to naming things ... > > If I am using "vader", it is totally inadvertent. > There was no such a thing in Open MPI 1.6 and earlier. > > Now that you mentioned, I can see lots of it in the 1.8.3 > ompi_info output. > In addition, my stderr files show messages like this: > > imb.e38352:[1,5]<stddiag>:[node13:16334] mca: bml: Not using sm btl to > [[59987,1],26] on node node13 because vader btl has higher exclusivity > (65536 > 65535) > > So, you are right, "vader" is taking over and knocking off "sm" (and openib > and everybody else). > Darn Vader! > Probably knem is going down the tubes along with sm, right?
Depends. If there is a reason to continue supporting knem then vader will be updated to support it. I don't currently see a reason to at this time though (since sm continues to live for now). > I was used to sm, openib, self and tcp BTLs. > I normally just do "btl = ^tcp" in the MCA parameters file, > to stick to sm, openib, and self. > > That worked fine in 1.6.5 (and earlier), and knem worked > flawlessly there. > The same settings in 1.8.3 don't bring up the knem functionality. > So, this seems to be yet another change in 1.8.3 that I need to learn. > > Can you or some other list subscriber elaborate a bit about > this 'vader' btl? > The Open MPI FAQ doesn't have anthing about it. > What is it after all? > Does it play the same role as "sm", i.e., an intra-node btl? > Considering the name, is "vader" good or bad? > Or better: In which circumstances is "vader" good and when is it bad? Vader is a btl I originally wrote to support Cray's XPMEM shared memory interface. It was designed to be cleaner than btl/sm have better small message latency, bandwidth, and message rates. Because its latency is so much better than sm I removed the XPMEM requirement and added CMA support. > Should I give in to the dark side of the force and keep "vader" > turned on, or should I just do something like > "btl = ^tcp,^vader" ? You can turn off vader if you want to use knem. I would run some tests to see if there is much of a difference between sm/knem and vader though. I don't have any systems that have knem installed so I haven't been able to run these tests myself. I would primarily focus on the memory usage and the bandwidth. -Nathan
pgpk1r7Q9MgIv.pgp
Description: PGP signature
