Hmm, wait. Yes. Your change went in after 1.8.4 and has the same effect. If yama ins't installed it is safe to assume that the ptrace scope is effectively 0. So, your patch does fix the issue.
-Nathan On Thu, Feb 19, 2015 at 02:53:47PM -0700, Nathan Hjelm wrote: > > I don't think that will fix this issue. In this case yama is not > installed and it appears PR_SET_PTRACER is not available. This forces > vader to assume that CMA can not be used when that isn't always the > case. I think it might be safe to assume that CMA is unrestricted here. > > -Nathan > > On Thu, Feb 19, 2015 at 04:35:00PM -0500, Aurélien Bouteiller wrote: > > Nathan, > > > > I think I already pushed a patch for this particular issue last month. I do > > not know if it has been back ported to release yet. > > > > See > > here:https://github.com/open-mpi/ompi/commit/ee3b0903164898750137d3b71a8f067e16521102 > > > > Aurelien > > > > -- > > ~~~ Aurélien Bouteiller, Ph.D. ~~~ > > ~ Research Scientist @ ICL ~ > > The University of Tennessee, Innovative Computing Laboratory > > 1122 Volunteer Blvd, suite 309, Knoxville, TN 37996 > > tel: +1 (865) 974-9375 fax: +1 (865) 974-8296 > > https://icl.cs.utk.edu/~bouteill/ > > > > > > > > > > > Le 19 févr. 2015 à 15:53, Nathan Hjelm <hje...@lanl.gov> a écrit : > > > > > > > > > Great! I will add an MCA variable to force CMA and also enable it if 1) > > > no yama and 2) no PR_SET_PTRACER. > > > > > > You might also look at using xpmem. You can find a version that supports > > > 3.x @ https://github.com/hjelmn/xpmem . It is a kernel module + > > > userspace library that can be used by vader as a single-copy mechanism. > > > > > > In benchmarks it performs better than CMA but it may or may not perform > > > better with a real application. > > > > > > See: > > > > > > http://blogs.cisco.com/performance/the-vader-shared-memory-transport-in-open-mpi-now-featuring-3-flavors-of-zero-copy > > > > > > -Nathan > > > > > > On Thu, Feb 19, 2015 at 03:32:43PM -0500, Eric Chamberland wrote: > > >> On 02/19/2015 02:58 PM, Nathan Hjelm wrote: > > >>> On Thu, Feb 19, 2015 at 12:16:49PM -0500, Eric Chamberland wrote: > > >>>> > > >>>> On 02/19/2015 11:56 AM, Nathan Hjelm wrote: > > >>>>> > > >>>>> If you have yama installed you can try: > > >>>> > > >>>> Nope, I do not have it installed... is it absolutely necessary? (and > > >>>> would > > >>>> it change something when it fails when I am root?) > > >>>> > > >>>> Other question: In addition to "--with-cma" configure flag, do we have > > >>>> to > > >>>> pass any options to "mpicc" when compiling/linking an mpi application > > >>>> to use > > >>>> cma? > > >>> > > >>> No. CMA should work out of the box. You appear to have a setup I haven't > > >>> yet tested. It doesn't have yama nor does it have the PR_SET_PTRACER > > >>> prctl. Its quite possible there are no restriction on ptrace in this > > >>> setup. Can you try changing the following line at > > >>> opal/mca/btl/vader/btl_vader_component.c:370 from: > > >>> > > >>> bool cma_happy = false; > > >>> > > >>> to > > >>> > > >>> bool cma_happy = true; > > >>> > > >> > > >> ok! (as of the officiel release, this is line 386.) > > >> > > >>> and let me know if that works. If it does I will update vader to allow > > >>> CMA in this configuration. > > >> > > >> Yep! It now works perfectly. Testing with > > >> https://computing.llnl.gov/tutorials/mpi/samples/C/mpi_bandwidth.c, on my > > >> own computer (dual Xeon), I have this: > > >> > > >> Without CMA: > > >> > > >> ***Message size: 1000000 *** best / avg / worst (MB/sec) > > >> task pair: 0 - 1: 8363.52 / 7946.77 / 5391.14 > > >> > > >> with CMA: > > >> task pair: 0 - 1: 9137.92 / 8955.98 / 7489.83 > > >> > > >> Great! > > >> > > >> Now I have to bench my real application... ;-) > > >> > > >> Thanks! > > >> > > >> Eric > > >> > > >> _______________________________________________ > > >> users mailing list > > >> us...@open-mpi.org > > >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > >> Link to this post: > > >> http://www.open-mpi.org/community/lists/users/2015/02/26355.php > > > _______________________________________________ > > > users mailing list > > > us...@open-mpi.org > > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > > Link to this post: > > > http://www.open-mpi.org/community/lists/users/2015/02/26356.php > > > > _______________________________________________ > > users mailing list > > us...@open-mpi.org > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > > http://www.open-mpi.org/community/lists/users/2015/02/26358.php > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26359.php
pgpO1eJLnaTfh.pgp
Description: PGP signature
