I should also say that I think this is something that may be worth clarifying in the standard. Either semantic is fine with me but there is no reason to change the behavior if it does not violate the standard.
-Nathan On Thu, Feb 11, 2016 at 01:35:28PM -0700, Nathan Hjelm wrote: > > Jeff probably ran with MPICH. Open MPI's are consistent with our choice > of definition for size=0: > > query: me=1, them=0, size=0, disp=1, base=0x0 > query: me=1, them=1, size=4, disp=1, base=0x1097e30f8 > query: me=1, them=2, size=4, disp=1, base=0x1097e30fc > query: me=1, them=3, size=4, disp=1, base=0x1097e3100 > query: me=2, them=0, size=0, disp=1, base=0x0 > query: me=2, them=1, size=4, disp=1, base=0x109fe10f8 > query: me=2, them=2, size=4, disp=1, base=0x109fe10fc > query: me=2, them=3, size=4, disp=1, base=0x109fe1100 > query: me=2, them=PROC_NULL, size=4, disp=1, base=0x109fe10f8 > query: me=3, them=0, size=0, disp=1, base=0x0 > query: me=3, them=1, size=4, disp=1, base=0x1088f30f8 > query: me=3, them=2, size=4, disp=1, base=0x1088f30fc > query: me=3, them=3, size=4, disp=1, base=0x1088f3100 > query: me=3, them=PROC_NULL, size=4, disp=1, base=0x1088f30f8 > query: me=0, them=0, size=0, disp=1, base=0x0 > query: me=0, them=1, size=4, disp=1, base=0x105b890f8 > query: me=0, them=2, size=4, disp=1, base=0x105b890fc > query: me=0, them=3, size=4, disp=1, base=0x105b89100 > query: me=0, them=PROC_NULL, size=4, disp=1, base=0x105b890f8 > query: me=1, them=PROC_NULL, size=4, disp=1, base=0x1097e30f8 > > To be portable only use MPI_Win_shared_query and do not rely on the > return value of base if you pass size = 0. > > -Nathan > > On Thu, Feb 11, 2016 at 08:23:16PM +0000, Peter Wind wrote: > > Thanks Jeff, that was an interesting result. The pointers are here well > > defined, also for the zero size segment. > > However I can't reproduce your output. I still get null pointers (output > > below). > > (I tried both 1.8.5 and 1.10.2 versions) > > What could be the difference? > > Peter > > mpirun -np 4 a.out > > query: me=0, them=0, size=0, disp=1, base=(nil) > > query: me=0, them=1, size=4, disp=1, base=0x2aee280030d0 > > query: me=0, them=2, size=4, disp=1, base=0x2aee280030d4 > > query: me=0, them=3, size=4, disp=1, base=0x2aee280030d8 > > query: me=0, them=PROC_NULL, size=4, disp=1, base=0x2aee280030d0 > > query: me=1, them=0, size=0, disp=1, base=(nil) > > query: me=1, them=1, size=4, disp=1, base=0x2aabbb9ce0d0 > > query: me=1, them=2, size=4, disp=1, base=0x2aabbb9ce0d4 > > query: me=1, them=3, size=4, disp=1, base=0x2aabbb9ce0d8 > > query: me=1, them=PROC_NULL, size=4, disp=1, base=0x2aabbb9ce0d0 > > query: me=2, them=0, size=0, disp=1, base=(nil) > > query: me=2, them=1, size=4, disp=1, base=0x2b1579dd40d0 > > query: me=2, them=2, size=4, disp=1, base=0x2b1579dd40d4 > > query: me=2, them=3, size=4, disp=1, base=0x2b1579dd40d8 > > query: me=2, them=PROC_NULL, size=4, disp=1, base=0x2b1579dd40d0 > > query: me=3, them=0, size=0, disp=1, base=(nil) > > query: me=3, them=1, size=4, disp=1, base=0x2ac8d2c350d0 > > query: me=3, them=2, size=4, disp=1, base=0x2ac8d2c350d4 > > query: me=3, them=3, size=4, disp=1, base=0x2ac8d2c350d8 > > query: me=3, them=PROC_NULL, size=4, disp=1, base=0x2ac8d2c350d0 > > > > ---------------------------------------------------------------------- > > > > See attached. Output below. Note that the base you get for ranks 0 > > and > > 1 is the same, so you need to use the fact that size=0 at rank=0 to > > know > > not to dereference that pointer and expect to be writing into rank 0's > > memory, since you will write into rank 1's. > > I would probably add "if (size==0) base=NULL;" for good measure. > > Jeff > > > > $ mpirun -n 4 ./a.out > > > > query: me=0, them=0, size=0, disp=1, base=0x10bd64000 > > > > query: me=0, them=1, size=4, disp=1, base=0x10bd64000 > > > > query: me=0, them=2, size=4, disp=1, base=0x10bd64004 > > > > query: me=0, them=3, size=4, disp=1, base=0x10bd64008 > > > > query: me=0, them=PROC_NULL, size=4, disp=1, base=0x10bd64000 > > > > query: me=1, them=0, size=0, disp=1, base=0x102d3b000 > > > > query: me=1, them=1, size=4, disp=1, base=0x102d3b000 > > > > query: me=1, them=2, size=4, disp=1, base=0x102d3b004 > > > > query: me=1, them=3, size=4, disp=1, base=0x102d3b008 > > > > query: me=1, them=PROC_NULL, size=4, disp=1, base=0x102d3b000 > > > > query: me=2, them=0, size=0, disp=1, base=0x10aac1000 > > > > query: me=2, them=1, size=4, disp=1, base=0x10aac1000 > > > > query: me=2, them=2, size=4, disp=1, base=0x10aac1004 > > > > query: me=2, them=3, size=4, disp=1, base=0x10aac1008 > > > > query: me=2, them=PROC_NULL, size=4, disp=1, base=0x10aac1000 > > > > query: me=3, them=0, size=0, disp=1, base=0x100fa2000 > > > > query: me=3, them=1, size=4, disp=1, base=0x100fa2000 > > > > query: me=3, them=2, size=4, disp=1, base=0x100fa2004 > > > > query: me=3, them=3, size=4, disp=1, base=0x100fa2008 > > > > query: me=3, them=PROC_NULL, size=4, disp=1, base=0x100fa2000 > > > > On Thu, Feb 11, 2016 at 8:55 AM, Jeff Hammond <jeff.scie...@gmail.com> > > wrote: > > > > On Thu, Feb 11, 2016 at 8:46 AM, Nathan Hjelm <hje...@lanl.gov> > > wrote: > > > > > > > > > On Thu, Feb 11, 2016 at 02:17:40PM +0000, Peter Wind wrote: > > > > I would add that the present situation is bound to give > > problems for some > > > > users. > > > > It is natural to divide an array in segments, each process > > treating its > > > > own segment, but needing to read adjacent segments too. > > > > MPI_Win_allocate_shared seems to be designed for this. > > > > This will work fine as long as no segment as size zero. It can > > also be > > > > expected that most testing would be done with all segments > > larger than > > > > zero. > > > > The document adding "size = 0 is valid", would also make > > people > > confident > > > > that it will be consistent for that special case too. > > > > > > Nope, that statement says its ok for a rank to specify that the > > local > > > shared memory segment is 0 bytes. Nothing more. The standard > > > unfortunately does not define what pointer value is returned for a > > rank > > > that specifies size = 0. Not sure if the RMA working group > > intentionally > > > left that undefine... Anyway, Open MPI does not appear to be out of > > > compliance with the standard here. > > > > > MPI_Alloc_mem doesn't say what happens if you pass size=0 either. > > The > > RMA working group intentionally tries to maintain consistency with > > the > > rest of the MPI standard whenever possible, so we did not create a > > new > > semantic here. > > MPI_Win_shared_query text includes this: > > "If all processes in the group attached to the window specified size > > = > > 0, then the call returns size = 0 and a baseptr as if MPI_ALLOC_MEM > > was called with size = 0." > > > > > > > > To be safe you should use MPI_Win_shared_query as suggested. You > > can > > > pass MPI_PROC_NULL as the rank to get the pointer for the first > > non-zero > > > sized segment in the shared memory window. > > Indeed! I forgot about that. MPI_Win_shared_query solves this > > problem for the user brilliantly. > > Jeff > > -- > > Jeff Hammond > > jeff.scie...@gmail.com > > http://jeffhammond.github.io/ > > > > -- > > Jeff Hammond > > jeff.scie...@gmail.com > > http://jeffhammond.github.io/ > > _______________________________________________ > > users mailing list > > us...@open-mpi.org > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > > http://www.open-mpi.org/community/lists/users/2016/02/28508.php > > > _______________________________________________ > > users mailing list > > us...@open-mpi.org > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > > http://www.open-mpi.org/community/lists/users/2016/02/28511.php > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2016/02/28513.php
pgppwfBLvKlpm.pgp
Description: PGP signature
