[OMPI devel] QE, mpif.h and the Intel compiler
Dear Open MPI developer, QE is an MPI program, cf. http://qe-forge.org/gf/download/frsrelease/224/1044/qe-6.0.tar.gz In FFTXlib a questionable source code part is contained which cannot be compiled using version 17 of the Intel compilers and Open MPI; I've condensed it (see attachment). Note that - using Intel MPI all tested compilers (Intel/GCC/Oracle Studio/PGI) can compile stick_base_TEST.f90 with no errors/warnings issued - using Open MPI the same is true for GCC/Oracle Studio/PGI; - using Intel compiler up to version 16 and Open MPI, *warnings* like (1) are issued (compilation succeds); - using Intel compiler /17, *errors* like (2) are issued (compilation fails). - when 'PRIVATE' statement in line #3 removed, also Intel compiler compile the snippet with no error. Well the questions is - is stick_base_TEST.f90 a correct MPI code? - if YES why does the Intel/17 + OpenMPI combination do not like this? If NO why to hell none of other compiler+MPI combinations complain about this? :o) Have a nice day, Paul Kapinos P.S. Did you noticed also this one? https://www.mail-archive.com/users@lists.open-mpi.org//msg30320.html - 1 -- /opt/MPI/openmpi-1.10.4/linux/intel_16.0.2.181/include/mpif-sizeof.h(2254): warning #6738: The type/rank/keyword signature for this specific procedure matches another specific procedure that shares the same generic-name. [PMPI_SIZEOF_REAL64_R15] SUBROUTINE PMPI_Sizeof_real64_r15(x, size, ierror) -^ - 2 -- /opt/MPI/openmpi-1.10.4/linux/intel_17.0.0.064/include/mpif-sizeof.h(220): error #5286: Ambiguous generic interface MPI_SIZEOF: previously declared specific procedure MPI_SIZEOF_COMPLEX32_R13 is not distinguishable from this declaration. [MPI_SIZEOF_COMPLEX32_R13] SUBROUTINE MPI_Sizeof_complex32_r13(x, size, ierror) -^ -- Dipl.-Inform. Paul Kapinos - High Performance Computing, RWTH Aachen University, IT Center Seffenter Weg 23, D 52074 Aachen (Germany) Tel: +49 241/80-24915 MODULE stick_base IMPLICIT NONE PRIVATE INCLUDE 'mpif.h' PUBLIC :: sticks_map_set CONTAINS SUBROUTINE sticks_map_set() INCLUDE 'mpif.h' END SUBROUTINE sticks_map_set END MODULE stick_base smime.p7s Description: S/MIME Cryptographic Signature ___ devel mailing list devel@lists.open-mpi.org https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
[OMPI devel] MPI_Win_lock semantic
Nathan,
we briefly discussed the test_lock1 test from the onesided test suite
using osc/pt2pt
https://github.com/open-mpi/ompi-tests/blob/master/onesided/test_lock1.c#L57-L70
task 0 does
MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
MPI_Send(...,dest=2,...)
and task 2 does
MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
MPI_Recv(...,source=0,...)
hoping to guarantee task 0 will acquire the lock first.
once in a while, the test fails when task 2 acquires the lock first
/* MPI_Win_lock() only sends a lock request, and return without owning
the lock */
so if task 1 is running on a loaded server, and even if task 2 requests
the lock *after* task 0,
lock request from task 2 can be processed first, and hence task 2 is not
guaranteed to acquire the lock *before* task 0.
can you please confirm MPI_Win_lock() behaves as it is supposed to ?
if yes, is there a way for task 0 to block until it acquires the lock ?
i modified the test, and inserted in task 0 a MPI_Get of 1 MPI_Double
*before* MPI_Send.
see my patch below (note i increased the message length)
my expectation is that the test would either success (e.g. task 0 gets
the lock first) or hang
(if task 1 gets the lock first)
surprisingly, the test never hangs (so far ...) but once in a while, it
fails (!), which makes me very confused
Any thoughts ?
Cheers,
Gilles
diff --git a/onesided/test_lock1.c b/onesided/test_lock1.c
index c549093..9fa3f8d 100644
--- a/onesided/test_lock1.c
+++ b/onesided/test_lock1.c
@@ -20,7 +20,7 @@ int
test_lock1(void)
{
double *a = NULL;
-size_t len = 10;
+size_t len = 100;
MPI_Winwin;
inti;
@@ -56,6 +56,7 @@ test_lock1(void)
*/
if (me == 0) {
MPI_Win_lock(MPI_LOCK_EXCLUSIVE, 1, 0, win);
+ MPI_Get(a,1,MPI_DOUBLE,1,0,1,MPI_DOUBLE,win);
MPI_Send(NULL, 0, MPI_BYTE, 2, 1001, MPI_COMM_WORLD);
MPI_Get(a,len,MPI_DOUBLE,1,0,len,MPI_DOUBLE,win);
MPI_Win_unlock(1, win);
@@ -76,6 +77,7 @@ test_lock1(void)
/* make sure 0 got the data from 1 */
for (i = 0; i < len; i++) {
if (a[i] != (double)(10*1+i)) {
+if (0 == nfail) fprintf(stderr, "at index %d, expected
%lf but got %lf\n", i, (double)10*1+i, a[i]);
nfail++;
}
}
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
Re: [OMPI devel] MPI_Win_lock semantic
MPI_Win_lock does not have to be blocking. In osc/rdma it is blocking in most
cases but not others (lock all with on-demand is non-blocking) but in osc/pt2pt
is is almost always non-blocking (it has to be blocking for proc self). If you
really want to ensure the lock is acquired you can call MPI_Win_flush. I think
this should work even if you have not started any RMA operations inside the
epoch.
-Nathan
> On Nov 21, 2016, at 7:53 PM, Gilles Gouaillardet wrote:
>
> Nathan,
>
>
> we briefly discussed the test_lock1 test from the onesided test suite using
> osc/pt2pt
>
> https://github.com/open-mpi/ompi-tests/blob/master/onesided/test_lock1.c#L57-L70
>
>
> task 0 does
>
> MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
>
> MPI_Send(...,dest=2,...)
>
>
> and task 2 does
>
> MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
>
> MPI_Recv(...,source=0,...)
>
>
> hoping to guarantee task 0 will acquire the lock first.
>
>
> once in a while, the test fails when task 2 acquires the lock first
>
> /* MPI_Win_lock() only sends a lock request, and return without owning the
> lock */
>
> so if task 1 is running on a loaded server, and even if task 2 requests the
> lock *after* task 0,
>
> lock request from task 2 can be processed first, and hence task 2 is not
> guaranteed to acquire the lock *before* task 0.
>
>
> can you please confirm MPI_Win_lock() behaves as it is supposed to ?
>
> if yes, is there a way for task 0 to block until it acquires the lock ?
>
>
> i modified the test, and inserted in task 0 a MPI_Get of 1 MPI_Double
> *before* MPI_Send.
>
> see my patch below (note i increased the message length)
>
>
> my expectation is that the test would either success (e.g. task 0 gets the
> lock first) or hang
>
> (if task 1 gets the lock first)
>
>
>
> surprisingly, the test never hangs (so far ...) but once in a while, it fails
> (!), which makes me very confused
>
>
> Any thoughts ?
>
>
> Cheers,
>
>
> Gilles
>
>
>
> diff --git a/onesided/test_lock1.c b/onesided/test_lock1.c
> index c549093..9fa3f8d 100644
> --- a/onesided/test_lock1.c
> +++ b/onesided/test_lock1.c
> @@ -20,7 +20,7 @@ int
> test_lock1(void)
> {
> double *a = NULL;
> -size_t len = 10;
> +size_t len = 100;
> MPI_Winwin;
> inti;
>
> @@ -56,6 +56,7 @@ test_lock1(void)
> */
> if (me == 0) {
>MPI_Win_lock(MPI_LOCK_EXCLUSIVE, 1, 0, win);
> + MPI_Get(a,1,MPI_DOUBLE,1,0,1,MPI_DOUBLE,win);
> MPI_Send(NULL, 0, MPI_BYTE, 2, 1001, MPI_COMM_WORLD);
>MPI_Get(a,len,MPI_DOUBLE,1,0,len,MPI_DOUBLE,win);
> MPI_Win_unlock(1, win);
> @@ -76,6 +77,7 @@ test_lock1(void)
> /* make sure 0 got the data from 1 */
>for (i = 0; i < len; i++) {
>if (a[i] != (double)(10*1+i)) {
> +if (0 == nfail) fprintf(stderr, "at index %d, expected %lf
> but got %lf\n", i, (double)10*1+i, a[i]);
>nfail++;
>}
>}
>
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
Re: [OMPI devel] MPI_Win_lock semantic
Why is MPI_Win_flush required to ensure the lock is acquired ? According to
the standard MPI_Win_flush "completes all outstanding RMA operations
initiated by the calling process to the target rank on the specified
window", which can be read as being a noop if no pending operations exists.
George.
On Mon, Nov 21, 2016 at 8:29 PM, Nathan Hjelm wrote:
> MPI_Win_lock does not have to be blocking. In osc/rdma it is blocking in
> most cases but not others (lock all with on-demand is non-blocking) but in
> osc/pt2pt is is almost always non-blocking (it has to be blocking for proc
> self). If you really want to ensure the lock is acquired you can call
> MPI_Win_flush. I think this should work even if you have not started any
> RMA operations inside the epoch.
>
> -Nathan
>
> > On Nov 21, 2016, at 7:53 PM, Gilles Gouaillardet
> wrote:
> >
> > Nathan,
> >
> >
> > we briefly discussed the test_lock1 test from the onesided test suite
> using osc/pt2pt
> >
> > https://github.com/open-mpi/ompi-tests/blob/master/
> onesided/test_lock1.c#L57-L70
> >
> >
> > task 0 does
> >
> > MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
> >
> > MPI_Send(...,dest=2,...)
> >
> >
> > and task 2 does
> >
> > MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
> >
> > MPI_Recv(...,source=0,...)
> >
> >
> > hoping to guarantee task 0 will acquire the lock first.
> >
> >
> > once in a while, the test fails when task 2 acquires the lock first
> >
> > /* MPI_Win_lock() only sends a lock request, and return without owning
> the lock */
> >
> > so if task 1 is running on a loaded server, and even if task 2 requests
> the lock *after* task 0,
> >
> > lock request from task 2 can be processed first, and hence task 2 is not
> guaranteed to acquire the lock *before* task 0.
> >
> >
> > can you please confirm MPI_Win_lock() behaves as it is supposed to ?
> >
> > if yes, is there a way for task 0 to block until it acquires the lock ?
> >
> >
> > i modified the test, and inserted in task 0 a MPI_Get of 1 MPI_Double
> *before* MPI_Send.
> >
> > see my patch below (note i increased the message length)
> >
> >
> > my expectation is that the test would either success (e.g. task 0 gets
> the lock first) or hang
> >
> > (if task 1 gets the lock first)
> >
> >
> >
> > surprisingly, the test never hangs (so far ...) but once in a while, it
> fails (!), which makes me very confused
> >
> >
> > Any thoughts ?
> >
> >
> > Cheers,
> >
> >
> > Gilles
> >
> >
> >
> > diff --git a/onesided/test_lock1.c b/onesided/test_lock1.c
> > index c549093..9fa3f8d 100644
> > --- a/onesided/test_lock1.c
> > +++ b/onesided/test_lock1.c
> > @@ -20,7 +20,7 @@ int
> > test_lock1(void)
> > {
> > double *a = NULL;
> > -size_t len = 10;
> > +size_t len = 100;
> > MPI_Winwin;
> > inti;
> >
> > @@ -56,6 +56,7 @@ test_lock1(void)
> > */
> > if (me == 0) {
> >MPI_Win_lock(MPI_LOCK_EXCLUSIVE, 1, 0, win);
> > + MPI_Get(a,1,MPI_DOUBLE,1,0,1,MPI_DOUBLE,win);
> > MPI_Send(NULL, 0, MPI_BYTE, 2, 1001, MPI_COMM_WORLD);
> >MPI_Get(a,len,MPI_DOUBLE,1,0,len,MPI_DOUBLE,win);
> > MPI_Win_unlock(1, win);
> > @@ -76,6 +77,7 @@ test_lock1(void)
> > /* make sure 0 got the data from 1 */
> >for (i = 0; i < len; i++) {
> >if (a[i] != (double)(10*1+i)) {
> > +if (0 == nfail) fprintf(stderr, "at index %d, expected
> %lf but got %lf\n", i, (double)10*1+i, a[i]);
> >nfail++;
> >}
> >}
> >
> > ___
> > devel mailing list
> > devel@lists.open-mpi.org
> > https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
>
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
>
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
Re: [OMPI devel] MPI_Win_lock semantic
True. The lock itself is not an RMA operation so I am not sure if it is
supposed to complete with flush. He may have to initiate an RMA operation to
get the semantics desired.
I do think osc/pt2pt will still flush and wait for the lock message. The
semantics are not wrong but could be more than an implementation has to provide.
-Nathan
> On Nov 21, 2016, at 8:47 PM, George Bosilca wrote:
>
> Why is MPI_Win_flush required to ensure the lock is acquired ? According to
> the standard MPI_Win_flush "completes all outstanding RMA operations
> initiated by the calling process to the target rank on the specified window",
> which can be read as being a noop if no pending operations exists.
>
> George.
>
>
>
> On Mon, Nov 21, 2016 at 8:29 PM, Nathan Hjelm wrote:
> MPI_Win_lock does not have to be blocking. In osc/rdma it is blocking in most
> cases but not others (lock all with on-demand is non-blocking) but in
> osc/pt2pt is is almost always non-blocking (it has to be blocking for proc
> self). If you really want to ensure the lock is acquired you can call
> MPI_Win_flush. I think this should work even if you have not started any RMA
> operations inside the epoch.
>
> -Nathan
>
> > On Nov 21, 2016, at 7:53 PM, Gilles Gouaillardet wrote:
> >
> > Nathan,
> >
> >
> > we briefly discussed the test_lock1 test from the onesided test suite using
> > osc/pt2pt
> >
> > https://github.com/open-mpi/ompi-tests/blob/master/onesided/test_lock1.c#L57-L70
> >
> >
> > task 0 does
> >
> > MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
> >
> > MPI_Send(...,dest=2,...)
> >
> >
> > and task 2 does
> >
> > MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
> >
> > MPI_Recv(...,source=0,...)
> >
> >
> > hoping to guarantee task 0 will acquire the lock first.
> >
> >
> > once in a while, the test fails when task 2 acquires the lock first
> >
> > /* MPI_Win_lock() only sends a lock request, and return without owning the
> > lock */
> >
> > so if task 1 is running on a loaded server, and even if task 2 requests the
> > lock *after* task 0,
> >
> > lock request from task 2 can be processed first, and hence task 2 is not
> > guaranteed to acquire the lock *before* task 0.
> >
> >
> > can you please confirm MPI_Win_lock() behaves as it is supposed to ?
> >
> > if yes, is there a way for task 0 to block until it acquires the lock ?
> >
> >
> > i modified the test, and inserted in task 0 a MPI_Get of 1 MPI_Double
> > *before* MPI_Send.
> >
> > see my patch below (note i increased the message length)
> >
> >
> > my expectation is that the test would either success (e.g. task 0 gets the
> > lock first) or hang
> >
> > (if task 1 gets the lock first)
> >
> >
> >
> > surprisingly, the test never hangs (so far ...) but once in a while, it
> > fails (!), which makes me very confused
> >
> >
> > Any thoughts ?
> >
> >
> > Cheers,
> >
> >
> > Gilles
> >
> >
> >
> > diff --git a/onesided/test_lock1.c b/onesided/test_lock1.c
> > index c549093..9fa3f8d 100644
> > --- a/onesided/test_lock1.c
> > +++ b/onesided/test_lock1.c
> > @@ -20,7 +20,7 @@ int
> > test_lock1(void)
> > {
> > double *a = NULL;
> > -size_t len = 10;
> > +size_t len = 100;
> > MPI_Winwin;
> > inti;
> >
> > @@ -56,6 +56,7 @@ test_lock1(void)
> > */
> > if (me == 0) {
> >MPI_Win_lock(MPI_LOCK_EXCLUSIVE, 1, 0, win);
> > + MPI_Get(a,1,MPI_DOUBLE,1,0,1,MPI_DOUBLE,win);
> > MPI_Send(NULL, 0, MPI_BYTE, 2, 1001, MPI_COMM_WORLD);
> >MPI_Get(a,len,MPI_DOUBLE,1,0,len,MPI_DOUBLE,win);
> > MPI_Win_unlock(1, win);
> > @@ -76,6 +77,7 @@ test_lock1(void)
> > /* make sure 0 got the data from 1 */
> >for (i = 0; i < len; i++) {
> >if (a[i] != (double)(10*1+i)) {
> > +if (0 == nfail) fprintf(stderr, "at index %d, expected %lf
> > but got %lf\n", i, (double)10*1+i, a[i]);
> >nfail++;
> >}
> >}
> >
> > ___
> > devel mailing list
> > devel@lists.open-mpi.org
> > https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
>
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
>
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
Re: [OMPI devel] MPI_Win_lock semantic
Thanks Nathan,
any thoughts about my modified version of the test ?
do i need to MPI_Win_flush() after the first MPI_Get() in order to
ensure the lock was acquired ?
(and hence the program will either success or hang, but never fail)
Cheers,
Gilles
On 11/22/2016 12:29 PM, Nathan Hjelm wrote:
MPI_Win_lock does not have to be blocking. In osc/rdma it is blocking in most
cases but not others (lock all with on-demand is non-blocking) but in osc/pt2pt
is is almost always non-blocking (it has to be blocking for proc self). If you
really want to ensure the lock is acquired you can call MPI_Win_flush. I think
this should work even if you have not started any RMA operations inside the
epoch.
-Nathan
On Nov 21, 2016, at 7:53 PM, Gilles Gouaillardet wrote:
Nathan,
we briefly discussed the test_lock1 test from the onesided test suite using
osc/pt2pt
https://github.com/open-mpi/ompi-tests/blob/master/onesided/test_lock1.c#L57-L70
task 0 does
MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
MPI_Send(...,dest=2,...)
and task 2 does
MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
MPI_Recv(...,source=0,...)
hoping to guarantee task 0 will acquire the lock first.
once in a while, the test fails when task 2 acquires the lock first
/* MPI_Win_lock() only sends a lock request, and return without owning the lock
*/
so if task 1 is running on a loaded server, and even if task 2 requests the
lock *after* task 0,
lock request from task 2 can be processed first, and hence task 2 is not
guaranteed to acquire the lock *before* task 0.
can you please confirm MPI_Win_lock() behaves as it is supposed to ?
if yes, is there a way for task 0 to block until it acquires the lock ?
i modified the test, and inserted in task 0 a MPI_Get of 1 MPI_Double *before*
MPI_Send.
see my patch below (note i increased the message length)
my expectation is that the test would either success (e.g. task 0 gets the lock
first) or hang
(if task 1 gets the lock first)
surprisingly, the test never hangs (so far ...) but once in a while, it fails
(!), which makes me very confused
Any thoughts ?
Cheers,
Gilles
diff --git a/onesided/test_lock1.c b/onesided/test_lock1.c
index c549093..9fa3f8d 100644
--- a/onesided/test_lock1.c
+++ b/onesided/test_lock1.c
@@ -20,7 +20,7 @@ int
test_lock1(void)
{
double *a = NULL;
-size_t len = 10;
+size_t len = 100;
MPI_Winwin;
inti;
@@ -56,6 +56,7 @@ test_lock1(void)
*/
if (me == 0) {
MPI_Win_lock(MPI_LOCK_EXCLUSIVE, 1, 0, win);
+ MPI_Get(a,1,MPI_DOUBLE,1,0,1,MPI_DOUBLE,win);
MPI_Send(NULL, 0, MPI_BYTE, 2, 1001, MPI_COMM_WORLD);
MPI_Get(a,len,MPI_DOUBLE,1,0,len,MPI_DOUBLE,win);
MPI_Win_unlock(1, win);
@@ -76,6 +77,7 @@ test_lock1(void)
/* make sure 0 got the data from 1 */
for (i = 0; i < len; i++) {
if (a[i] != (double)(10*1+i)) {
+if (0 == nfail) fprintf(stderr, "at index %d, expected %lf but got
%lf\n", i, (double)10*1+i, a[i]);
nfail++;
}
}
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
Re: [OMPI devel] MPI_Win_lock semantic
To be safe I would call MPI_Get then MPI_Win_flush. That lock will always be
acquired before the MPI_Win_flush call returns. As long as it is more than 0
bytes. We always short-circuit 0-byte operations in both osc/rdma and osc/pt2pt.
-Nathan
> On Nov 21, 2016, at 8:54 PM, Gilles Gouaillardet wrote:
>
> Thanks Nathan,
>
>
> any thoughts about my modified version of the test ?
>
> do i need to MPI_Win_flush() after the first MPI_Get() in order to ensure the
> lock was acquired ?
>
> (and hence the program will either success or hang, but never fail)
>
>
> Cheers,
>
>
> Gilles
>
>
> On 11/22/2016 12:29 PM, Nathan Hjelm wrote:
>> MPI_Win_lock does not have to be blocking. In osc/rdma it is blocking in
>> most cases but not others (lock all with on-demand is non-blocking) but in
>> osc/pt2pt is is almost always non-blocking (it has to be blocking for proc
>> self). If you really want to ensure the lock is acquired you can call
>> MPI_Win_flush. I think this should work even if you have not started any RMA
>> operations inside the epoch.
>>
>> -Nathan
>>
>>> On Nov 21, 2016, at 7:53 PM, Gilles Gouaillardet wrote:
>>>
>>> Nathan,
>>>
>>>
>>> we briefly discussed the test_lock1 test from the onesided test suite using
>>> osc/pt2pt
>>>
>>> https://github.com/open-mpi/ompi-tests/blob/master/onesided/test_lock1.c#L57-L70
>>>
>>>
>>> task 0 does
>>>
>>> MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
>>>
>>> MPI_Send(...,dest=2,...)
>>>
>>>
>>> and task 2 does
>>>
>>> MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
>>>
>>> MPI_Recv(...,source=0,...)
>>>
>>>
>>> hoping to guarantee task 0 will acquire the lock first.
>>>
>>>
>>> once in a while, the test fails when task 2 acquires the lock first
>>>
>>> /* MPI_Win_lock() only sends a lock request, and return without owning the
>>> lock */
>>>
>>> so if task 1 is running on a loaded server, and even if task 2 requests the
>>> lock *after* task 0,
>>>
>>> lock request from task 2 can be processed first, and hence task 2 is not
>>> guaranteed to acquire the lock *before* task 0.
>>>
>>>
>>> can you please confirm MPI_Win_lock() behaves as it is supposed to ?
>>>
>>> if yes, is there a way for task 0 to block until it acquires the lock ?
>>>
>>>
>>> i modified the test, and inserted in task 0 a MPI_Get of 1 MPI_Double
>>> *before* MPI_Send.
>>>
>>> see my patch below (note i increased the message length)
>>>
>>>
>>> my expectation is that the test would either success (e.g. task 0 gets the
>>> lock first) or hang
>>>
>>> (if task 1 gets the lock first)
>>>
>>>
>>>
>>> surprisingly, the test never hangs (so far ...) but once in a while, it
>>> fails (!), which makes me very confused
>>>
>>>
>>> Any thoughts ?
>>>
>>>
>>> Cheers,
>>>
>>>
>>> Gilles
>>>
>>>
>>>
>>> diff --git a/onesided/test_lock1.c b/onesided/test_lock1.c
>>> index c549093..9fa3f8d 100644
>>> --- a/onesided/test_lock1.c
>>> +++ b/onesided/test_lock1.c
>>> @@ -20,7 +20,7 @@ int
>>> test_lock1(void)
>>> {
>>> double *a = NULL;
>>> -size_t len = 10;
>>> +size_t len = 100;
>>> MPI_Winwin;
>>> inti;
>>>
>>> @@ -56,6 +56,7 @@ test_lock1(void)
>>> */
>>> if (me == 0) {
>>>MPI_Win_lock(MPI_LOCK_EXCLUSIVE, 1, 0, win);
>>> + MPI_Get(a,1,MPI_DOUBLE,1,0,1,MPI_DOUBLE,win);
>>> MPI_Send(NULL, 0, MPI_BYTE, 2, 1001, MPI_COMM_WORLD);
>>>MPI_Get(a,len,MPI_DOUBLE,1,0,len,MPI_DOUBLE,win);
>>> MPI_Win_unlock(1, win);
>>> @@ -76,6 +77,7 @@ test_lock1(void)
>>> /* make sure 0 got the data from 1 */
>>>for (i = 0; i < len; i++) {
>>>if (a[i] != (double)(10*1+i)) {
>>> +if (0 == nfail) fprintf(stderr, "at index %d, expected %lf
>>> but got %lf\n", i, (double)10*1+i, a[i]);
>>>nfail++;
>>>}
>>>}
>>>
>>> ___
>>> devel mailing list
>>> devel@lists.open-mpi.org
>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
>> ___
>> devel mailing list
>> devel@lists.open-mpi.org
>> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
>>
>
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
Re: [OMPI devel] MPI_Win_lock semantic
Gilles,
I looked at the test and I think the current behavior is indeed correct.
What matters for an exclusive lock is that all operations in an epoch
(everything surrounded by lock/unlock) are atomically applied to the
destination (and are not interleaved with other updates). As Nathan stated,
MPI_Win_lock might be implemented as non-blocking, in which case it is
totally legit for the process 2 to acquire the lock first, and update the
array before the process 0 access it. Thus the test will fail.
The test will never deadlock, because even if the MPI_Win_lock is
implemented as a blocking operation (which is also legit), the send and
receive match correctly with the lock/unlock.
Moreover, I think the behavior described by the comments can only be
implemented by enforcing an order between the only conceptually meaningful
operations unlock/send/recv.
if (me == 0) {
MPI_Win_lock(MPI_LOCK_EXCLUSIVE, 1, 0, win);
MPI_Get(a,len,MPI_DOUBLE,1,0,len,MPI_DOUBLE,win);
MPI_Win_unlock(1, win);
MPI_Send(NULL, 0, MPI_BYTE, 2, 1001, MPI_COMM_WORLD);
}
if (me == 2) {
/* this should block till 0 releases the lock. */
MPI_Recv(NULL, 0, MPI_BYTE, 0, 1001, MPI_COMM_WORLD,
MPI_STATUS_IGNORE);
MPI_Win_lock(MPI_LOCK_EXCLUSIVE, 1, 0, win);
MPI_Put(a,len,MPI_DOUBLE,1,0,len,MPI_DOUBLE,win);
MPI_Win_unlock(1, win);
}
However, if we relax the code a little and want to ensure the atomicity of
the operations, then we need to change the check to make sure that either
no elements of the array have been altered or all of them have been altered
(set to zero by process 2).
George.
On Mon, Nov 21, 2016 at 8:57 PM, Nathan Hjelm wrote:
> To be safe I would call MPI_Get then MPI_Win_flush. That lock will always
> be acquired before the MPI_Win_flush call returns. As long as it is more
> than 0 bytes. We always short-circuit 0-byte operations in both osc/rdma
> and osc/pt2pt.
>
> -Nathan
>
> > On Nov 21, 2016, at 8:54 PM, Gilles Gouaillardet
> wrote:
> >
> > Thanks Nathan,
> >
> >
> > any thoughts about my modified version of the test ?
> >
> > do i need to MPI_Win_flush() after the first MPI_Get() in order to
> ensure the lock was acquired ?
> >
> > (and hence the program will either success or hang, but never fail)
> >
> >
> > Cheers,
> >
> >
> > Gilles
> >
> >
> > On 11/22/2016 12:29 PM, Nathan Hjelm wrote:
> >> MPI_Win_lock does not have to be blocking. In osc/rdma it is blocking
> in most cases but not others (lock all with on-demand is non-blocking) but
> in osc/pt2pt is is almost always non-blocking (it has to be blocking for
> proc self). If you really want to ensure the lock is acquired you can call
> MPI_Win_flush. I think this should work even if you have not started any
> RMA operations inside the epoch.
> >>
> >> -Nathan
> >>
> >>> On Nov 21, 2016, at 7:53 PM, Gilles Gouaillardet
> wrote:
> >>>
> >>> Nathan,
> >>>
> >>>
> >>> we briefly discussed the test_lock1 test from the onesided test suite
> using osc/pt2pt
> >>>
> >>> https://github.com/open-mpi/ompi-tests/blob/master/
> onesided/test_lock1.c#L57-L70
> >>>
> >>>
> >>> task 0 does
> >>>
> >>> MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
> >>>
> >>> MPI_Send(...,dest=2,...)
> >>>
> >>>
> >>> and task 2 does
> >>>
> >>> MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
> >>>
> >>> MPI_Recv(...,source=0,...)
> >>>
> >>>
> >>> hoping to guarantee task 0 will acquire the lock first.
> >>>
> >>>
> >>> once in a while, the test fails when task 2 acquires the lock first
> >>>
> >>> /* MPI_Win_lock() only sends a lock request, and return without owning
> the lock */
> >>>
> >>> so if task 1 is running on a loaded server, and even if task 2
> requests the lock *after* task 0,
> >>>
> >>> lock request from task 2 can be processed first, and hence task 2 is
> not guaranteed to acquire the lock *before* task 0.
> >>>
> >>>
> >>> can you please confirm MPI_Win_lock() behaves as it is supposed to ?
> >>>
> >>> if yes, is there a way for task 0 to block until it acquires the lock ?
> >>>
> >>>
> >>> i modified the test, and inserted in task 0 a MPI_Get of 1 MPI_Double
> *before* MPI_Send.
> >>>
> >>> see my patch below (note i increased the message length)
> >>>
> >>>
> >>> my expectation is that the test would either success (e.g. task 0 gets
> the lock first) or hang
> >>>
> >>> (if task 1 gets the lock first)
> >>>
> >>>
> >>>
> >>> surprisingly, the test never hangs (so far ...) but once in a while,
> it fails (!), which makes me very confused
> >>>
> >>>
> >>> Any thoughts ?
> >>>
> >>>
> >>> Cheers,
> >>>
> >>>
> >>> Gilles
> >>>
> >>>
> >>>
> >>> diff --git a/onesided/test_lock1.c b/onesided/test_lock1.c
> >>> index c549093..9fa3f8d 100644
> >>> --- a/onesided/test_lock1.c
> >>> +++ b/onesided/test_lock1.c
> >>> @@ -20,7 +20,7 @@ int
> >>> test_lock1(void)
> >>> {
> >>> double *a = NULL;
> >>> -size_t len = 10;
> >>> +size_t len = 100;
> >>> MPI_Winwin;
> >>> inti;
> >>>
> >>> @@ -56,6
Re: [OMPI devel] QE, mpif.h and the Intel compiler
Paul, short answer, i have no clue. that being said, consider the following simple program program test_mpi_sizeof implicit none include 'mpif.h' integer i double precision d integer sze,ierr call MPI_Sizeof(i, sze, ierr) write (*,*) 'MPI_Sizeof(integer) = ', sze call MPI_Sizeof(d, sze, ierr) write (*,*) 'MPI_Sizeof(double precision) = ', sze stop end program on one hand, Open MPI (built with intel compiler) - build successfully (compile & link) - produces the correct output on the other hand, Intel MPI fails to link (MPI_Sizeof is undefined) the best i can do is dodge the question and point to the MPI Forum http://mpi-forum.org/docs/mpi-3.1/mpi31-report/node411.htm : "The use of the mpif.h include file is strongly discouraged and may be deprecated in a future version of MPI." as far as i understand, mpif.h was meant to be used with F77 programs or obsolete compilers that do not support modules among other things. /* MPI_Sizeof requires compiler support for module and a few extra F90 goodies */ the piece of code you posted is a Fortran module, so i do not see any rationale for using "INCLUDE 'mpif.h'" over "USE mpi" (or even better, "USE mpi_f08") Cheers, Gilles On 11/21/2016 11:55 PM, Paul Kapinos wrote: Dear Open MPI developer, QE is an MPI program, cf. http://qe-forge.org/gf/download/frsrelease/224/1044/qe-6.0.tar.gz In FFTXlib a questionable source code part is contained which cannot be compiled using version 17 of the Intel compilers and Open MPI; I've condensed it (see attachment). Note that - using Intel MPI all tested compilers (Intel/GCC/Oracle Studio/PGI) can compile stick_base_TEST.f90 with no errors/warnings issued - using Open MPI the same is true for GCC/Oracle Studio/PGI; - using Intel compiler up to version 16 and Open MPI, *warnings* like (1) are issued (compilation succeds); - using Intel compiler /17, *errors* like (2) are issued (compilation fails). - when 'PRIVATE' statement in line #3 removed, also Intel compiler compile the snippet with no error. Well the questions is - is stick_base_TEST.f90 a correct MPI code? - if YES why does the Intel/17 + OpenMPI combination do not like this? If NO why to hell none of other compiler+MPI combinations complain about this? :o) Have a nice day, Paul Kapinos P.S. Did you noticed also this one? https://www.mail-archive.com/users@lists.open-mpi.org//msg30320.html - 1 -- /opt/MPI/openmpi-1.10.4/linux/intel_16.0.2.181/include/mpif-sizeof.h(2254): warning #6738: The type/rank/keyword signature for this specific procedure matches another specific procedure that shares the same generic-name. [PMPI_SIZEOF_REAL64_R15] SUBROUTINE PMPI_Sizeof_real64_r15(x, size, ierror) -^ - 2 -- /opt/MPI/openmpi-1.10.4/linux/intel_17.0.0.064/include/mpif-sizeof.h(220): error #5286: Ambiguous generic interface MPI_SIZEOF: previously declared specific procedure MPI_SIZEOF_COMPLEX32_R13 is not distinguishable from this declaration. [MPI_SIZEOF_COMPLEX32_R13] SUBROUTINE MPI_Sizeof_complex32_r13(x, size, ierror) -^ ___ devel mailing list devel@lists.open-mpi.org https://rfd.newmexicoconsortium.org/mailman/listinfo/devel ___ devel mailing list devel@lists.open-mpi.org https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
Re: [OMPI devel] [OMPI users] funny SIGSEGV in 'ompi_info'
Paul,
SIGSEGV is always a bad idea, even after having displayed a
comprehensive and user friendly error message
--
MCA framework parameters can only take a single negation operator
("^"), and it must be at the beginning of the value. The following
value violates this rule:
^tcp,^ib
When used, the negation operator sets the "exclusive" behavior mode,
meaning that it will exclude all specified components (and implicitly
include all others). If the negation operator is not specified, the
"inclusive" mode is assumed, meaning that all specified components
will be included (and implicitly exclude all others).
For example, "^a,b" specifies the exclusive behavior and means "use
all components *except* a and b", while "c,d" specifies the inclusive
behavior and means "use *only* components c and d."
You cannot mix inclusive and exclusive behavior.
--
that raises the question, what should we do when we run into this case ?
- one option is to propagate the error (currently, functions do not
return anything) (and do what after ?)
- an other option is to brutally exit(1)
- yet an other option is to disregard the incorrect value of the
parameter and continue
any thoughts anyone ?
Cheers,
Gilles
On 11/14/2016 9:28 PM, Paul Kapinos wrote:
Dear developers,
also the following issue is defintely raised by a misconfiguration of
Open MPI, SIGSEGV's in 'ompi_info' isn'n a good thing, thus this one
mail.
Just call:
$ export OMPI_MCA_mtl="^tcp,^ib"
$ ompi_info --param all all --level 9
... and take a look at the below core dump of 'ompi_info' like below one.
(yes we know that "^tcp,^ib" is a bad idea).
Have a nice day,
Paul Kapinos
P.S. Open MPI: 1.10.4 and 2.0.1 have the same behaviour
--
[lnm001:39957] *** Process received signal ***
[lnm001:39957] Signal: Segmentation fault (11)
[lnm001:39957] Signal code: Address not mapped (1)
[lnm001:39957] Failing at address: (nil)
[lnm001:39957] [ 0] /lib64/libpthread.so.0(+0xf100)[0x2b30f1a79100]
[lnm001:39957] [ 1]
/opt/MPI/openmpi-1.10.4/linux/intel_16.0.2.181/lib/libopen-pal.so.13(+0x2f11f)[0x2b30f084911f]
[lnm001:39957] [ 2]
/opt/MPI/openmpi-1.10.4/linux/intel_16.0.2.181/lib/libopen-pal.so.13(+0x2f265)[0x2b30f0849265]
[lnm001:39957] [ 3]
/opt/MPI/openmpi-1.10.4/linux/intel_16.0.2.181/lib/libopen-pal.so.13(opal_info_show_mca_params+0x91)[0x2b30f0849031]
[lnm001:39957] [ 4]
/opt/MPI/openmpi-1.10.4/linux/intel_16.0.2.181/lib/libopen-pal.so.13(opal_info_do_params+0x1f4)[0x2b30f0848e84]
[lnm001:39957] [ 5] ompi_info[0x402643]
[lnm001:39957] [ 6]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x2b30f1ca7b15]
[lnm001:39957] [ 7] ompi_info[0x4022a9]
[lnm001:39957] *** End of error message ***
zsh: segmentation fault (core dumped) ompi_info --param all all
--level 9
--
___
users mailing list
us...@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
Re: [OMPI devel] MPI_Win_lock semantic
Thanks George,
i understand the change you suggest will fix the test.
tasks will no more fight for the same lock, and i believe that was the
initial intent of the test
(e.g. will task 2 wait for task 0 to release the lock before MPI_Put() data)
but since there is no other way to test this, i will go ahead and update
the test
Cheers,
Gilles
On 11/22/2016 1:13 PM, George Bosilca wrote:
Gilles,
I looked at the test and I think the current behavior is indeed
correct. What matters for an exclusive lock is that all operations in
an epoch (everything surrounded by lock/unlock) are atomically applied
to the destination (and are not interleaved with other updates). As
Nathan stated, MPI_Win_lock might be implemented as non-blocking, in
which case it is totally legit for the process 2 to acquire the lock
first, and update the array before the process 0 access it. Thus the
test will fail.
The test will never deadlock, because even if the MPI_Win_lock is
implemented as a blocking operation (which is also legit), the send
and receive match correctly with the lock/unlock.
Moreover, I think the behavior described by the comments can only be
implemented by enforcing an order between the only conceptually
meaningful operations unlock/send/recv.
if (me == 0) {
MPI_Win_lock(MPI_LOCK_EXCLUSIVE, 1, 0, win);
MPI_Get(a,len,MPI_DOUBLE,1,0,len,MPI_DOUBLE,win);
MPI_Win_unlock(1, win);
MPI_Send(NULL, 0, MPI_BYTE, 2, 1001, MPI_COMM_WORLD);
}
if (me == 2) {
/* this should block till 0 releases the lock. */
MPI_Recv(NULL, 0, MPI_BYTE, 0, 1001, MPI_COMM_WORLD,
MPI_STATUS_IGNORE);
MPI_Win_lock(MPI_LOCK_EXCLUSIVE, 1, 0, win);
MPI_Put(a,len,MPI_DOUBLE,1,0,len,MPI_DOUBLE,win);
MPI_Win_unlock(1, win);
}
However, if we relax the code a little and want to ensure the
atomicity of the operations, then we need to change the check to make
sure that either no elements of the array have been altered or all of
them have been altered (set to zero by process 2).
George.
On Mon, Nov 21, 2016 at 8:57 PM, Nathan Hjelm > wrote:
To be safe I would call MPI_Get then MPI_Win_flush. That lock will
always be acquired before the MPI_Win_flush call returns. As long
as it is more than 0 bytes. We always short-circuit 0-byte
operations in both osc/rdma and osc/pt2pt.
-Nathan
> On Nov 21, 2016, at 8:54 PM, Gilles Gouaillardet
mailto:gil...@rist.or.jp>> wrote:
>
> Thanks Nathan,
>
>
> any thoughts about my modified version of the test ?
>
> do i need to MPI_Win_flush() after the first MPI_Get() in order
to ensure the lock was acquired ?
>
> (and hence the program will either success or hang, but never fail)
>
>
> Cheers,
>
>
> Gilles
>
>
> On 11/22/2016 12:29 PM, Nathan Hjelm wrote:
>> MPI_Win_lock does not have to be blocking. In osc/rdma it is
blocking in most cases but not others (lock all with on-demand is
non-blocking) but in osc/pt2pt is is almost always non-blocking
(it has to be blocking for proc self). If you really want to
ensure the lock is acquired you can call MPI_Win_flush. I think
this should work even if you have not started any RMA operations
inside the epoch.
>>
>> -Nathan
>>
>>> On Nov 21, 2016, at 7:53 PM, Gilles Gouaillardet
mailto:gil...@rist.or.jp>> wrote:
>>>
>>> Nathan,
>>>
>>>
>>> we briefly discussed the test_lock1 test from the onesided
test suite using osc/pt2pt
>>>
>>>
https://github.com/open-mpi/ompi-tests/blob/master/onesided/test_lock1.c#L57-L70
>>>
>>>
>>> task 0 does
>>>
>>> MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
>>>
>>> MPI_Send(...,dest=2,...)
>>>
>>>
>>> and task 2 does
>>>
>>> MPI_Win_lock(MPI_LOCK_EXCLUSIVE, rank=1,...);
>>>
>>> MPI_Recv(...,source=0,...)
>>>
>>>
>>> hoping to guarantee task 0 will acquire the lock first.
>>>
>>>
>>> once in a while, the test fails when task 2 acquires the lock
first
>>>
>>> /* MPI_Win_lock() only sends a lock request, and return
without owning the lock */
>>>
>>> so if task 1 is running on a loaded server, and even if task 2
requests the lock *after* task 0,
>>>
>>> lock request from task 2 can be processed first, and hence
task 2 is not guaranteed to acquire the lock *before* task 0.
>>>
>>>
>>> can you please confirm MPI_Win_lock() behaves as it is
supposed to ?
>>>
>>> if yes, is there a way for task 0 to block until it acquires
the lock ?
>>>
>>>
>>> i modified the test, and inserted in task 0 a MPI_Get of 1
MPI_Double *before* MPI_Send.
>>>
>>> see my patch below (note i increased the message length)
>>>
>>>
>>> my e
