Re: [OMPI devel] non-blocking barrier

[Richard Graham]

Forget what I just posted - I looked at George's words, and not the code - 
wait() is the synchronization point, so George's response is correct.

Rich

-Original Message-
From: devel-boun...@open-mpi.org [mailto:devel-boun...@open-mpi.org] On Behalf 
Of George Bosilca
Sent: Friday, July 06, 2012 2:48 AM
To: eugene@oracle.com; Open MPI Developers
Cc: Open MPI Developers
Subject: Re: [OMPI devel] non-blocking barrier

No, it is not right. With the ibarrier usage you're making below, the output 
should be similar to the first case (all should leave at earlist at 6.0). The 
ibarrier is still a synchronizing point, all processes MUST reach it before 
anyone is allowed to leave.

However, if you move the ibarrier on proc < 2 before the sleep, the output you 
got become possible.

  George



On Jul 6, 2012, at 7:53, Eugene Loh <eugene@oracle.com> wrote:

> Either there is a problem with MPI_Ibarrier or I don't understand the 
> semantics.
> 
> The following example is with openmpi-1.9a1r26747.  (Thanks for the fix in 
> 26757.  I tried with that as well with same results.)  I get similar results 
> for different OSes, compilers, bitness, etc.
> 
> % cat ibarrier.c
> #include 
> #include 
> #include 
> #include 
> 
> int main(int argc, char** argv) {
>int i, me;
>double t0, t1, t2;
>MPI_Request req;
> 
>MPI_Init(,);
>MPI_Comm_rank(MPI_COMM_WORLD,);
> 
>MPI_Barrier(MPI_COMM_WORLD);
>MPI_Barrier(MPI_COMM_WORLD);
>MPI_Barrier(MPI_COMM_WORLD);
>t0 = MPI_Wtime();  /* set "time zero" */
> 
>if ( me < 2 ) sleep(3);/* two processes delay before hitting 
> barrier */
>t1 = MPI_Wtime() - t0;
>MPI_Barrier(MPI_COMM_WORLD);
>t2 = MPI_Wtime() - t0;
>printf("%d entered at %3.1lf and exited at %3.1lf\n", me, t1, t2);
> 
>if ( me < 2 ) sleep(3);/* two processes delay before hitting 
> barrier */
>t1 = MPI_Wtime() - t0;
>MPI_Ibarrier(MPI_COMM_WORLD, );
>MPI_Wait(, MPI_STATUS_IGNORE);
>t2 = MPI_Wtime() - t0;
>printf("%d entered at %3.1lf and exited at %3.1lf\n", me, t1, t2);
> 
>MPI_Finalize();
>return 0;
> }
> % mpirun -n 4 ./a.out
> 0 entered at 3.0 and exited at 3.0
> 1 entered at 3.0 and exited at 3.0
> 2 entered at 0.0 and exited at 3.0
> 3 entered at 0.0 and exited at 3.0
> 0 entered at 6.0 and exited at 6.0
> 1 entered at 6.0 and exited at 6.0
> 2 entered at 3.0 and exited at 3.0
> 3 entered at 3.0 and exited at 3.0
> 
> With the first barrier, no one leaves until the last process has entered.  
> With the non-blocking barrier, two processes enter and leave before the two 
> laggards arrive at the barrier.  Is that right?
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] non-blocking barrier

[Richard Graham]

Don't agree here - the only synchronization point is the completion.   Ibarrier 
can't be completed until all have entered the barrier, but each process can 
leave the ibarrier() call as soon as they want to.

Rich

-Original Message-
From: devel-boun...@open-mpi.org [mailto:devel-boun...@open-mpi.org] On Behalf 
Of George Bosilca
Sent: Friday, July 06, 2012 2:48 AM
To: eugene@oracle.com; Open MPI Developers
Cc: Open MPI Developers
Subject: Re: [OMPI devel] non-blocking barrier

No, it is not right. With the ibarrier usage you're making below, the output 
should be similar to the first case (all should leave at earlist at 6.0). The 
ibarrier is still a synchronizing point, all processes MUST reach it before 
anyone is allowed to leave.

However, if you move the ibarrier on proc < 2 before the sleep, the output you 
got become possible.

  George



On Jul 6, 2012, at 7:53, Eugene Loh <eugene@oracle.com> wrote:

> Either there is a problem with MPI_Ibarrier or I don't understand the 
> semantics.
> 
> The following example is with openmpi-1.9a1r26747.  (Thanks for the fix in 
> 26757.  I tried with that as well with same results.)  I get similar results 
> for different OSes, compilers, bitness, etc.
> 
> % cat ibarrier.c
> #include 
> #include 
> #include 
> #include 
> 
> int main(int argc, char** argv) {
>int i, me;
>double t0, t1, t2;
>MPI_Request req;
> 
>MPI_Init(,);
>MPI_Comm_rank(MPI_COMM_WORLD,);
> 
>MPI_Barrier(MPI_COMM_WORLD);
>MPI_Barrier(MPI_COMM_WORLD);
>MPI_Barrier(MPI_COMM_WORLD);
>t0 = MPI_Wtime();  /* set "time zero" */
> 
>if ( me < 2 ) sleep(3);/* two processes delay before hitting 
> barrier */
>t1 = MPI_Wtime() - t0;
>MPI_Barrier(MPI_COMM_WORLD);
>t2 = MPI_Wtime() - t0;
>printf("%d entered at %3.1lf and exited at %3.1lf\n", me, t1, t2);
> 
>if ( me < 2 ) sleep(3);/* two processes delay before hitting 
> barrier */
>t1 = MPI_Wtime() - t0;
>MPI_Ibarrier(MPI_COMM_WORLD, );
>MPI_Wait(, MPI_STATUS_IGNORE);
>t2 = MPI_Wtime() - t0;
>printf("%d entered at %3.1lf and exited at %3.1lf\n", me, t1, t2);
> 
>MPI_Finalize();
>return 0;
> }
> % mpirun -n 4 ./a.out
> 0 entered at 3.0 and exited at 3.0
> 1 entered at 3.0 and exited at 3.0
> 2 entered at 0.0 and exited at 3.0
> 3 entered at 0.0 and exited at 3.0
> 0 entered at 6.0 and exited at 6.0
> 1 entered at 6.0 and exited at 6.0
> 2 entered at 3.0 and exited at 3.0
> 3 entered at 3.0 and exited at 3.0
> 
> With the first barrier, no one leaves until the last process has entered.  
> With the non-blocking barrier, two processes enter and leave before the two 
> laggards arrive at the barrier.  Is that right?
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] non-blocking barrier

[George Bosilca]

No, it is not right. With the ibarrier usage you're making below, the output 
should be similar to the first case (all should leave at earlist at 6.0). The 
ibarrier is still a synchronizing point, all processes MUST reach it before 
anyone is allowed to leave.

However, if you move the ibarrier on proc < 2 before the sleep, the output you 
got become possible.

  George



On Jul 6, 2012, at 7:53, Eugene Loh  wrote:

> Either there is a problem with MPI_Ibarrier or I don't understand the 
> semantics.
> 
> The following example is with openmpi-1.9a1r26747.  (Thanks for the fix in 
> 26757.  I tried with that as well with same results.)  I get similar results 
> for different OSes, compilers, bitness, etc.
> 
> % cat ibarrier.c
> #include 
> #include 
> #include 
> #include 
> 
> int main(int argc, char** argv) {
>int i, me;
>double t0, t1, t2;
>MPI_Request req;
> 
>MPI_Init(,);
>MPI_Comm_rank(MPI_COMM_WORLD,);
> 
>MPI_Barrier(MPI_COMM_WORLD);
>MPI_Barrier(MPI_COMM_WORLD);
>MPI_Barrier(MPI_COMM_WORLD);
>t0 = MPI_Wtime();  /* set "time zero" */
> 
>if ( me < 2 ) sleep(3);/* two processes delay before hitting 
> barrier */
>t1 = MPI_Wtime() - t0;
>MPI_Barrier(MPI_COMM_WORLD);
>t2 = MPI_Wtime() - t0;
>printf("%d entered at %3.1lf and exited at %3.1lf\n", me, t1, t2);
> 
>if ( me < 2 ) sleep(3);/* two processes delay before hitting 
> barrier */
>t1 = MPI_Wtime() - t0;
>MPI_Ibarrier(MPI_COMM_WORLD, );
>MPI_Wait(, MPI_STATUS_IGNORE);
>t2 = MPI_Wtime() - t0;
>printf("%d entered at %3.1lf and exited at %3.1lf\n", me, t1, t2);
> 
>MPI_Finalize();
>return 0;
> }
> % mpirun -n 4 ./a.out
> 0 entered at 3.0 and exited at 3.0
> 1 entered at 3.0 and exited at 3.0
> 2 entered at 0.0 and exited at 3.0
> 3 entered at 0.0 and exited at 3.0
> 0 entered at 6.0 and exited at 6.0
> 1 entered at 6.0 and exited at 6.0
> 2 entered at 3.0 and exited at 3.0
> 3 entered at 3.0 and exited at 3.0
> 
> With the first barrier, no one leaves until the last process has entered.  
> With the non-blocking barrier, two processes enter and leave before the two 
> laggards arrive at the barrier.  Is that right?
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

[OMPI devel] non-blocking barrier

[Eugene Loh]

Either there is a problem with MPI_Ibarrier or I don't understand the 
semantics.


The following example is with openmpi-1.9a1r26747.  (Thanks for the fix 
in 26757.  I tried with that as well with same results.)  I get similar 
results for different OSes, compilers, bitness, etc.


% cat ibarrier.c
#include 
#include 
#include 
#include 

int main(int argc, char** argv) {
int i, me;
double t0, t1, t2;
MPI_Request req;

MPI_Init(,);
MPI_Comm_rank(MPI_COMM_WORLD,);

MPI_Barrier(MPI_COMM_WORLD);
MPI_Barrier(MPI_COMM_WORLD);
MPI_Barrier(MPI_COMM_WORLD);
t0 = MPI_Wtime();  /* set "time zero" */

if ( me < 2 ) sleep(3);/* two processes delay before 
hitting barrier */

t1 = MPI_Wtime() - t0;
MPI_Barrier(MPI_COMM_WORLD);
t2 = MPI_Wtime() - t0;
printf("%d entered at %3.1lf and exited at %3.1lf\n", me, t1, t2);

if ( me < 2 ) sleep(3);/* two processes delay before 
hitting barrier */

t1 = MPI_Wtime() - t0;
MPI_Ibarrier(MPI_COMM_WORLD, );
MPI_Wait(, MPI_STATUS_IGNORE);
t2 = MPI_Wtime() - t0;
printf("%d entered at %3.1lf and exited at %3.1lf\n", me, t1, t2);

MPI_Finalize();
return 0;
}
% mpirun -n 4 ./a.out
0 entered at 3.0 and exited at 3.0
1 entered at 3.0 and exited at 3.0
2 entered at 0.0 and exited at 3.0
3 entered at 0.0 and exited at 3.0
0 entered at 6.0 and exited at 6.0
1 entered at 6.0 and exited at 6.0
2 entered at 3.0 and exited at 3.0
3 entered at 3.0 and exited at 3.0

With the first barrier, no one leaves until the last process has 
entered.  With the non-blocking barrier, two processes enter and leave 
before the two laggards arrive at the barrier.  Is that right?