Re: [OMPI users] Calculate time spent on non blocking communication?

[Eugene Loh]

Okay, so forget about Peruse.

You can basically figure that your user process will either be inside
an MPI call or else not.  If it's inside an MPI call, then that's time
spent in communications (and notably in the synchronization that's
implicit to communication).  If it's not inside an MPI call, then
that's time spent in computation.  Basically, no time in this model is
attributed to both communication and computation at once.

There is an OMPI FAQ on performance tools. 
http://www.open-mpi.org/faq/?category=perftools  Perhaps something
there will be helpful for you.  Specifically, the "Sun Studio
Performance Analyzer" allows you to divide that "communication" time
between "data transfer time" and "synchronization time".  But a basic
classification as either communication or else computation is pretty
central to all the tools.

Bibrak Qamar wrote:

  As asked the reason of such calculation of non
blocking communication, the main reason is that I want to look into the
program as how much it percent time is consumed on communication alone,
computation alone and the intersection of both.
  
  On Thu, Feb 3, 2011 at 5:08 AM, Eugene Loh 
wrote:
  Again,
you can try the Peruse instrumentation.  Configure OMPI with
--enable-peruse.  The instrumentation points might help you decide how
you want to define the time you want to measure.  Again, you really
have to spend a bunch of your own time deciding what is meaningful to
measure.

Gustavo Correa wrote:


  However, OpenMPI may give this info, with non-MPI
(hence non-portable) functions, I'd guess.
  
  
  
From: Eugene Loh 


Anyhow, the Peruse instrumentation in OMPI
might help.

  

  
  
  

Re: [OMPI users] Calculate time spent on non blocking communication?

[Bibrak Qamar]

Thanks all,

As asked the reason of such calculation of non blocking communication, the
main reason is that I want to look into the program as how much it percent
time is consumed on communication alone, computation alone and the
intersection of both.

Bibrak Qamar
Undergraduate Student BIT-9
Member Center for High Performance Scientific Computing
NUST-School of Electrical Engineering and Computer Science.


On Thu, Feb 3, 2011 at 5:08 AM, Eugene Loh  wrote:

> Again, you can try the Peruse instrumentation.  Configure OMPI with
> --enable-peruse.  The instrumentation points might help you decide how you
> want to define the time you want to measure.  Again, you really have to
> spend a bunch of your own time deciding what is meaningful to measure.
>
> Gustavo Correa wrote:
>
>  However, OpenMPI may give this info, with non-MPI (hence non-portable)
>> functions, I'd guess.
>>
>>  From: Eugene Loh 
>>>
>>> Anyhow, the Peruse instrumentation in OMPI might help.
>>>
>>>
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>

Re: [OMPI users] Calculate time spent on non blocking communication?

[Eugene Loh]

Again, you can try the Peruse instrumentation.  Configure OMPI with 
--enable-peruse.  The instrumentation points might help you decide how 
you want to define the time you want to measure.  Again, you really have 
to spend a bunch of your own time deciding what is meaningful to measure.


Gustavo Correa wrote:

However, OpenMPI may give this info, with non-MPI (hence non-portable) 
functions, I'd guess.



From: Eugene Loh 

Anyhow, the Peruse instrumentation in OMPI might help.

Re: [OMPI users] Calculate time spent on non blocking communication?

[Gustavo Correa]

On Feb 2, 2011, at 7:35 AM, Bibrak Qamar wrote:

> Gus Correa, But it will include the time of computation which took place 
> before waitAll( ).
> 
Correct.
Now I realize you want the communication time only,
not the overall time.

Even to define precisely what this means may be a bit tricky.
I guess Eugene Loh was hinting this difficulty.

Is it the time MPI_Isend takes to return?
It returns as soon as the send is posted, I think,
but this doesn't mean that the message was received at the other end,
or that you are free to use the send buffer either.
You could do it with 
start=MPI_Wtime()
call MPI_Isend(...)
end=MPI_Wtime()
but this may not be very useful.

Is it the time until MPI_[I]recv returns?
Well, that one could be measured the way I suggested,
but it will probably contain also time used to compute.

Is it something else, perhaps?

My guess is that it may be hard to nail down communication time alone, 
particularly in non-blocking case,
at least if you try to get it only with MPI calls.
Non-blocking is really designed to overlap communication and computation,
not to separate them.
Also, I guess MPI itself doesn't give user access to the details of *how* it 
provides communication, timing information included.
However, OpenMPI may give this info, with non-MPI (hence non-portable) 
functions, I'd guess.

Have you tried inserting MPI_Test calls as Jeff suggested? 
(Perhaps along with MPI_Wtime)

Maybe there is more timing information available outside MPI,
at another  OpenMPI internal level.
Perhaps this is accessible from  within the program, via non-MPI calls,
but I don't really know.
Maybe the OpenMPI developers have a hint.
Have you read Eugene Loh's suggestions?

Now I am asking more questions than answering anything. :)

Gus Correa
> 
> Date: Tue, 1 Feb 2011 10:09:03 +0400
> From: Bibrak Qamar <bibr...@gmail.com>
> Subject: [OMPI users] Calculate time spent on non blocking
>communication?
> To: us...@open-mpi.org
> Message-ID:
><aanlktinewz_xyx88pgouojvogaf6ld8nwf_nchsy0...@mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> Hello All,
> 
> I am using non-blocking send and receive, and i want to calculate the time
> it took for the communication. Is there any method or a way to do this using
> openmpi.
> 
> Thanks
> Bibrak Qamar
> Undergraduate Student BIT-9
> Member Center for High Performance Scientific Computing
> NUST-School of Electrical Engineering and Computer Science.
> -- next part --
> HTML attachment scrubbed and removed
> 
> --
> 
> Message: 4
> Date: Mon, 31 Jan 2011 22:14:53 -0800
> From: Eugene Loh <eugene@oracle.com>
> Subject: Re: [OMPI users] Calculate time spent on non blocking
>communication?
> To: Open MPI Users <us...@open-mpi.org>
> Message-ID: <4d47a4dd.5010...@oracle.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> 
> Bibrak Qamar wrote:
> 
> > Hello All,
> >
> > I am using non-blocking send and receive, and i want to calculate the
> > time it took for the communication. Is there any method or a way to do
> > this using openmpi.
> 
> You probably have to start by defining what you mean by "the time it
> took for the communication".  Anyhow, the Peruse instrumentation in OMPI
> might help.
> 
> 
> --
> 
> Message: 5
> Date: Tue, 1 Feb 2011 01:20:36 -0500
> From: Gustavo Correa <g...@ldeo.columbia.edu>
> Subject: Re: [OMPI users] Calculate time spent on non blocking
>communication?
> To: Open MPI Users <us...@open-mpi.org>
> Message-ID: <8f16054c-6fca-4e65-9c83-5efbfcb18...@ldeo.columbia.edu
> >
> Content-Type: text/plain; charset=us-ascii
> 
> 
> On Feb 1, 2011, at 1:09 AM, Bibrak Qamar wrote:
> 
> 
> 
> 
> > Hello All,
> >
> > I am using non-blocking send and receive, and i want to calculate the time 
> > it took for the communication. Is there any method or a way to do this 
> > using openmpi.
> >
> > Thanks
> > Bibrak Qamar
> > Undergraduate Student BIT-9
> > Member Center for High Performance Scientific Computing
> > NUST-School of Electrical Engineering and Computer Science.
> > ___
> 
> About the same as with blocking communication, I guess.
> 
> Would this do work for you?
> 
> start=MPI_Wtime()
> MPI_Isend(...)
> ...
> MPI_Irecv(...)
> ...
> MPI_Wait[all](...)
> end=MPI_Wtime()
> print *, 'walltime = ', end-start
> 
> My two cents,
> Gus Correa
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] Calculate time spent on non blocking communication?

[Eugene Loh]

Bibrak Qamar wrote:

  Gus Correa, But it will include the time of
computation which took place before waitAll( ).
  

What's wrong with that?

  From: Bibrak Qamar 
I am using non-blocking send and receive, and i want to calculate the
time
it took for the communication.
  
From: Eugene Loh 
You probably have to start by defining what you mean by "the time it
took for the communication".  Anyhow, the Peruse instrumentation in OMPI
might help.
  

Again, you should probably start by thinking more precisely about what
time you want to measure.  E.g., ask yourself why the answer even
matters.

Re: [OMPI users] Calculate time spent on non blocking communication?

[Bibrak Qamar]

Gus Correa, But it will include the time of computation which took place
before waitAll( ).


List-Post: users@lists.open-mpi.org
Date: Tue, 1 Feb 2011 10:09:03 +0400
From: Bibrak Qamar <bibr...@gmail.com>
Subject: [OMPI users] Calculate time spent on non blocking
   communication?
To: us...@open-mpi.org
Message-ID:
   <aanlktinewz_xyx88pgouojvogaf6ld8nwf_nchsy0...@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Hello All,

I am using non-blocking send and receive, and i want to calculate the time
it took for the communication. Is there any method or a way to do this using
openmpi.

Thanks
Bibrak Qamar
Undergraduate Student BIT-9
Member Center for High Performance Scientific Computing
NUST-School of Electrical Engineering and Computer Science.
-- next part --
HTML attachment scrubbed and removed

--

Message: 4
List-Post: users@lists.open-mpi.org
Date: Mon, 31 Jan 2011 22:14:53 -0800
From: Eugene Loh <eugene@oracle.com>
Subject: Re: [OMPI users] Calculate time spent on non blocking
   communication?
To: Open MPI Users <us...@open-mpi.org>
Message-ID: <4d47a4dd.5010...@oracle.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Bibrak Qamar wrote:

> Hello All,
>
> I am using non-blocking send and receive, and i want to calculate the
> time it took for the communication. Is there any method or a way to do
> this using openmpi.

You probably have to start by defining what you mean by "the time it
took for the communication".  Anyhow, the Peruse instrumentation in OMPI
might help.


--

Message: 5
List-Post: users@lists.open-mpi.org
Date: Tue, 1 Feb 2011 01:20:36 -0500
From: Gustavo Correa <g...@ldeo.columbia.edu>
Subject: Re: [OMPI users] Calculate time spent on non blocking
   communication?
To: Open MPI Users <us...@open-mpi.org>
Message-ID: <8f16054c-6fca-4e65-9c83-5efbfcb18...@ldeo.columbia.edu
>
Content-Type: text/plain; charset=us-ascii


On Feb 1, 2011, at 1:09 AM, Bibrak Qamar wrote:




> Hello All,
>
> I am using non-blocking send and receive, and i want to calculate the time
it took for the communication. Is there any method or a way to do this using
openmpi.
>
> Thanks
> Bibrak Qamar
> Undergraduate Student BIT-9
> Member Center for High Performance Scientific Computing
> NUST-School of Electrical Engineering and Computer Science.
> ___

About the same as with blocking communication, I guess.

Would this do work for you?

start=MPI_Wtime()
MPI_Isend(...)
...
MPI_Irecv(...)
...
MPI_Wait[all](...)
end=MPI_Wtime()
print *, 'walltime = ', end-start

My two cents,
Gus Correa

Re: [OMPI users] Calculate time spent on non blocking communication?

[Gustavo Correa]

On Feb 1, 2011, at 1:09 AM, Bibrak Qamar wrote:

> Hello All,
> 
> I am using non-blocking send and receive, and i want to calculate the time it 
> took for the communication. Is there any method or a way to do this using 
> openmpi.
> 
> Thanks
> Bibrak Qamar
> Undergraduate Student BIT-9
> Member Center for High Performance Scientific Computing
> NUST-School of Electrical Engineering and Computer Science.
> ___

About the same as with blocking communication, I guess.

Would this do work for you?

start=MPI_Wtime()
MPI_Isend(...)
...
MPI_Irecv(...)
...
MPI_Wait[all](...)
end=MPI_Wtime()
print *, 'walltime = ', end-start

My two cents,
Gus Correa

[OMPI users] Calculate time spent on non blocking communication?

[Bibrak Qamar]

Hello All,

I am using non-blocking send and receive, and i want to calculate the time
it took for the communication. Is there any method or a way to do this using
openmpi.

Thanks
Bibrak Qamar
Undergraduate Student BIT-9
Member Center for High Performance Scientific Computing
NUST-School of Electrical Engineering and Computer Science.