Re: [O-MPI devel] Memory registration question.

2005-08-16 Thread Gleb Natapov
On Tue, Aug 16, 2005 at 10:47:52AM -0600, Tim S. Woodall wrote: > We're working on an approach to this now, that I think will address what > you are looking for. Will let you know when we have something a little more > concrete. > I have the initial code that does what I've described. Do you want

Re: [O-MPI devel] Memory registration question.

2005-08-16 Thread Tim S. Woodall
Hello Gleb, After thinking about this a bit more - I agree w/ your approach. We're going to look at moving the cache/mru list into each mpool, and maintaining the registrations on a per btl basis. This should actually clean up a lot of code. We're working on an approach to this now, that I think

Re: [O-MPI devel] Memory registration question.

2005-08-15 Thread Jeff Squyres
On Aug 15, 2005, at 8:37 AM, Gleb Natapov wrote: This is actually an erroneous MPI program. MPI specifically states that a buffer can be used for exactly one communication at a time. This is actually the way MPI_Bcast works right now IMHO :) Shhh!! *Users* aren't allowed to do this. But w

Re: [O-MPI devel] Memory registration question.

2005-08-15 Thread Gleb Natapov
On Mon, Aug 15, 2005 at 08:09:16AM -0400, Jeff Squyres wrote: > On Aug 14, 2005, at 8:53 AM, Gleb Natapov wrote: > > > Lets say application wants to send buffer B to 100 other ranks. It does > > 100 MPI_Isend (B). > > This is actually an erroneous MPI program. MPI specifically states > that a b

Re: [O-MPI devel] Memory registration question.

2005-08-15 Thread Jeff Squyres
On Aug 14, 2005, at 8:53 AM, Gleb Natapov wrote: Lets say application wants to send buffer B to 100 other ranks. It does 100 MPI_Isend (B). This is actually an erroneous MPI program. MPI specifically states that a buffer can be used for exactly one communication at a time. I don't know if

Re: [O-MPI devel] Memory registration question.

2005-08-14 Thread Gleb Natapov
Hello Tim, On Thu, Aug 11, 2005 at 10:08:04AM -0600, Tim S. Woodall wrote: > Hello Gleb, > > A couple of general comments: > > We initially started by maintaining the cache at the btl/mpool level. However, > we needed to expose the registrations to the upper level (pml), to > allow the pml to ma

Re: [O-MPI devel] Memory registration question.

2005-08-11 Thread Tim S. Woodall
Hello Gleb, A couple of general comments: We initially started by maintaining the cache at the btl/mpool level. However, we needed to expose the registrations to the upper level (pml), to allow the pml to make scheduling decisions (which btl/protocol to use), so we re-organized this to maintain

Re: [O-MPI devel] Memory registration question.

2005-08-10 Thread Gleb Natapov
Hello Tim, On Tue, Aug 09, 2005 at 10:22:34AM -0600, Timothy B. Prins wrote: > If you have anyother ideas of how to do it please let us know. > > I have to confess I don't like current pindown cache implementation much or perhaps I don't understand it enough. What I managed to understand from

Re: [O-MPI devel] Memory registration question.

2005-08-09 Thread Timothy B. Prins
Gleb, We just talked about this problem and we decided that we needed to support overlapping registrations. Our idea is to add another funtion to the mpool that would return a list of registrations that correspond to a base address. If you have anyother ideas of how to do it please let us know.

Re: [O-MPI devel] Memory registration question.

2005-08-09 Thread Gleb Natapov
On Tue, Aug 09, 2005 at 08:20:38AM -0600, Timothy B. Prins wrote: > Gleb. > > The memory pool does not support overlapping registrations. The > registrations are stored in a balanced tree, so which ever of the two it > encounters first it will return. This was my impression. Is this inefficient? I

Re: [O-MPI devel] Memory registration question.

2005-08-09 Thread Timothy B. Prins
Gleb. The memory pool does not support overlapping registrations. The registrations are stored in a balanced tree, so which ever of the two it encounters first it will return. Tim Prins > Hello, > > I am trying to understand how memory registration works in openMPI and > I have a question. Do

[O-MPI devel] Memory registration question.

2005-08-09 Thread Gleb Natapov
Hello, I am trying to understand how memory registration works in openMPI and I have a question. Does mca_mpool_base_(insert|insert) interface supports overlapping registrations? If one module register memory from 0 to 100 and another from 50 to 150 what mca_mpool_base_find(80) will return to ob