date:20120622

Re: [OMPI devel] RFC: Pineapple Runtime Interposition Project

2012-06-22 Thread Josh Hursey

The pineapple branch is ready for testing. It was last sync'ed to the
trunk at r26620. I'll try to update that later today, and keep current
with the trunk moving forward.

I created a wiki page to discuss some of the build options, for those
interested. At the bottom of the page are some remaining todo items
that I will probably need some help resolving.
  https://svn.open-mpi.org/trac/ompi/wiki/Runtime_Interposition

The branch is at:
  https://bitbucket.org/jjhursey/ompi-pineapple

I would like to bring this into the trunk the evening after the
teleconf on Tuesday, June 26, 2012. So if you have cycles to test this
branch I would appreciate it.

Thanks,
Josh

On Fri, Jun 15, 2012 at 2:55 PM, Josh Hursey  wrote:
> What: A Runtime Interposition Project - Codename Pineapple
>
> Why: Define clear API and semantics for runtime requirements of the OMPI 
> layer.
>
> When:
>  - F June 22, 2012 - Work completed
>  - T June 26, 2012 - Discuss on teleconf
>  - R June 28, 2012 - Commit to trunk
>
> Where: Trunk (development BitBucket branch below)
>  https://bitbucket.org/jjhursey/ompi-pineapple
>
> Attached:
>  PDF of slides presented on the June 12, 2012 teleconf. Note that the
> timeline was slightly adjusted above (work completed date moved
> ealier).
>
>
> Description: Short Version
> --
> Define, in an 'rte.h', the interfaces and semantics that the OMPI
> layer requires of a runtime environment. Currently this interface
> matches the subset of ORTE functionality that is used by the OMPI
> layer. Runtime symbols (e.g., orte_ess.proc_get_locality) are isolated
> to a framework inside this project to provide linker-level protection
> against accidental breakage of the pineapple interposition layer.
>
> The interposition project provides researchers working on side
> projects above and below the 'rte.h' interface a single location in
> the code base to watch for interface and semantic changes that they
> need to be concerned about. Researchers working above the pineapple
> layer might explore something other than (or in addition to) OMPI
> (e.g., Extended OMPI, UPC+OMPI). Researchers working below the
> pineapple layer might explore something other than (or in addition to)
> ORTE under OMPI (e.g., specialized runtimes for specific
> environments).
>
>
> Description: Other notes
> 
> The pineapple interface provides OMPI developers with a runtime API to
> program against without requiring detailed knowledge of the layout of
> ORTE and its frameworks. In some places in OMPI a single source file
> needs to include >5 (up to 12 in one place) different header files to
> get all of the necessary symbols. Developers must not only know where
> these headers are, but must also understand the differences between
> the various frameworks in ORTE to use ORTE. The developer must also be
> aware that there are certain APIs and data structure fields that are
> not available to the MPI process, so should not be used. The pineapple
> project provides an API representing the small subset of ORTE that is
> used by OMPI. With this API a developer only needs to look at a single
> location in the code base to understand what is provided by the
> runtime for use in the OMPI layer.
>
> A similar statement could be made for runtime developers trying to
> figure out what the OMPI layer requires from the a runtime
> environment. Currently they need a deep understanding of the behavior
> of ORTE to understand the semantics of various calls to ORTE from the
> OMPI layer. Then they must develop a custom patch for the OMPI layer
> that extracts the ORTE symbols, and replaces them with their own
> symbols. This process is messy, error prone, and tedious to say the
> least. Having a single set of interfaces and semantics will allow such
> developers to focus their efforts on supporting the Open MPI community
> defined API, and not necessarily the evolution of the ORTE or OMPI
> project internals. This is advantageous when porting Open MPI to an
> environment with a full featured runtime already running on the
> machine, and for researchers exploring radical runtime designs for
> future systems. The pineapple API allows such projects to develop
> beside the mainline Open MPI trunk a little more easily than without
> the pineapple API.
>
>
> FAQ:
> 
> (1) Why is this a separate project and not a framework of OMPI? or a
> framework of ORTE?
>
> After much deliberation between the developers, from a software
> engineering perspective, making the pineapple rte.h interface a
> separate project was the most flexible solution. So neither the OMPI
> layer nor the ORTE layer 'own' the interface, but it is 'owned' by the
> Open MPI project primarily to support the interaction between these
> two layers.
>
> Consider that if we decided to place the interface in the OMPI layer
> as a framework then we would be able to place something other than (or
> in addition to) ORTE underneath OMPI, but we would be lim

[OMPI devel] bug in r26626

2012-06-22 Thread TERRY DONTJE

It looks like compilation of 32 bit platforms is failing due to a 
missing field.  It looks to me that for some reason r26626 deleted 
hdr_segkey in ompi/mca/osc/rdma/osc_rdma_header.h which is used in the 
macro OMPI_OSC_RDMA_RDMA_INFO_HDR_NTOH and HTON.  Is there a reason that 
hdr_segkey was removed, if so more changes are needed.


--
Terry D. Dontje | Principal Software Engineer
Developer Tools Engineering | +1.781.442.2631
Oracle *- Performance Technologies*
95 Network Drive, Burlington, MA 01803
Email terry.don...@oracle.com

Re: [OMPI devel] bug in r26626

2012-06-22 Thread Hjelm, Nathan T

Looks like I missed a few places in udapl and osc. Fixed with r26635 and 
r26634. Hopefully thats the last of them outside of btl/vw.

-Nathan

On Friday, June 22, 2012 7:05 AM, TERRY DONTJE  wrote:
> To: Open MPI Developers; Hjelm, Nathan T
> Subject: bug in r26626
> 
> It looks like compilation of 32 bit platforms is failing due to a missing 
> field.  It looks to me that for some reason r26626 deleted hdr_segkey in 
> ompi/mca/osc/rdma/osc_rdma_header.h which is used in the macro 
> OMPI_OSC_RDMA_RDMA_INFO_HDR_NTOH and HTON.  Is there a reason that hdr_segkey 
> was removed, if so more changes are needed.
> 
> --
> Terry D. Dontje | Principal Software Engineer
> Developer Tools Engineering | +1.781.442.2631
> Oracle - Performance Technologies
> 95 Network Drive, Burlington, MA 01803
> Email terry.don...@oracle.com
> 
> 
>

Re: [OMPI devel] bug in r26626

2012-06-22 Thread Eugene Loh

Looking good.  Just a few more:  btl_udapl_endpoint.c has instances of 
seg_len and seg_addr.  udapl may not have much of a future, but for now 
it's still there.


On 6/22/2012 7:22 AM, Hjelm, Nathan T wrote:

Looks like I missed a few places in udapl and osc. Fixed with r26635 and 
r26634. Hopefully thats the last of them outside of btl/vw.

-Nathan

On Friday, June 22, 2012 7:05 AM, TERRY DONTJE  wrote:

To: Open MPI Developers; Hjelm, Nathan T
Subject: bug in r26626

It looks like compilation of 32 bit platforms is failing due to a missing 
field.  It looks to me that for some reason r26626 deleted hdr_segkey in 
ompi/mca/osc/rdma/osc_rdma_header.h which is used in the macro 
OMPI_OSC_RDMA_RDMA_INFO_HDR_NTOH and HTON.  Is there a reason that hdr_segkey 
was removed, if so more changes are needed.

--
Terry D. Dontje | Principal Software Engineer
Developer Tools Engineering | +1.781.442.2631
Oracle - Performance Technologies
95 Network Drive, Burlington, MA 01803
Email terry.don...@oracle.com




___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] RFC: Pineapple Runtime Interposition Project

2012-06-22 Thread Josh Hursey

In response to some early feedback, I fixed the following in the
branch and updated the wiki:
 * Fixed 'make distcheck' (missing a file)
 * Cleanup some warnings from autogen and configure for -no-orte builds
 * Added an ./autogen.pl -no-pineapple option (give you just the
ORTE/OPAl stack) - see wiki

Thanks for the feedback and keep it coming.

Also - The name will be changed before the commit to the trunk, I'm
working on some suggestions, but if you have any that you want to
lobby for let me know.

Thanks,
Josh

On Fri, Jun 22, 2012 at 8:41 AM, Josh Hursey  wrote:
> The pineapple branch is ready for testing. It was last sync'ed to the
> trunk at r26620. I'll try to update that later today, and keep current
> with the trunk moving forward.
>
> I created a wiki page to discuss some of the build options, for those
> interested. At the bottom of the page are some remaining todo items
> that I will probably need some help resolving.
>  https://svn.open-mpi.org/trac/ompi/wiki/Runtime_Interposition
>
> The branch is at:
>  https://bitbucket.org/jjhursey/ompi-pineapple
>
> I would like to bring this into the trunk the evening after the
> teleconf on Tuesday, June 26, 2012. So if you have cycles to test this
> branch I would appreciate it.
>
> Thanks,
> Josh
>
> On Fri, Jun 15, 2012 at 2:55 PM, Josh Hursey  wrote:
>> What: A Runtime Interposition Project - Codename Pineapple
>>
>> Why: Define clear API and semantics for runtime requirements of the OMPI 
>> layer.
>>
>> When:
>>  - F June 22, 2012 - Work completed
>>  - T June 26, 2012 - Discuss on teleconf
>>  - R June 28, 2012 - Commit to trunk
>>
>> Where: Trunk (development BitBucket branch below)
>>  https://bitbucket.org/jjhursey/ompi-pineapple
>>
>> Attached:
>>  PDF of slides presented on the June 12, 2012 teleconf. Note that the
>> timeline was slightly adjusted above (work completed date moved
>> ealier).
>>
>>
>> Description: Short Version
>> --
>> Define, in an 'rte.h', the interfaces and semantics that the OMPI
>> layer requires of a runtime environment. Currently this interface
>> matches the subset of ORTE functionality that is used by the OMPI
>> layer. Runtime symbols (e.g., orte_ess.proc_get_locality) are isolated
>> to a framework inside this project to provide linker-level protection
>> against accidental breakage of the pineapple interposition layer.
>>
>> The interposition project provides researchers working on side
>> projects above and below the 'rte.h' interface a single location in
>> the code base to watch for interface and semantic changes that they
>> need to be concerned about. Researchers working above the pineapple
>> layer might explore something other than (or in addition to) OMPI
>> (e.g., Extended OMPI, UPC+OMPI). Researchers working below the
>> pineapple layer might explore something other than (or in addition to)
>> ORTE under OMPI (e.g., specialized runtimes for specific
>> environments).
>>
>>
>> Description: Other notes
>> 
>> The pineapple interface provides OMPI developers with a runtime API to
>> program against without requiring detailed knowledge of the layout of
>> ORTE and its frameworks. In some places in OMPI a single source file
>> needs to include >5 (up to 12 in one place) different header files to
>> get all of the necessary symbols. Developers must not only know where
>> these headers are, but must also understand the differences between
>> the various frameworks in ORTE to use ORTE. The developer must also be
>> aware that there are certain APIs and data structure fields that are
>> not available to the MPI process, so should not be used. The pineapple
>> project provides an API representing the small subset of ORTE that is
>> used by OMPI. With this API a developer only needs to look at a single
>> location in the code base to understand what is provided by the
>> runtime for use in the OMPI layer.
>>
>> A similar statement could be made for runtime developers trying to
>> figure out what the OMPI layer requires from the a runtime
>> environment. Currently they need a deep understanding of the behavior
>> of ORTE to understand the semantics of various calls to ORTE from the
>> OMPI layer. Then they must develop a custom patch for the OMPI layer
>> that extracts the ORTE symbols, and replaces them with their own
>> symbols. This process is messy, error prone, and tedious to say the
>> least. Having a single set of interfaces and semantics will allow such
>> developers to focus their efforts on supporting the Open MPI community
>> defined API, and not necessarily the evolution of the ORTE or OMPI
>> project internals. This is advantageous when porting Open MPI to an
>> environment with a full featured runtime already running on the
>> machine, and for researchers exploring radical runtime designs for
>> future systems. The pineapple API allows such projects to develop
>> beside the mainline Open MPI trunk a little more easily than without
>> the pineapple API.

Re: [OMPI devel] OpenIB compile error

2012-06-22 Thread Jeff Squyres

To update everyone: there was much more discussion about this off-list.  :-)

We decided to do the following:

1. The name --with-verbs seems better than --with-openfabrics, if for no other 
reason than the name "openfabrics" encompasses more things than we intend with 
this name (e.g., udapl, psm, etc.).

2. There is a definite problem that needs to be fixed, but it is only partially 
related to the renaming.  Basically: the proposed option renaming is occurring 
opportunistically with this bug fix.

3. We are *not* renaming the openib BTL at this time.  It would be great if 
someone would do this in the future, hint hint!

4. The behavior of --with[out]-verbs is as was described in a prior mail:
  - if --with-verbs is specified, all 3 verbs-based components must succeed
  - if --without-verbs is specified, all 4 verbs-based components will not build
  - if --with-verbs=DIR is specified, all 3 verbs-based components must succeed 
and will use DIR to find verbs headers and libraries

5. The new collectives / non-blocking collectives will likely revise some more 
configury in this area when it comes in mid/late next week, because a bunch of 
verbs stuff moved out of the openib BTL and into common.  Pasha and I will 
revisit this when all that stuff is merged in next week.

6. I'll be committing the current --with-verbs stuff right now in order to fix 
the bug that Brian is running in to.

On Jun 21, 2012, at 2:41 PM, Jeff Squyres wrote:

> On Jun 21, 2012, at 1:54 PM, Shamis, Pavel wrote:
> 
>> About naming - I would agree with Terry, it makes sense to name it after 
>> network API used for this btl - "verbs" (it is not obverts).
> 
> I don't think that "verbs" is terrible, but I think that "openfabrics" has 
> more user-level recognition.
> 
> For example, if you ask a customer what kind of network stack they have 
> installed on their machine, they'll say "I have OFED installed".  They won't 
> say "I have the verbs stack installed."
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI devel] OpenIB compile error

2012-06-22 Thread TERRY DONTJE

On 6/22/2012 3:36 PM, Jeff Squyres wrote:

To update everyone: there was much more discussion about this off-list. :-)

We decided to do the following:

1. The name --with-verbs seems better than --with-openfabrics, if for no other reason
than the name "openfabrics" encompasses more things than we intend with this
name (e.g., udapl, psm, etc.).

2. There is a definite problem that needs to be fixed, but it is only partially
related to the renaming. Basically: the proposed option renaming is occurring
opportunistically with this bug fix.

3. We are *not* renaming the openib BTL at this time. It would be great if
someone would do this in the future, hint hint!

4. The behavior of --with[out]-verbs is as was described in a prior mail:
- if --with-verbs is specified, all 3 verbs-based components must succeed
- if --without-verbs is specified, all 4 verbs-based components will not
build
- if --with-verbs=DIR is specified, all 3 verbs-based components must
succeed and will use DIR to find verbs headers and libraries

What does it mean that "all 3 verbs-based components must succeed"?
Does that mean you cannot specify --with-verbs=DIR --with-openib
--without-ofud?
Does it mean that if you specify --with-verbs=DIR and some other
dependency is not found for openib btl that the configure fails?
What is the 4th verbs-based component this is not built when one
specifies --without-verbs.

--td

5. The new collectives / non-blocking collectives will likely revise some more
configury in this area when it comes in mid/late next week, because a bunch of
verbs stuff moved out of the openib BTL and into common. Pasha and I will
revisit this when all that stuff is merged in next week.

6. I'll be committing the current --with-verbs stuff right now in order to fix
the bug that Brian is running in to.

On Jun 21, 2012, at 2:41 PM, Jeff Squyres wrote:

On Jun 21, 2012, at 1:54 PM, Shamis, Pavel wrote:

About naming - I would agree with Terry, it makes sense to name it after network API used
for this btl - "verbs" (it is not obverts).

I don't think that "verbs" is terrible, but I think that "openfabrics" has more
user-level recognition.

For example, if you ask a customer what kind of network stack they have installed on their machine,
they'll say "I have OFED installed". They won't say "I have the verbs stack
installed."

--
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel

--
Terry D. Dontje | Principal Software Engineer
Developer Tools Engineering | +1.781.442.2631
Oracle *- Performance Technologies*
95 Network Drive, Burlington, MA 01803
Email terry.don...@oracle.com

Re: [OMPI devel] RFC: Pineapple Runtime Interposition Project

[OMPI devel] bug in r26626

Re: [OMPI devel] bug in r26626

Re: [OMPI devel] bug in r26626

Re: [OMPI devel] RFC: Pineapple Runtime Interposition Project

Re: [OMPI devel] OpenIB compile error

Re: [OMPI devel] OpenIB compile error

7 matches

Site Navigation

Mail list logo

Footer information