[OMPI devel] RFC: support for Mellanox's "libhcoll" library

[Joshua Ladd]

Request for Change:

What: Add support for Mellanox Technologies' next-generation non-blocking 
collectives, code-named "libhcoll". This comes in the form of a new "hcoll" 
component to the "coll" framework.

Where: Trunk and 1.7

When: July 1

Why: In support of MPI 3, Mellanox Technologies will make available its next 
generation collectives library, "libhcoll",  in MOFED 2.0 releases and higher 
starting in the late 2013 timeframe. "Libhcoll" adds support for truly 
asynchronous non-blocking collectives on supported HCAs (Connect X-3 and 
higher) via Mellanox Technologies' CORE-Direct technology. "Libhcoll" also adds 
support for hierarchical collectives and features a highly scalable 
infrastructure battle tested and proven on some of the world's largest HPC 
systems.





Joshua S. Ladd, PhD
HPC Algorithms Engineer
Mellanox Technologies

Email: josh...@mellanox.com
Cell: +1 (865) 258 - 8898

Re: [OMPI devel] RFC: support for Mellanox's "libhcoll" library

[Jeff Squyres (jsquyres)]

Sounds good; +1.

On Jun 18, 2013, at 8:02 AM, Joshua Ladd  wrote:

> Request for Change:
>  
> What: Add support for Mellanox Technologies’ next-generation non-blocking 
> collectives, code-named “libhcoll”. This comes in the form of a new “hcoll” 
> component to the “coll” framework.   
>  
> Where: Trunk and 1.7
>  
> When: July 1
>  
> Why: In support of MPI 3, Mellanox Technologies will make available its next 
> generation collectives library, “libhcoll”,  in MOFED 2.0 releases and higher 
> starting in the late 2013 timeframe. “Libhcoll” adds support for truly 
> asynchronous non-blocking collectives on supported HCAs (Connect X-3 and 
> higher) via Mellanox Technologies’ CORE-Direct technology. “Libhcoll” also 
> adds support for hierarchical collectives and features a highly scalable 
> infrastructure battle tested and proven on some of the world’s largest HPC 
> systems.
>  
>  
>  
>  
>  
> Joshua S. Ladd, PhD
> HPC Algorithms Engineer
> Mellanox Technologies
>  
> Email: josh...@mellanox.com
> Cell: +1 (865) 258 - 8898
>  
>  
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI devel] [EXTERNAL] Re: RFC: support for Mellanox's "libhcoll" library

[Barrett, Brian W]

In general, I'm ok with it.  I think we should let it soak for a week or
two in the trunk before we file the CMR to 1.7.

Brian

On 6/18/13 6:51 AM, "Jeff Squyres (jsquyres)"  wrote:

>Sounds good; +1.
>
>On Jun 18, 2013, at 8:02 AM, Joshua Ladd  wrote:
>
>> Request for Change:
>>  
>> What: Add support for Mellanox Technologies¹ next-generation
>>non-blocking collectives, code-named ³libhcoll². This comes in the form
>>of a new ³hcoll² component to the ³coll² framework.
>>  
>> Where: Trunk and 1.7
>>  
>> When: July 1
>>  
>> Why: In support of MPI 3, Mellanox Technologies will make available its
>>next generation collectives library, ³libhcoll²,  in MOFED 2.0 releases
>>and higher starting in the late 2013 timeframe. ³Libhcoll² adds support
>>for truly asynchronous non-blocking collectives on supported HCAs
>>(Connect X-3 and higher) via Mellanox Technologies¹ CORE-Direct
>>technology. ³Libhcoll² also adds support for hierarchical collectives
>>and features a highly scalable infrastructure battle tested and proven
>>on some of the world¹s largest HPC systems.
>>  
>>  
>>  
>>  
>>  
>> Joshua S. Ladd, PhD
>> HPC Algorithms Engineer
>> Mellanox Technologies
>>  
>> Email: josh...@mellanox.com
>> Cell: +1 (865) 258 - 8898
>>  
>>  
>> ___
>> devel mailing list
>> de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>
>
>-- 
>Jeff Squyres
>jsquy...@cisco.com
>For corporate legal information go to:
>http://www.cisco.com/web/about/doing_business/legal/cri/
>
>
>___
>devel mailing list
>de...@open-mpi.org
>http://www.open-mpi.org/mailman/listinfo.cgi/devel
>


--
  Brian W. Barrett
  Scalable System Software Group
  Sandia National Laboratories





smime.p7s
Description: S/MIME cryptographic signature

Re: [OMPI devel] Update IMB in ompi-tests

[Jeff Squyres (jsquyres)]

No one cared today on the call.  So I'll be removing the old IMB 3.2 and 3.2.3 
directories from ompi-tests later today.

Go update your MTT configurations to use the new "imb" directory.



On Jun 14, 2013, at 12:00 PM, Jeff Squyres (jsquyres)  
wrote:

> I'm sorry -- I was not clear:
> 
> - I'm adding 3.2.4 as the imb/ directory *today*.
>  --> You can go update your MTT configs today.
> - I'll refrain from removing the IMB-3.2 and imb_3.2.3 directories until 
> Tuesday.
> 
> 
> On Jun 14, 2013, at 11:57 AM, "Jeff Squyres (jsquyres)"  
> wrote:
> 
>> I see we have 2 versions of IMB in the ompi-tests SVN repo: 3.2, and 3.2.3.
>> 
>> I'm going to update the version to 3.2.4 and remove the other two copies 
>> (IMB_3.2 and imb_3.2.3).  The new directory will be version-independent so 
>> that we don't have to do an announcement when updating the version again:
>> 
>> https://svn.open-mpi.org/svn/ompi-tests/trunk/imb
>> 
>> *** THIS CHANGE WILL AFFECT YOUR MTT CONFIGURATIONS!
>> 
>> I'll do this change after the teleconf this upcoming Tuesday (June 18) to 
>> give everyone time to prepare.
>> 
>> -- 
>> Jeff Squyres
>> jsquy...@cisco.com
>> For corporate legal information go to: 
>> http://www.cisco.com/web/about/doing_business/legal/cri/
>> 
>> 
>> ___
>> devel mailing list
>> de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
> 
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI devel] BTL sendi

[Jeff Squyres (jsquyres)]

George replied to me in IM -- posting here for completeness:

> Yes, there is a reason. if sendi succeeds, it sends a very small data (at 
> least on the devices that supports it), otherwise it returns a descriptor 
> similar to btl_alloc()
> thus you will have to pack the data yourself, and the PML doesn't know if 
> IN_PLACE should be used or not
> thus the resulting solution is slower than the default in the spart_prepare 
> case (which is prepare_src + send)


On Jun 14, 2013, at 3:46 PM, Jeff Squyres (jsquyres)  wrote:

> In working on the upcoming Cisco USNIC BTL, we noticed that btl.sendi is 
> invoked by OB1 in the non-MCA_BTL_FLAGS_SEND_INPLACE case.
> 
> Is there a reason for this?  Or is it only because no one who uses INPLACE 
> has cared about sendi?
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI devel] [EXTERNAL] Re: RFC: support for Mellanox's "libhcoll" library

[Aurélien Bouteiller]

If it is Mellanox specific, maybe the component name could reflect this (like 
mlxhcoll), as it will be visible to end-users. 


Aurelien


Le 18 juin 2013 à 11:25, "Barrett, Brian W"  a écrit :

> In general, I'm ok with it.  I think we should let it soak for a week or
> two in the trunk before we file the CMR to 1.7.
> 
> Brian
> 
> On 6/18/13 6:51 AM, "Jeff Squyres (jsquyres)"  wrote:
> 
>> Sounds good; +1.
>> 
>> On Jun 18, 2013, at 8:02 AM, Joshua Ladd  wrote:
>> 
>>> Request for Change:
>>> 
>>> What: Add support for Mellanox Technologies¹ next-generation
>>> non-blocking collectives, code-named ³libhcoll². This comes in the form
>>> of a new ³hcoll² component to the ³coll² framework.
>>> 
>>> Where: Trunk and 1.7
>>> 
>>> When: July 1
>>> 
>>> Why: In support of MPI 3, Mellanox Technologies will make available its
>>> next generation collectives library, ³libhcoll²,  in MOFED 2.0 releases
>>> and higher starting in the late 2013 timeframe. ³Libhcoll² adds support
>>> for truly asynchronous non-blocking collectives on supported HCAs
>>> (Connect X-3 and higher) via Mellanox Technologies¹ CORE-Direct
>>> technology. ³Libhcoll² also adds support for hierarchical collectives
>>> and features a highly scalable infrastructure battle tested and proven
>>> on some of the world¹s largest HPC systems.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Joshua S. Ladd, PhD
>>> HPC Algorithms Engineer
>>> Mellanox Technologies
>>> 
>>> Email: josh...@mellanox.com
>>> Cell: +1 (865) 258 - 8898
>>> 
>>> 
>>> ___
>>> devel mailing list
>>> de...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> 
>> 
>> -- 
>> Jeff Squyres
>> jsquy...@cisco.com
>> For corporate legal information go to:
>> http://www.cisco.com/web/about/doing_business/legal/cri/
>> 
>> 
>> ___
>> devel mailing list
>> de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> 
> 
> 
> --
>  Brian W. Barrett
>  Scalable System Software Group
>  Sandia National Laboratories
> 
> 
> 
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

--
* Dr. Aurélien Bouteiller
* Researcher at Innovative Computing Laboratory
* University of Tennessee
* 1122 Volunteer Boulevard, suite 309b
* Knoxville, TN 37996
* 865 974 9375