[hwloc-devel] Create success (hwloc git dev-149-gaa488c6)

[MPI Team]

Creating nightly hwloc snapshot git tarball was a success.

Snapshot:   hwloc dev-149-gaa488c6
Start time: Thu Apr  3 21:01:01 EDT 2014
End time:   Thu Apr  3 21:02:27 EDT 2014

Your friendly daemon,
Cyrador

Re: [OMPI devel] cleanup unused btl parameters

[Hjelm, Nathan T]

Bah. Ignore part of that patch. The btl_rdma_pipeline_send_length is used. The 
other one was not. I will provide an updated patch.

-Nathan

From: devel [devel-boun...@open-mpi.org] on behalf of Hjelm, Nathan T 
[hje...@lanl.gov]
Sent: Thursday, April 03, 2014 12:05 PM
To: de...@open-mpi.org
Subject: [OMPI devel] RFC: cleanup unused btl parameters

What: Remove btl parameters that are no longer used. The parameters are 
btl_min_rdma_pipeline_size and btl_min_rdma_pipeline_size

Why: I was working on a short talk talking about the various parameters that 
can be used for tuning Open MPI on infiniband systems and I noticed that 
several parameters are not used in any way. The existence of these parameters 
could be confusing to users so now that 1.8 is out the door it might be a good 
time to remove them. I also took this opportunity to update the various 
components to use C99 sub-object naming on module struct initialization.

When: There may be off-trunk components that still use these parameters so lets 
give this a couple of weeks. Timeout on April 17.

Patch attached.

Re: [OMPI devel] [OMPI svn] svn:open-mpi r31302 - in trunk: opal/mca/base orte/tools/orterun

[Ralph Castain]

Maybe?? It would help avoid the unexpected behavior problem, but may ultimately 
be too unwieldy for widespread adoption. Still, an option to ponder.


On Apr 3, 2014, at 9:27 AM, Kenneth A. Lloyd  wrote:

> Would you consider a user-defined process language library outside of
> OpenMPI? Process functors could be defined by compositions in this external
> area, and maintenance of the language simply the user's responsibility?
> 
> -Original Message-
> From: devel [mailto:devel-boun...@open-mpi.org] On Behalf Of Ralph Castain
> Sent: Thursday, April 03, 2014 8:17 AM
> To: Open MPI Developers
> Subject: Re: [OMPI devel] [OMPI svn] svn:open-mpi r31302 - in trunk:
> opal/mca/base orte/tools/orterun
> 
> I can see the potential utility, but I do have concerns about how to make it
> work without causing a lot of user problems:
> 
> * as currently implemented, it only affects procs launched via mpirun. This
> seems odd - if the user does a direct launch, they would get totally
> different behavior? Shouldn't the registering and parsing of this MCA param
> follow our usual procedure and be done by the application itself instead of
> by orterun?
> 
> * Imagine someone has entered this MCA param into the default MCA param
> file, and that it includes "foo=car" in it. Now the user sets "foo=baz" in
> their environment. How many hours will the user spend tearing out his/her
> hair trying to understand why the application behavior isn't as expected
> before they finally realize that the default MCA param file is messing with
> their non-OMPI envars? Once they finally do figure it out, how do they
> "zero" that MCA param so it isn't processed? We don't have a mechanism for
> overriding a value with "NULL" - doesn't this option require one?
> 
> * again, someone puts an entry in the default MCA param file that includes
> "foo=car". The user executes mpirun with "-x foo=baz", which is perfectly
> legitimate. What is the precedence rule we use to determine the value of
> foo? If we consolidate the two options (as you suggest), then this would be
> alleviated - but one is an MCA param and the other a non-MCA cmd-line
> option, so we would have to break backward compatibility to resolve it
> (which isn't impossible - just worth a discussion).
> 
> * assume an entry in the MCA param file that includes multiple envars, one
> of which is "foo=car". If the user then puts "-mca env_list foo=baz" on
> their cmd line, do we delete all the other envars in the original entry and
> only do the new one? Or would someone expect that only the new one would be
> modified or added, but the others would remain?
> 
> Hence I think this requires some discussion at next week's call. Remember,
> by policy, we don't forward non-MCA envars - but now we are forcibly setting
> them only in the app procs. Strikes me as a major change in behavior, and
> I'm not sure it won't cause more trouble than it solves.
> 
> 
> On Apr 3, 2014, at 1:01 AM, Shamis, Pavel  wrote:
> 
>> 
>>> mca param file treats any key=val as mca parameter only.
>>> In order to add parser support for something that is not mca param, will
> require change file syntax and it will look bad, i.e.:
>>> 
>>> mca btl = sm,self,openib
>>> env DISPLAY = console:0
>>> 
>>> I think the current implementation is less intrusive and re-uses existing
> infra in the most elegant way.
>>> The param file syntax change is too big effort to justify this feature
> (IMHO) which can be provided with existing infra w/o breaking anything.
>> 
>> 
>> IMHO this is a useful parameter option to have. If we may consolidate 
>> these two parameters (-x and the new one) into single one, it might be
> even more helpful.
>> 
>> Best,
>> Pasha.
>> ___
>> devel mailing list
>> de...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> Link to this post: 
>> http://www.open-mpi.org/community/lists/devel/2014/04/14452.php
> 
> ___
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post:
> http://www.open-mpi.org/community/lists/devel/2014/04/14453.php
> 
> 
> -
> No virus found in this message.
> Checked by AVG - www.avg.com
> Version: 2014.0.4355 / Virus Database: 3722/7293 - Release Date: 04/03/14
> 
> ___
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post: 
> http://www.open-mpi.org/community/lists/devel/2014/04/14454.php

Re: [OMPI devel] [OMPI svn] svn:open-mpi r31302 - in trunk: opal/mca/base orte/tools/orterun

[Kenneth A. Lloyd]

Would you consider a user-defined process language library outside of
OpenMPI? Process functors could be defined by compositions in this external
area, and maintenance of the language simply the user's responsibility?

-Original Message-
From: devel [mailto:devel-boun...@open-mpi.org] On Behalf Of Ralph Castain
Sent: Thursday, April 03, 2014 8:17 AM
To: Open MPI Developers
Subject: Re: [OMPI devel] [OMPI svn] svn:open-mpi r31302 - in trunk:
opal/mca/base orte/tools/orterun

I can see the potential utility, but I do have concerns about how to make it
work without causing a lot of user problems:

* as currently implemented, it only affects procs launched via mpirun. This
seems odd - if the user does a direct launch, they would get totally
different behavior? Shouldn't the registering and parsing of this MCA param
follow our usual procedure and be done by the application itself instead of
by orterun?

* Imagine someone has entered this MCA param into the default MCA param
file, and that it includes "foo=car" in it. Now the user sets "foo=baz" in
their environment. How many hours will the user spend tearing out his/her
hair trying to understand why the application behavior isn't as expected
before they finally realize that the default MCA param file is messing with
their non-OMPI envars? Once they finally do figure it out, how do they
"zero" that MCA param so it isn't processed? We don't have a mechanism for
overriding a value with "NULL" - doesn't this option require one?

* again, someone puts an entry in the default MCA param file that includes
"foo=car". The user executes mpirun with "-x foo=baz", which is perfectly
legitimate. What is the precedence rule we use to determine the value of
foo? If we consolidate the two options (as you suggest), then this would be
alleviated - but one is an MCA param and the other a non-MCA cmd-line
option, so we would have to break backward compatibility to resolve it
(which isn't impossible - just worth a discussion).

* assume an entry in the MCA param file that includes multiple envars, one
of which is "foo=car". If the user then puts "-mca env_list foo=baz" on
their cmd line, do we delete all the other envars in the original entry and
only do the new one? Or would someone expect that only the new one would be
modified or added, but the others would remain?

Hence I think this requires some discussion at next week's call. Remember,
by policy, we don't forward non-MCA envars - but now we are forcibly setting
them only in the app procs. Strikes me as a major change in behavior, and
I'm not sure it won't cause more trouble than it solves.


On Apr 3, 2014, at 1:01 AM, Shamis, Pavel  wrote:

> 
>> mca param file treats any key=val as mca parameter only.
>> In order to add parser support for something that is not mca param, will
require change file syntax and it will look bad, i.e.:
>> 
>> mca btl = sm,self,openib
>> env DISPLAY = console:0
>> 
>> I think the current implementation is less intrusive and re-uses existing
infra in the most elegant way.
>> The param file syntax change is too big effort to justify this feature
(IMHO) which can be provided with existing infra w/o breaking anything.
> 
> 
> IMHO this is a useful parameter option to have. If we may consolidate 
> these two parameters (-x and the new one) into single one, it might be
even more helpful.
> 
> Best,
> Pasha.
> ___
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post: 
> http://www.open-mpi.org/community/lists/devel/2014/04/14452.php

___
devel mailing list
de...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
Link to this post:
http://www.open-mpi.org/community/lists/devel/2014/04/14453.php


-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2014.0.4355 / Virus Database: 3722/7293 - Release Date: 04/03/14

Re: [OMPI devel] [OMPI svn] svn:open-mpi r31302 - in trunk: opal/mca/base orte/tools/orterun

[Ralph Castain]

I can see the potential utility, but I do have concerns about how to make it 
work without causing a lot of user problems:

* as currently implemented, it only affects procs launched via mpirun. This 
seems odd - if the user does a direct launch, they would get totally different 
behavior? Shouldn't the registering and parsing of this MCA param follow our 
usual procedure and be done by the application itself instead of by orterun?

* Imagine someone has entered this MCA param into the default MCA param file, 
and that it includes "foo=car" in it. Now the user sets "foo=baz" in their 
environment. How many hours will the user spend tearing out his/her hair trying 
to understand why the application behavior isn't as expected before they 
finally realize that the default MCA param file is messing with their non-OMPI 
envars? Once they finally do figure it out, how do they "zero" that MCA param 
so it isn't processed? We don't have a mechanism for overriding a value with 
"NULL" - doesn't this option require one?

* again, someone puts an entry in the default MCA param file that includes 
"foo=car". The user executes mpirun with "-x foo=baz", which is perfectly 
legitimate. What is the precedence rule we use to determine the value of foo? 
If we consolidate the two options (as you suggest), then this would be 
alleviated - but one is an MCA param and the other a non-MCA cmd-line option, 
so we would have to break backward compatibility to resolve it (which isn't 
impossible - just worth a discussion).

* assume an entry in the MCA param file that includes multiple envars, one of 
which is "foo=car". If the user then puts "-mca env_list foo=baz" on their cmd 
line, do we delete all the other envars in the original entry and only do the 
new one? Or would someone expect that only the new one would be modified or 
added, but the others would remain?

Hence I think this requires some discussion at next week's call. Remember, by 
policy, we don't forward non-MCA envars - but now we are forcibly setting them 
only in the app procs. Strikes me as a major change in behavior, and I'm not 
sure it won't cause more trouble than it solves.


On Apr 3, 2014, at 1:01 AM, Shamis, Pavel  wrote:

> 
>> mca param file treats any key=val as mca parameter only.
>> In order to add parser support for something that is not mca param, will 
>> require change file syntax and it will look bad, i.e.:
>> 
>> mca btl = sm,self,openib
>> env DISPLAY = console:0
>> 
>> I think the current implementation is less intrusive and re-uses existing 
>> infra in the most elegant way.
>> The param file syntax change is too big effort to justify this feature 
>> (IMHO) which can be provided with existing infra w/o breaking anything.
> 
> 
> IMHO this is a useful parameter option to have. If we may consolidate these 
> two parameters (-x and the new one) into
> single one, it might be even more helpful.
> 
> Best,
> Pasha.
> ___
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post: 
> http://www.open-mpi.org/community/lists/devel/2014/04/14452.php

Re: [OMPI devel] [OMPI svn] svn:open-mpi r31302 - in trunk: opal/mca/base orte/tools/orterun

[Shamis, Pavel]

> mca param file treats any key=val as mca parameter only.
> In order to add parser support for something that is not mca param, will 
> require change file syntax and it will look bad, i.e.:
> 
> mca btl = sm,self,openib
> env DISPLAY = console:0
> 
> I think the current implementation is less intrusive and re-uses existing 
> infra in the most elegant way.
> The param file syntax change is too big effort to justify this feature (IMHO) 
> which can be provided with existing infra w/o breaking anything.


IMHO this is a useful parameter option to have. If we may consolidate these two 
parameters (-x and the new one) into
single one, it might be even more helpful.

Best,
Pasha.