Re: [OMPI devel] Use of OPAL_PREFIX to relocate a lib
Hi Brian, I finally found some time to test your patch and it solves my problem. Thanks a lot ! Sylvain On Wed, 27 Oct 2010, Barrett, Brian W wrote: I found the issue - somehow, we let the priorities used in installdirs get lost when we rewrote part of the configure system a couple months ago. I have a fix, but it involves changing the configure system, so I won't commit it until this evening. Thanks for pointing out the bug! Brian On Oct 26, 2010, at 8:36 AM, Barrett, Brian W wrote: I'll take a look at fixing this the right way today. Since I wrote both the original autogen.sh that guaranteed static-components was ordered and PREFIX code, I had considered it to be a documented feature that there was strong otdering in the static-components list. So personally, I'd consider it a bug in autogen.pl, but I think we can work around it. Brian On Oct 26, 2010, at 6:01 AM, Sylvain Jeaugey wrote: On Tue, 26 Oct 2010, Jeff Squyres wrote: I don't think this is the right way to fix it. Sorry! :-( I don't think it is the right way to do it either :-) I say this because it worked somewhat by luck before, and now it's broken. If we put in another "it'll work because of a side effect of an unintentional characteristic of the build system" hack, it'll just likely break again someday if/when we change the build system. I completely agree. I'd prefer a more robust solution that won't break as a side-effect of the build system. I'd prefer too, but it would require adding much more logic in the framework, including component sort with priority. And since no-one except me seems to care about this functionality, I'm fine with this patch. More generally, I understand your demand for high quality patches that do things The Right Way. However, I feel it's sometimes exagerated, especially when talking about parts of the code that don't meet these high quality standards. In the end, my feeling is that we don't replace very bad (broken) code with bad (working) code because we want to wait for a perfect (never happening) code. I don't think it's beneficial to the project. Sylvain ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Brian W. Barrett Dept. 1423: Scalable System Software Sandia National Laboratories ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Brian W. Barrett Dept. 1423: Scalable System Software Sandia National Laboratories ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] Use of OPAL_PREFIX to relocate a lib
I found the issue - somehow, we let the priorities used in installdirs get lost when we rewrote part of the configure system a couple months ago. I have a fix, but it involves changing the configure system, so I won't commit it until this evening. Thanks for pointing out the bug! Brian On Oct 26, 2010, at 8:36 AM, Barrett, Brian W wrote: > I'll take a look at fixing this the right way today. > > Since I wrote both the original autogen.sh that guaranteed static-components > was ordered and PREFIX code, I had considered it to be a documented feature > that there was strong otdering in the static-components list. So personally, > I'd consider it a bug in autogen.pl, but I think we can work around it. > > Brian > > On Oct 26, 2010, at 6:01 AM, Sylvain Jeaugey wrote: > >> On Tue, 26 Oct 2010, Jeff Squyres wrote: >> >>> I don't think this is the right way to fix it. Sorry! :-( >> I don't think it is the right way to do it either :-) >> >>> I say this because it worked somewhat by luck before, and now it's >>> broken. If we put in another "it'll work because of a side effect of an >>> unintentional characteristic of the build system" hack, it'll just >>> likely break again someday if/when we change the build system. >> I completely agree. >> >>> I'd prefer a more robust solution that won't break as a side-effect of >>> the build system. >> I'd prefer too, but it would require adding much more logic in the >> framework, including component sort with priority. And since no-one except >> me seems to care about this functionality, I'm fine with this patch. >> >> More generally, I understand your demand for high quality patches that do >> things The Right Way. However, I feel it's sometimes exagerated, >> especially when talking about parts of the code that don't meet these high >> quality standards. >> >> In the end, my feeling is that we don't replace very bad (broken) code >> with bad (working) code because we want to wait for a perfect (never >> happening) code. I don't think it's beneficial to the project. >> >> Sylvain >> ___ >> devel mailing list >> de...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/devel >> > > -- > Brian W. Barrett > Dept. 1423: Scalable System Software > Sandia National Laboratories > > > > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel > -- Brian W. Barrett Dept. 1423: Scalable System Software Sandia National Laboratories
Re: [OMPI devel] Use of OPAL_PREFIX to relocate a lib
I'll take a look at fixing this the right way today. Since I wrote both the original autogen.sh that guaranteed static-components was ordered and PREFIX code, I had considered it to be a documented feature that there was strong otdering in the static-components list. So personally, I'd consider it a bug in autogen.pl, but I think we can work around it. Brian On Oct 26, 2010, at 6:01 AM, Sylvain Jeaugey wrote: > On Tue, 26 Oct 2010, Jeff Squyres wrote: > >> I don't think this is the right way to fix it. Sorry! :-( > I don't think it is the right way to do it either :-) > >> I say this because it worked somewhat by luck before, and now it's >> broken. If we put in another "it'll work because of a side effect of an >> unintentional characteristic of the build system" hack, it'll just >> likely break again someday if/when we change the build system. > I completely agree. > >> I'd prefer a more robust solution that won't break as a side-effect of >> the build system. > I'd prefer too, but it would require adding much more logic in the > framework, including component sort with priority. And since no-one except > me seems to care about this functionality, I'm fine with this patch. > > More generally, I understand your demand for high quality patches that do > things The Right Way. However, I feel it's sometimes exagerated, > especially when talking about parts of the code that don't meet these high > quality standards. > > In the end, my feeling is that we don't replace very bad (broken) code > with bad (working) code because we want to wait for a perfect (never > happening) code. I don't think it's beneficial to the project. > > Sylvain > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel > -- Brian W. Barrett Dept. 1423: Scalable System Software Sandia National Laboratories
Re: [OMPI devel] Use of OPAL_PREFIX to relocate a lib
On Tue, 26 Oct 2010, Jeff Squyres wrote: I don't think this is the right way to fix it. Sorry! :-( I don't think it is the right way to do it either :-) I say this because it worked somewhat by luck before, and now it's broken. If we put in another "it'll work because of a side effect of an unintentional characteristic of the build system" hack, it'll just likely break again someday if/when we change the build system. I completely agree. I'd prefer a more robust solution that won't break as a side-effect of the build system. I'd prefer too, but it would require adding much more logic in the framework, including component sort with priority. And since no-one except me seems to care about this functionality, I'm fine with this patch. More generally, I understand your demand for high quality patches that do things The Right Way. However, I feel it's sometimes exagerated, especially when talking about parts of the code that don't meet these high quality standards. In the end, my feeling is that we don't replace very bad (broken) code with bad (working) code because we want to wait for a perfect (never happening) code. I don't think it's beneficial to the project. Sylvain
Re: [OMPI devel] Use of OPAL_PREFIX to relocate a lib
On Oct 26, 2010, at 6:55 AM, Sylvain Jeaugey wrote: > This problem may be a detail, but it bugs me a lot, so I'd like to have it > fixed. Fair enough. :-) > Here is a patch that changes the path setting algorithm to "last component > wins" instead of "first component wins". > > This is as wrong as was the original code, except that it is consistent with > the way autogen.pl generates static-components.h. > > If nobody objects, I'll commit it tomorrow. I don't think this is the right way to fix it. Sorry! :-( I say this because it worked somewhat by luck before, and now it's broken. If we put in another "it'll work because of a side effect of an unintentional characteristic of the build system" hack, it'll just likely break again someday if/when we change the build system. I'd prefer a more robust solution that won't break as a side-effect of the build system. -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI devel] Use of OPAL_PREFIX to relocate a lib
Hi all,
This problem may be a detail, but it bugs me a lot, so I'd like to have it
fixed. Here is a patch that changes the path setting algorithm to "last
component wins" instead of "first component wins".
This is as wrong as was the original code, except that it is
consistent with the way autogen.pl generates static-components.h.
If nobody objects, I'll commit it tomorrow.
Sylvain
diff -r c9746f7a683a opal/mca/installdirs/base/installdirs_base_components.c
--- a/opal/mca/installdirs/base/installdirs_base_components.c Tue Oct 26
10:56:53 2010 +0200
+++ b/opal/mca/installdirs/base/installdirs_base_components.c Tue Oct 26
12:48:41 2010 +0200
@@ -25,7 +25,7 @@
#define CONDITIONAL_COPY(target, origin, field) \
do {\
-if (origin.field != NULL && target.field == NULL) { \
+if (origin.field != NULL) { \
target.field = origin.field;\
} \
} while (0)
On Thu, 7 Oct 2010, Ralph Castain wrote:
What you are seeing is just the difference in how the build system (old vs new
vs RPM script) travels across the directory tree. The new build system and RPM
do it in alphabetical order, so config comes before env. The old autogen.sh did
it in reverse alpha order, so env came before config. I don't think anyone
thought it made a difference, though you correctly point to one place where it
does.
Modifying the build system to have one place do it differently would be a
mistake, IMO. The better solution would be to use priorities to order the
processing, though that means two passes through the components (one to get the
priorities, and then another to execute) and additional API functions in the
various modules.
On Oct 7, 2010, at 6:25 AM, Sylvain Jeaugey wrote:
Hi list,
Remember this old bug ? I think I finally found out what was going wrong.
The opal "installdirs" framework has two static components : config and env.
These components are automatically detected by the MCA system and they are listed in
opal/mca/installdirs/base/static-components.h.
The problem is that no priority is given, while the order matters : the first
opened component sets the value.
When I build the v1.5 branch, I get 1.env 2.config :
const mca_base_component_t *mca_installdirs_base_static_components[] = {
_installdirs_env_component,
_installdirs_config_component,
NULL
};
When I build an RPM *or* the new default branch, I get 1.config 2.env :
const mca_base_component_t *mca_installdirs_base_static_components[] = {
_installdirs_config_component,
_installdirs_env_component,
NULL
};
I don't know why the generated file is not consistent. It may be related to the
order in which directories are created.
Anyway, the first case seems to be what was intended in the first place. Since
config sets all the values, having it in first position makes env useless.
Besides, in the first configuration, env only needs to sets OPAL_PREFIX and
since config sets all other pathes relatively to ${prefix}, it works.
So, how could we solve this ?
1. Make autogen/configure/whatever generate a consistent static-components.h
with env THEN config ;
2. Add priorities to these components so that env is opened first regardless of
its position in the static components array ;
3. Any other idea ?
Sylvain
On Fri, 19 Jun 2009, Sylvain Jeaugey wrote:
On Thu, 18 Jun 2009, Jeff Squyres wrote:
On Jun 18, 2009, at 11:25 AM, Sylvain Jeaugey wrote:
My problem seems related to library generation through RPM, not with
1.3.2, nor the patch.
I'm not sure I understand -- is there something we need to fix in our SRPM?
I need to dig a bit, but here is the thing : I generated an RPM from the
official openmpi-1.3.2-1.src.rpm (with some defines like install-in-opt, ...)
and the OPAL_PREFIX trick doesn't seem to work with it.
But don't take too much time on this, I'll find out why and maybe this is just
me building it the wrong way.
Sylvain
___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel
___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] Use of OPAL_PREFIX to relocate a lib
What you are seeing is just the difference in how the build system (old vs new
vs RPM script) travels across the directory tree. The new build system and RPM
do it in alphabetical order, so config comes before env. The old autogen.sh did
it in reverse alpha order, so env came before config. I don't think anyone
thought it made a difference, though you correctly point to one place where it
does.
Modifying the build system to have one place do it differently would be a
mistake, IMO. The better solution would be to use priorities to order the
processing, though that means two passes through the components (one to get the
priorities, and then another to execute) and additional API functions in the
various modules.
On Oct 7, 2010, at 6:25 AM, Sylvain Jeaugey wrote:
> Hi list,
>
> Remember this old bug ? I think I finally found out what was going wrong.
>
> The opal "installdirs" framework has two static components : config and env.
> These components are automatically detected by the MCA system and they are
> listed in opal/mca/installdirs/base/static-components.h.
>
> The problem is that no priority is given, while the order matters : the first
> opened component sets the value.
>
> When I build the v1.5 branch, I get 1.env 2.config :
> const mca_base_component_t *mca_installdirs_base_static_components[] = {
> _installdirs_env_component,
> _installdirs_config_component,
> NULL
> };
>
> When I build an RPM *or* the new default branch, I get 1.config 2.env :
> const mca_base_component_t *mca_installdirs_base_static_components[] = {
> _installdirs_config_component,
> _installdirs_env_component,
> NULL
> };
>
> I don't know why the generated file is not consistent. It may be related to
> the order in which directories are created.
>
> Anyway, the first case seems to be what was intended in the first place.
> Since config sets all the values, having it in first position makes env
> useless. Besides, in the first configuration, env only needs to sets
> OPAL_PREFIX and since config sets all other pathes relatively to ${prefix},
> it works.
>
> So, how could we solve this ?
>
> 1. Make autogen/configure/whatever generate a consistent static-components.h
> with env THEN config ;
>
> 2. Add priorities to these components so that env is opened first regardless
> of its position in the static components array ;
>
> 3. Any other idea ?
>
> Sylvain
>
> On Fri, 19 Jun 2009, Sylvain Jeaugey wrote:
>
>> On Thu, 18 Jun 2009, Jeff Squyres wrote:
>>
>>> On Jun 18, 2009, at 11:25 AM, Sylvain Jeaugey wrote:
My problem seems related to library generation through RPM, not with
1.3.2, nor the patch.
>>> I'm not sure I understand -- is there something we need to fix in our SRPM?
>>
>> I need to dig a bit, but here is the thing : I generated an RPM from the
>> official openmpi-1.3.2-1.src.rpm (with some defines like install-in-opt,
>> ...) and the OPAL_PREFIX trick doesn't seem to work with it.
>>
>> But don't take too much time on this, I'll find out why and maybe this is
>> just me building it the wrong way.
>>
>> Sylvain
>>
>>
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] Use of OPAL_PREFIX to relocate a lib
Hi list,
Remember this old bug ? I think I finally found out what was going wrong.
The opal "installdirs" framework has two static components : config and
env. These components are automatically detected by the MCA system and
they are listed in opal/mca/installdirs/base/static-components.h.
The problem is that no priority is given, while the order matters : the
first opened component sets the value.
When I build the v1.5 branch, I get 1.env 2.config :
const mca_base_component_t *mca_installdirs_base_static_components[] = {
_installdirs_env_component,
_installdirs_config_component,
NULL
};
When I build an RPM *or* the new default branch, I get 1.config 2.env :
const mca_base_component_t *mca_installdirs_base_static_components[] = {
_installdirs_config_component,
_installdirs_env_component,
NULL
};
I don't know why the generated file is not consistent. It may be related
to the order in which directories are created.
Anyway, the first case seems to be what was intended in the first place.
Since config sets all the values, having it in first position makes env
useless. Besides, in the first configuration, env only needs to sets
OPAL_PREFIX and since config sets all other pathes relatively to
${prefix}, it works.
So, how could we solve this ?
1. Make autogen/configure/whatever generate a consistent
static-components.h with env THEN config ;
2. Add priorities to these components so that env is opened first
regardless of its position in the static components array ;
3. Any other idea ?
Sylvain
On Fri, 19 Jun 2009, Sylvain Jeaugey wrote:
On Thu, 18 Jun 2009, Jeff Squyres wrote:
On Jun 18, 2009, at 11:25 AM, Sylvain Jeaugey wrote:
My problem seems related to library generation through RPM, not with
1.3.2, nor the patch.
I'm not sure I understand -- is there something we need to fix in our SRPM?
I need to dig a bit, but here is the thing : I generated an RPM from the
official openmpi-1.3.2-1.src.rpm (with some defines like install-in-opt, ...)
and the OPAL_PREFIX trick doesn't seem to work with it.
But don't take too much time on this, I'll find out why and maybe this is
just me building it the wrong way.
Sylvain
Re: [OMPI devel] Use of OPAL_PREFIX to relocate a lib
On Thu, 18 Jun 2009, Jeff Squyres wrote: On Jun 18, 2009, at 11:25 AM, Sylvain Jeaugey wrote: My problem seems related to library generation through RPM, not with 1.3.2, nor the patch. I'm not sure I understand -- is there something we need to fix in our SRPM? I need to dig a bit, but here is the thing : I generated an RPM from the official openmpi-1.3.2-1.src.rpm (with some defines like install-in-opt, ...) and the OPAL_PREFIX trick doesn't seem to work with it. But don't take too much time on this, I'll find out why and maybe this is just me building it the wrong way. Sylvain
Re: [OMPI devel] Use of OPAL_PREFIX to relocate a lib
On Jun 18, 2009, at 11:25 AM, Sylvain Jeaugey wrote: My problem seems related to library generation through RPM, not with 1.3.2, nor the patch. I'm not sure I understand -- is there something we need to fix in our SRPM? -- Jeff Squyres Cisco Systems
Re: [OMPI devel] Use of OPAL_PREFIX to relocate a lib
Ok, never mind.
My problem seems related to library generation through RPM, not with
1.3.2, nor the patch.
Sylvain
On Thu, 18 Jun 2009, Sylvain Jeaugey wrote:
Hi all,
Until Open MPI 1.3 (maybe 1.3.1), I used to find it convenient to be able to
move a library from its "normal" place (either /usr or /opt) to somewhere
else (i.e. my NFS home account) to be able to try things only on my account.
So, I used to set OPAL_PREFIX to the root of the Open MPI directory and all
went fine.
I don't know if relocation was intended in the first place, but with 1.3.2,
this seems to be broken.
It may have something to do with this patch (and maybe others) :
# HG changeset patch
# User bosilca
# Date 1159647750 0
# Node ID c7152b893f1ce1bc54eea2dc3f06c7e359011fdd
# Parent 676a8fbdbb161f0b84a1c6bb12e2324c8a749c56
All the OPAL_ defines from the install_dirs.h contain ABSOLUTE path.
Therefore,
there is no need to prepend OPAL_PREFIX to them.
diff -r 676a8fbdbb16 -r c7152b893f1c opal/tools/wrappers/opal_wrapper.c
--- a/opal/tools/wrappers/opal_wrapper.cFri Sep 29 23:58:58 2006
+
+++ b/opal/tools/wrappers/opal_wrapper.cSat Sep 30 20:22:30 2006
+
@@ -561,9 +561,9 @@
if (0 != strcmp(OPAL_INCLUDEDIR, "/usr/include")) {
char *line;
#if defined(__WINDOWS__)
-asprintf(, OPAL_INCLUDE_PATTERN OPAL_PREFIX "\"\\%s\"",
OPAL_INCLUDEDIR);
+asprintf(, OPAL_INCLUDE_PATTERN "\"\\%s\"",
OPAL_INCLUDEDIR);
#else
-asprintf(, OPAL_INCLUDE_PATTERN OPAL_PREFIX"/%s",
OPAL_INCLUDEDIR);
+asprintf(, OPAL_INCLUDE_PATTERN "/%s", OPAL_INCLUDEDIR);
#endif /* defined(__WINDOWS__) */
opal_argv_append_nosize(_flags, line);
free(line);
George, is there a rationale behind this patch for disabling relocation of
libraries ? Do you think reverting only this patch would bring back the
relocation functionality ?
TIA,
Sylvain
