Re: [OMPI devel] ompi-master build error : make can require autotools

[Ralph Castain]

I think that makes sense. I checked and my CentOS box is actually using 1.12.4 
and seems to be fine. I can update it though and probably should.

FWIW: good to remember that this problem only *can* occur - it doesn’t always 
happen, and it seems to require some unusual steps to trigger it. So I don’t 
think it’s necessary to change anything in 1.8 as we can legitimately say 
“don’t do that”.


> On Jan 7, 2015, at 6:43 AM, Jeff Squyres (jsquyres)  
> wrote:
> 
> On Jan 6, 2015, at 9:52 PM, Ralph Castain  wrote:
> 
>> The 1.12 series had a set of problems caused by a changeover in the 
>> maintainers, so it should never be used. I’m not sure if any of those issues 
>> are behind this problem, but it could be so.
>> 
>> Anyway, I think pretty much all of us are using automake 1.14 by this time. 
>> We can discuss how far up to move the requirement, but I think 1.14 would be 
>> the right place.
> 
> Fair enough.
> 
> I don't think we should change the set used to generate 1.8 series tarballs, 
> though.  I think it's a good policy to keep the same autotools tuple for the 
> entire series, unless there's a good reason to change.
> 
> As Dave noted, our default on our Cisco dev cluster is AM 1.13.3.  Probably 
> just because that was the most recent version when we last did maintenance on 
> our AM toolchain.  It's easy enough for us to update, though.
> 
> Another point would probably be to update 
> http://www.open-mpi.org/source/building.php to say that the versions listed 
> for master are a minimum version -- devs can use whatever they want (that 
> works :-) ).  The other table lines list what tuple is used to create those 
> tarballs.
> 
> Also note that per https://github.com/open-mpi/ompi/issues/311, we cannot 
> upgrade past LT 2.4.2 right now.  This looks like a bug in LT itself (not 
> OMPI), but I don't have a workaround/solution to go beyond LT 2.4.2 right now 
> (and I think it's not high priority... yet).
> 
> In conclusion: I'll bump the AM version for master on building.php to be 
> 1.13.3 because I know for a fact that it works.  If we want to move it higher 
> than that, I don't have too strong an opinion.  :-)  I think all the other 
> versions for master are fine.
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> ___
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post: 
> http://www.open-mpi.org/community/lists/devel/2015/01/16748.php

Re: [OMPI devel] ompi-master build error : make can require autotools

[Jeff Squyres (jsquyres)]

On Jan 6, 2015, at 9:52 PM, Ralph Castain  wrote:

> The 1.12 series had a set of problems caused by a changeover in the 
> maintainers, so it should never be used. I’m not sure if any of those issues 
> are behind this problem, but it could be so.
> 
> Anyway, I think pretty much all of us are using automake 1.14 by this time. 
> We can discuss how far up to move the requirement, but I think 1.14 would be 
> the right place.

Fair enough.

I don't think we should change the set used to generate 1.8 series tarballs, 
though.  I think it's a good policy to keep the same autotools tuple for the 
entire series, unless there's a good reason to change.

As Dave noted, our default on our Cisco dev cluster is AM 1.13.3.  Probably 
just because that was the most recent version when we last did maintenance on 
our AM toolchain.  It's easy enough for us to update, though.

Another point would probably be to update 
http://www.open-mpi.org/source/building.php to say that the versions listed for 
master are a minimum version -- devs can use whatever they want (that works :-) 
).  The other table lines list what tuple is used to create those tarballs.

Also note that per https://github.com/open-mpi/ompi/issues/311, we cannot 
upgrade past LT 2.4.2 right now.  This looks like a bug in LT itself (not 
OMPI), but I don't have a workaround/solution to go beyond LT 2.4.2 right now 
(and I think it's not high priority... yet).

In conclusion: I'll bump the AM version for master on building.php to be 1.13.3 
because I know for a fact that it works.  If we want to move it higher than 
that, I don't have too strong an opinion.  :-)  I think all the other versions 
for master are fine.

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI devel] ompi-master build error : make can require autotools

[Ralph Castain]

The 1.12 series had a set of problems caused by a changeover in the 
maintainers, so it should never be used. I’m not sure if any of those issues 
are behind this problem, but it could be so.

Anyway, I think pretty much all of us are using automake 1.14 by this time. We 
can discuss how far up to move the requirement, but I think 1.14 would be the 
right place.


> On Jan 6, 2015, at 6:48 PM, Gilles Gouaillardet 
>  wrote:
> 
> Dave,
> 
> this is an automake issue indeed.
> 
> i use automake 1.12.2 (as advised at
> http://www.open-mpi.org/source/building.php, and
> this is the version used to build the snapshots)
> 
> automake 1.12.2 adds the following line in ompi/include/Makefile.in :
> 
> $(srcdir)/mpi.h.in:  $(am__configure_deps)
>($(am__cd) $(top_srcdir) && $(AUTOHEADER))
>rm -f stamp-h2
>touch $@
> 
> 
> there is no such line with automake 1.14
> (it seems you are using automake 1.13.3, which does not issue such lines)
> 
> bottom line, both web site and build systems should be updated in order
> to use
> a more recent automake version (but which one ?)
> 
> Cheers,
> 
> Gilles
> On 2015/01/07 2:02, Dave Goodell (dgoodell) wrote:
>> On Jan 5, 2015, at 8:40 PM, Gilles Gouaillardet 
>>  wrote:
>> 
>>> Dave,
>>> 
>>> what if you do
>>> 
>>> touch ompi/include/mpi.h.in && sleep 1 && touch 
>>> config/opal_config_pthreads.m4 && ./autogen.pl && module unload 
>>> cisco/autotools/ac269-am1133-lt242 && ./configure --prefix=$PWD/_prefix && 
>>> make
>>> 
>>> 
>>> autogen.pl nor configure does not touch ompi/include/mpi.h.in, and as a
>>> consequence,
>>> config/opal_config_pthreads.m4 is newer than ompi/include/mpi.h when
>>> make is invoked.
>>> 
>>> then from ompi/include/Makefile:
>>> 
>>> $(srcdir)/mpi.h.in:  $(am__configure_deps)
>>>   ($(am__cd) $(top_srcdir) && $(AUTOHEADER))
>>>   rm -f stamp-h2
>>>   touch $@
>> I don't see that rule in my ompi/include/Makefile.  I only have a couple of 
>> mentions of mpi.h.in:
>> 
>> ✂
>> DIST_COMMON = $(top_srcdir)/Makefile.ompi-rules \
>>$(srcdir)/ompi/Makefile.am $(srcdir)/Makefile.in \
>>$(srcdir)/Makefile.am $(srcdir)/mpi.h.in $(srcdir)/mpif.h.in \
>>$(srcdir)/mpif-config.h.in $(am__include_HEADERS_DIST) \
>>$(am__nobase_dist_ompi_HEADERS_DIST) $(pkginclude_HEADERS)
>> [...]
>> am__tagged_files = $(HEADERS) $(SOURCES) $(TAGS_FILES) $(LISP)mpi.h.in
>> [...]
>> stamp-h2: $(srcdir)/mpi.h.in $(top_builddir)/config.status
>>@rm -f stamp-h2
>>cd $(top_builddir) && $(SHELL) ./config.status ompi/include/mpi.h
>> ✂
>> 
>> Maybe the rule in your version of ompi/include/Makefile comes from an older, 
>> buggy version of automake?
>> 
>>> this means $(AUTOHEADER) is invoked, and then ompi/include/mpi.h.in is
>>> touched.
>> I don't see $(AUTOHEADER) being invoked by make when I run the commands you 
>> listed (with "sleep 1" changed to "sleep 5" to be certain):
>> 
>> ✂
>> make[1]: Entering directory `/home/dgoodell/git/ompi-upstream/ompi'
>> Making all in include
>> make[2]: Entering directory `/home/dgoodell/git/ompi-upstream/ompi/include'
>> make  all-am
>> make[3]: Entering directory `/home/dgoodell/git/ompi-upstream/ompi/include'
>>  GENERATE mpif-sizeof.h
>>  LN_S mpi_portable_platform.h
>> make[3]: Leaving directory `/home/dgoodell/git/ompi-upstream/ompi/include'
>> make[2]: Leaving directory `/home/dgoodell/git/ompi-upstream/ompi/include'
>> Making all in datatype
>> ✂
>> 
>> Here are the timestamps on the relevant files after the build completes:
>> 
>> ✂
>> -rw-rw-r-- 1 dgoodell dgoodell 33 2015-01-06 08:21:18.414503328 -0800 
>> ompi/include/stamp-h2
>> -rw-rw-r-- 1 dgoodell dgoodell 166283 2015-01-06 08:21:18.408502854 -0800 
>> ompi/include/mpi.h
>> -rwxrwxr-x 1 dgoodell dgoodell 246260 2015-01-06 08:21:09.283782006 -0800 
>> config.status*
>> -rw-rw-r-- 1 dgoodell dgoodell  18853 2015-01-06 08:17:07.212658002 -0800 
>> config/opal_config_pthreads.m4
>> -rw-rw-r-- 1 dgoodell dgoodell 165986 2015-01-06 08:17:02.209262644 -0800 
>> ompi/include/mpi.h.in
>> ✂
>> 
>> -Dave
>> 
>> ___
>> devel mailing list
>> de...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> Link to this post: 
>> http://www.open-mpi.org/community/lists/devel/2015/01/16738.php
> 
> ___
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post: 
> http://www.open-mpi.org/community/lists/devel/2015/01/16745.php

Re: [OMPI devel] ompi-master build error : make can require autotools

[Gilles Gouaillardet]

Dave,

this is an automake issue indeed.

i use automake 1.12.2 (as advised at
http://www.open-mpi.org/source/building.php, and
this is the version used to build the snapshots)

automake 1.12.2 adds the following line in ompi/include/Makefile.in :

$(srcdir)/mpi.h.in:  $(am__configure_deps)
($(am__cd) $(top_srcdir) && $(AUTOHEADER))
rm -f stamp-h2
touch $@


there is no such line with automake 1.14
(it seems you are using automake 1.13.3, which does not issue such lines)

bottom line, both web site and build systems should be updated in order
to use
a more recent automake version (but which one ?)

Cheers,

Gilles
On 2015/01/07 2:02, Dave Goodell (dgoodell) wrote:
> On Jan 5, 2015, at 8:40 PM, Gilles Gouaillardet 
>  wrote:
>
>> Dave,
>>
>> what if you do
>>
>> touch ompi/include/mpi.h.in && sleep 1 && touch 
>> config/opal_config_pthreads.m4 && ./autogen.pl && module unload 
>> cisco/autotools/ac269-am1133-lt242 && ./configure --prefix=$PWD/_prefix && 
>> make
>>
>>
>> autogen.pl nor configure does not touch ompi/include/mpi.h.in, and as a
>> consequence,
>> config/opal_config_pthreads.m4 is newer than ompi/include/mpi.h when
>> make is invoked.
>>
>> then from ompi/include/Makefile:
>>
>> $(srcdir)/mpi.h.in:  $(am__configure_deps)
>>($(am__cd) $(top_srcdir) && $(AUTOHEADER))
>>rm -f stamp-h2
>>touch $@
> I don't see that rule in my ompi/include/Makefile.  I only have a couple of 
> mentions of mpi.h.in:
>
> ✂
> DIST_COMMON = $(top_srcdir)/Makefile.ompi-rules \
> $(srcdir)/ompi/Makefile.am $(srcdir)/Makefile.in \
> $(srcdir)/Makefile.am $(srcdir)/mpi.h.in $(srcdir)/mpif.h.in \
> $(srcdir)/mpif-config.h.in $(am__include_HEADERS_DIST) \
> $(am__nobase_dist_ompi_HEADERS_DIST) $(pkginclude_HEADERS)
> [...]
> am__tagged_files = $(HEADERS) $(SOURCES) $(TAGS_FILES) $(LISP)mpi.h.in
> [...]
> stamp-h2: $(srcdir)/mpi.h.in $(top_builddir)/config.status
> @rm -f stamp-h2
> cd $(top_builddir) && $(SHELL) ./config.status ompi/include/mpi.h
> ✂
>
> Maybe the rule in your version of ompi/include/Makefile comes from an older, 
> buggy version of automake?
>
>> this means $(AUTOHEADER) is invoked, and then ompi/include/mpi.h.in is
>> touched.
> I don't see $(AUTOHEADER) being invoked by make when I run the commands you 
> listed (with "sleep 1" changed to "sleep 5" to be certain):
>
> ✂
> make[1]: Entering directory `/home/dgoodell/git/ompi-upstream/ompi'
> Making all in include
> make[2]: Entering directory `/home/dgoodell/git/ompi-upstream/ompi/include'
> make  all-am
> make[3]: Entering directory `/home/dgoodell/git/ompi-upstream/ompi/include'
>   GENERATE mpif-sizeof.h
>   LN_S mpi_portable_platform.h
> make[3]: Leaving directory `/home/dgoodell/git/ompi-upstream/ompi/include'
> make[2]: Leaving directory `/home/dgoodell/git/ompi-upstream/ompi/include'
> Making all in datatype
> ✂
>
> Here are the timestamps on the relevant files after the build completes:
>
> ✂
> -rw-rw-r-- 1 dgoodell dgoodell 33 2015-01-06 08:21:18.414503328 -0800 
> ompi/include/stamp-h2
> -rw-rw-r-- 1 dgoodell dgoodell 166283 2015-01-06 08:21:18.408502854 -0800 
> ompi/include/mpi.h
> -rwxrwxr-x 1 dgoodell dgoodell 246260 2015-01-06 08:21:09.283782006 -0800 
> config.status*
> -rw-rw-r-- 1 dgoodell dgoodell  18853 2015-01-06 08:17:07.212658002 -0800 
> config/opal_config_pthreads.m4
> -rw-rw-r-- 1 dgoodell dgoodell 165986 2015-01-06 08:17:02.209262644 -0800 
> ompi/include/mpi.h.in
> ✂
>
> -Dave
>
> ___
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post: 
> http://www.open-mpi.org/community/lists/devel/2015/01/16738.php

Re: [OMPI devel] ompi-master build error : make can require autotools

[Dave Goodell (dgoodell)]

On Jan 5, 2015, at 8:40 PM, Gilles Gouaillardet  
wrote:

> Dave,
> 
> what if you do
> 
> touch ompi/include/mpi.h.in && sleep 1 && touch 
> config/opal_config_pthreads.m4 && ./autogen.pl && module unload 
> cisco/autotools/ac269-am1133-lt242 && ./configure --prefix=$PWD/_prefix && 
> make
> 
> 
> autogen.pl nor configure does not touch ompi/include/mpi.h.in, and as a
> consequence,
> config/opal_config_pthreads.m4 is newer than ompi/include/mpi.h when
> make is invoked.
> 
> then from ompi/include/Makefile:
> 
> $(srcdir)/mpi.h.in:  $(am__configure_deps)
>($(am__cd) $(top_srcdir) && $(AUTOHEADER))
>rm -f stamp-h2
>touch $@

I don't see that rule in my ompi/include/Makefile.  I only have a couple of 
mentions of mpi.h.in:

✂
DIST_COMMON = $(top_srcdir)/Makefile.ompi-rules \
$(srcdir)/ompi/Makefile.am $(srcdir)/Makefile.in \
$(srcdir)/Makefile.am $(srcdir)/mpi.h.in $(srcdir)/mpif.h.in \
$(srcdir)/mpif-config.h.in $(am__include_HEADERS_DIST) \
$(am__nobase_dist_ompi_HEADERS_DIST) $(pkginclude_HEADERS)
[...]
am__tagged_files = $(HEADERS) $(SOURCES) $(TAGS_FILES) $(LISP)mpi.h.in
[...]
stamp-h2: $(srcdir)/mpi.h.in $(top_builddir)/config.status
@rm -f stamp-h2
cd $(top_builddir) && $(SHELL) ./config.status ompi/include/mpi.h
✂

Maybe the rule in your version of ompi/include/Makefile comes from an older, 
buggy version of automake?

> this means $(AUTOHEADER) is invoked, and then ompi/include/mpi.h.in is
> touched.

I don't see $(AUTOHEADER) being invoked by make when I run the commands you 
listed (with "sleep 1" changed to "sleep 5" to be certain):

✂
make[1]: Entering directory `/home/dgoodell/git/ompi-upstream/ompi'
Making all in include
make[2]: Entering directory `/home/dgoodell/git/ompi-upstream/ompi/include'
make  all-am
make[3]: Entering directory `/home/dgoodell/git/ompi-upstream/ompi/include'
  GENERATE mpif-sizeof.h
  LN_S mpi_portable_platform.h
make[3]: Leaving directory `/home/dgoodell/git/ompi-upstream/ompi/include'
make[2]: Leaving directory `/home/dgoodell/git/ompi-upstream/ompi/include'
Making all in datatype
✂

Here are the timestamps on the relevant files after the build completes:

✂
-rw-rw-r-- 1 dgoodell dgoodell 33 2015-01-06 08:21:18.414503328 -0800 
ompi/include/stamp-h2
-rw-rw-r-- 1 dgoodell dgoodell 166283 2015-01-06 08:21:18.408502854 -0800 
ompi/include/mpi.h
-rwxrwxr-x 1 dgoodell dgoodell 246260 2015-01-06 08:21:09.283782006 -0800 
config.status*
-rw-rw-r-- 1 dgoodell dgoodell  18853 2015-01-06 08:17:07.212658002 -0800 
config/opal_config_pthreads.m4
-rw-rw-r-- 1 dgoodell dgoodell 165986 2015-01-06 08:17:02.209262644 -0800 
ompi/include/mpi.h.in
✂

-Dave

Re: [OMPI devel] ompi-master build error : make can require autotools

[Gilles Gouaillardet]

Hi Dave,

yes, i did experience this exact behaviour.

"by accident" meant i ran autogen.pl with the required autotools versions.
then, i ran configure and make with the RHEL6 stock autotools (that are too
old for OMPI)
configure worked just fine, but make crashed because of outdated autotools

if i ran make with the latest autotools, i would probably have not noticed
the issue.

note the issue occurs only when make is invoked for the first time.
if make success, autoheader does touch mpif.h.in, so the next make do not
require autotools.

if i read between the lines, it seems autoheader is not (correctly) invoked
by autogen.pl

please let me know if you cannot reproduce this issue.
(and once again, this is a very minor annoyance, and since tarballs are
generated with make dist, tarballs are very likely unaffected, so bottom
line,
only developers that update m4 files can be affected)


Cheers,

Gilles

On Tue, Dec 23, 2014 at 2:26 AM, Dave Goodell (dgoodell)  wrote:

> On Dec 22, 2014, at 2:42 AM, Gilles Gouaillardet <
> gilles.gouaillar...@iferc.org> wrote:
>
> > Jeff and all,
> >
> > i just found "by accident" that make can require autotools.
> >
> > for example:
> >
> > from (generated) ompi/include/Makefile :
> > $(srcdir)/mpi.h.in:  $(am__configure_deps)
> >($(am__cd) $(top_srcdir) && $(AUTOHEADER))
> >rm -f stamp-h2
> >touch $@
> >
> > and $(am__configure_deps) is a bunch (all?) of .m4 files.
> >
> > from a pragmatic point of view, it means that if update a m4 file, run
> > autogen.pl and configure,
> > then, the first invokation of make will run $(AUTOHEADER)
>
> Gilles,
>
> Have you actually experienced this exact behavior?  The sequence you
> mention above shouldn't cause autoheader to be invoked by make.  Running
> autogen.pl will invoke autoheader after the m4 files were touched, so the
> mpi.h.in file will be newer than its m4 dependencies, which should mean
> that this make rule won't be executed.
>
> -Dave
>
> ___
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post:
> http://www.open-mpi.org/community/lists/devel/2014/12/16713.php
>

Re: [OMPI devel] ompi-master build error : make can require autotools

[Dave Goodell (dgoodell)]

On Dec 22, 2014, at 2:42 AM, Gilles Gouaillardet 
 wrote:

> Jeff and all,
> 
> i just found "by accident" that make can require autotools.
> 
> for example:
> 
> from (generated) ompi/include/Makefile :
> $(srcdir)/mpi.h.in:  $(am__configure_deps)
>($(am__cd) $(top_srcdir) && $(AUTOHEADER))
>rm -f stamp-h2
>touch $@
> 
> and $(am__configure_deps) is a bunch (all?) of .m4 files.
> 
> from a pragmatic point of view, it means that if update a m4 file, run
> autogen.pl and configure,
> then, the first invokation of make will run $(AUTOHEADER)

Gilles,

Have you actually experienced this exact behavior?  The sequence you mention 
above shouldn't cause autoheader to be invoked by make.  Running autogen.pl 
will invoke autoheader after the m4 files were touched, so the mpi.h.in file 
will be newer than its m4 dependencies, which should mean that this make rule 
won't be executed.

-Dave

[OMPI devel] ompi-master build error : make can require autotools

[Gilles Gouaillardet]

Jeff and all,

i just found "by accident" that make can require autotools.

for example:

from (generated) ompi/include/Makefile :
$(srcdir)/mpi.h.in:  $(am__configure_deps)
($(am__cd) $(top_srcdir) && $(AUTOHEADER))
rm -f stamp-h2
touch $@

and $(am__configure_deps) is a bunch (all?) of .m4 files.

from a pragmatic point of view, it means that if update a m4 file, run
autogen.pl and configure,
then, the first invokation of make will run $(AUTOHEADER)

is this a bug (and if yes, is it OMPI related) ?
or is this just a minor annoyance that impacts only developers anyway ?

Cheers,

Gilles