Re: minor error and a question

[Ralf Wildenhues]

Hello Joakim,

* Joakim Tjernlund wrote on Wed, Nov 30, 2011 at 08:11:31AM PST:
> Question: "make install" always install all targets, even if some of then 
> haven't been rebuilt
> since last install. Is it possible to have some dependency sensitive install 
> so only rebuilt
> targets are reinstalled?

You want 'install -C'.  Recent install-sh also supports -C.
On GNU/Linux:
  make install INSTALL='install -C'

elsewhere:
  make install INSTALL='/path/to/install-sh -C'

(untested)

Cheers,
Ralf

Re: [RFC] Releasing automake 1.11.2

[Ralf Wildenhues]

Hello Stefano,

* Stefano Lattarini wrote on Sun, Oct 16, 2011 at 05:44:14PM CEST:
> I think it's about time to release automake 1.11.2 -- the `maint'
> branch contains various bug fixes w.r.t. the 1.11.1 release (some
> of them quite important), and offers some new small features and
> various warnings/deprecations that should prepare the users for the
> backward-incompatible changes planned for automake 1.12 (so, the
> more 1.11.2 precedes 1.12, the more these warnings will have a
> chance to be effective).

Agreed.

> Ralf, Jim, could we agree on a timeline/plan of sort for the release
> of automake 1.11.2?  I've never done nor witnessed "as an insider"
> an automake release, so I'd rather leave someone else take the lead
> on this; but I'll try to contribute and learn as much as I can of
> course ;-)

I think you should just try it out.  If you get stuck, just stop and
ask.  I'll try to glance at the list at least every couple of days.

Do you have access to test systems yet?  If so, could you start a
round of tests?  I have a script to do that (I think I sent you
before, otherwise look in my $HOME).

You will need upload access; I think Jim can help with that.

Thanks; and sorry for my continued lack of time.

Cheers,
Ralf

Re: Source files in different directory? Now: linker input file unused because linking not done

[Ralf Wildenhues]

* Paul Elliott wrote on Wed, Jul 20, 2011 at 08:34:11AM CEST:
> On Saturday, June 25, 2011 03:47:56 PM you wrote:
> > > I need to keep my source files (i.e. .c and .h ) in a different directory
> > > tree
> > > from the one that contains the autotools control files and the build
> > > directory.
> > > 
> > > That is, when building X.o or X.lo I need to tell it to always find the
> > > source
> > > file as
> > > ../other/src/X.c not ./X.c. I do not control the contents of the
> > > ../other/src/
> > > directory
> > > so I can not create any Autotools control files there or any object files
> > > there.
> > > To me, that directory is read only.
> > > 
> > > Is there a way to do this conveniently in automake? An implicit rule
> > > perhaps?
> > > 
> > you may try:
> > 
> > DIR = $(top_srcdir)/../other/src
> > 
> > to refer to that directory.

Who wrote this?  It is not good advice.

> I have moved all my .c and .h source to a defferent directory from the one 
> wherein I am building. I now get a lot of error messages of the form:
> "linker input file unused because linking not done"
> What am I doing wrong? Do I need some kind of flag to tell gcc where to look?
> Thank You.

Do not do it like that.

> I now have the source and the include files in a different read only
> directroy: 
> I am building a library:
> 
> 
> from my Makefile.am-
> > astrosrc = $(astrodienst)/src
> > lib_LTLIBRARIES = libswe.la
> > 
> > 
> > libswe_la_SOURCES = \
> > 
> > $(astrosrc)/swecl.c\
> > $(astrosrc)/swedate.c\

That's more complicated that you need to do.  Please remove the astrosrc
definition and all instances of $(astrosrc)/ from your Makefile.am,
rerun autotools.  Then, please create a fresh directory, enter it, and
run configure from there:

mkdir build
cd build
../configure -C

This will cause all objects, libraries and binaries to appear in the
so-called 'build tree', below the directory you just named 'build'.

Hope that helps.

Cheers,
Ralf

Re: bug#9088: Java support

[Ralf Wildenhues]

Hi Jack,

* Jack Kelly wrote on Mon, Jul 18, 2011 at 09:33:58AM CEST:
> On Mon, Jul 18, 2011 at 4:17 PM, Ralf Wildenhues wrote:
> > * Jack Kelly wrote on Sat, Jul 16, 2011 at 06:13:58AM CEST:
> >> Perhaps there should be support for a foo_jar_JARADD, that by analogy
> >> to _LDADD, that specifies additional files to be included in the jar?
> >
> > Why would it have to be a new primary, instead of just reusing _LDADD?
> 
> Because, IMO, it's conceptually different. The output's being
> assembled with `jar', not `ld'.

This argument is attached at the wrong reply of mine, and the rationale
is not conclusive: if the concept of a jar output file were different
from a library output file, then that would be an argument in favor of
using _JARS rather than _LIBRARIES, but not one for using _JARADD rather
than _LDADD.  Also, I'm with John, in that *conceptually*, creating a
jar is virtually the same as creating a library.  It's that currently,
compiler tools don't do a good job of hiding this concept behind a
consistent implementation, but instead expose the internal details of
the language.  Much like what prompted libtool (way back when) to treat
C and C++ libraries differently (which it unfortunately still does and
has to).

_JARS has some merits when its arguments are @substed@, but with
_LDADD, automake knows exactly that it is working on a library or a
jar by virtue of looking at .

Cheers,
Ralf

Re: Java support

[Ralf Wildenhues]

Hello,

allow me a couple of ranty comments:

* Stefano Lattarini wrote on Fri, Jul 15, 2011 at 10:58:01AM CEST:
> I'd rather deprecate the JAVA primary, and then introduce a new `JARS'
> primary, to be used e.g. as follows:

First off, we've _never_ removed support for a primary, and I don't
think we should.  Even just deprecating one is a sign for fairly big
time failure in the past, right?

(All the more reason to do good research before the second attempt.)

>   jar_JARS = foo.jar
>   zardoz_JARS = bar.jar

Why can they not be _LIBRARIES?  (Honest question)

>   foo_jar_SOURCES = all.java foo.java
>   bar_jar_SOURCES = all.java bar.java
>   bar_jar_JAVACFLAGS = -nowarn
>   bar_jar_JARFLAGS = -J-Xmx48M

> This should cause the following behaviour:
> 
>  1. javac will be called (with the `-d' option), to generate .class files
> from all.java and foo.java, and put them into a "private" directory
> `.classes/foo_jar/';

Why a private directory?  How would that play well with subdir-objects?
libtool?  What about per-target flags?  How would you write the rules
in absence of per-target flags?

What about other compilers/environments?  What features do other java
build systems offer that users will want to see?  (Even if they're not
implemented right away, an implementation should take care not to
prevent them being added in the future through wrong decisions early on.)

>  2. similarly, javac will be called (this time also with the `-nowarn'
> option) to generate .class files from all.java and bar.java, placing
> them into the directory `.classes/bar_jar/';
> 
>  3. jar will be called to build `foo.jar' from all the .class files found
> in .classes/foo_jar/;
>   
>  4. similarly, jar will be called (this time with the `-J-Xmx48M' option)
> to build `bar.jar' from all the .class files in .classes/foo_jar/;
> 
>  5. "make mostlyclean" will remove the `.classes' directory;
> 
>  6. "make clean" will also remove foo.jar and bar.jar;
> 
>  7. "make install" will install foo.jar in $(jardir) (which we could make
> default to `$(datadir)/jar') and bar.jar in $(zardozdir) (which we
> expect to be user-defined).
> 
> As my java foo is pretty weak, I'm not sure how to handle jar manifests,

(I think you meant 'fu' here ;-)

> jar entry points, or other jar/javac subtleties and advanced features.
> Suggestions welcome.
> 
> > My question now boils down to this:
> > Has anyone already got some thoughts on how to make Automake play well
> > with Java projects or what kind of workarounds can I use to leverage
> > as much of Automake as possible while still building my Java stuff the
> > way I want it?
> >
> > Alternate question: if Java was a brand new language Automake had
> > never heard of, how would I build my project with Automake without
> > doing what I'm doing now where I have custom rules for everything and
> > I need to hook myself into install- and dist-hooks to Do The Right
> > Thing?
> >
> Basically (and as far as my knowledge/understanding goes), you wouldn't;
> Automake is unfortunately quite unflexible in this regard.  Providing
> your own "all-local", "install-hook" and "uninstall-local" rules is
> probably the best route to follow.

Well, you can have all other kinds of make rules (and Benoit has them),
pretty much as in a normal makefile.  Automake doesn't help you, but
also doesn't take much away from you here (except maybe the odd GNU make
construct it doesn't grok).

> > Like isn't it possible to have a bin_PROGRAMS = foo.jar and
> > foo_jar_SOURCES = main.java and then tell automake how to transform a
> > .java into a .class and then how to turn all the .class into a .jar?
> > It seems that when I try this, Automake dies with "Java source seen
> > but `GCJ' is undefined".
> > 
> Yes, because Automake thinks you want to build foo.jar as a binary
> executable, and so it wants to use gcj to compile it.  This happens
> because you've listed foo.jar in a PROGRAMS primary, which is only
> meant for binary executables, not scripts or bytecode files (ok, I'm
> simplifying things a bit here, but the main point stands).

Well, that could be fixed, no?  Question is what about @substed_programs@.

Cheers,
Ralf

Re: bug#9088: Java support

[Ralf Wildenhues]

* Jack Kelly wrote on Sat, Jul 16, 2011 at 06:13:58AM CEST:
> On Sat, Jul 16, 2011 at 9:55 AM, tsuna wrote:
> > On Fri, Jul 15, 2011 at 1:58 AM, Stefano Lattarini wrote:
> >> As my java foo is pretty weak, I'm not sure how to handle jar manifests,
> >> jar entry points, or other jar/javac subtleties and advanced features.
> >> Suggestions welcome.
> >
> > You can create the manifest manually fairly easily.  Here's an example
> > in the project I'm in the process of autotoolizing:
> > https://github.com/stumbleupon/opentsdb/blob/6059488f38fc8a51d426d6972eee6fdd1033d851/Makefile#L207
> 
> Perhaps there should be support for a foo_jar_JARADD, that by analogy
> to _LDADD, that specifies additional files to be included in the jar?

Why would it have to be a new primary, instead of just reusing _LDADD?

Thanks,
Ralf

Re: Installing a list of files with dynamically generated names

[Ralf Wildenhues]

Hi Benoit,

On Fri, Jul 15, 2011 at 10:01:04PM -0700, tsuna wrote:
> I use the GWT compiler and it produces dynamically generated file
> names (they contain MD5 sums and they depend on what code is getting
> compiled, so they change often).  I get it to put all its output files
> in a given directory so I know where they are.  During `make install'
> I need to install these files somewhere under $pkgdatadir.  I couldn't
> find a way to tell Automake to install these files, since I can't
> formulate their names in Automake.am, am I missing something?  I ended
> up writing a install-data-local rule which is pretty much a
> copy-pasted-edited version of an install rule otherwise generated by
> Automake, except that instead of using a make variable to get the list
> of files to install, I use `find -L outputdir \! -type d`.  Is there a
> better way?

Probably not.  If you don't need to flatten directory structure, and the
number of files is not too large, you can use something like

  $(mkinstalldirs) $(DESTDIR)$(foodir)
  cd outputdir && $(INSTALL_DATA) * $(DESTDIR)$(foodir)

as recent Autoconfs will take care to find an install program that can
install multiple files at once.

If you want to be super-GCS-compliant, you can prepend your commands with
  @$(NORMAL_INSTALL)


Hope that helps.

Cheers,
Ralf

bug#8969: improve synchronization between examples in the manual and test cases

[Ralf Wildenhues]

* Stefano Lattarini wrote on Wed, Jul 06, 2011 at 10:32:56AM CEST:
> Oops, sorry.  Fixed by the attached patch.  OK for maint?  I'll wait
> the customary 72 hours before pushing.

Thanks.  OK, but please remove all the comments about tar unportability
in the tests.  It's so obvious.  ;-)
(and there are probably a dozen other places in the Automake source tree
that you'd have to put the comment at as well, for any amount of
consistency.  I think having one comment, the one that already exists in
lib/am/distdir.am, is fully sufficient.)

> > I'm ok with fixing the manual also, although it's
> > usually clear for people still having to use those vendor tars (and
> > inconvenient for the rest).
> >
> Yes, I'd say we leave the examples in manual untouched.  Agreed?

OK.

> Subject: [PATCH] tests: portability fixes in tests on amhello examples
> 
> * tests/amhello-binpkg.test: Don't use "tar xzf too.tag.gz" to
> extract a gzip-compressed tarball, that's unportable to some
> tar implementations; use the "gzip -dc fo.tar.gz | tar xf -"
> idiom instead.
> * tests/amhello-cflags.test: Likewise.
> * tests/amhello-cross-compile.test: Likewise.

Thanks,
Ralf

Re: Source files in different directory?

[Ralf Wildenhues]

Hello Paul,

* Paul Elliott wrote on Sat, Jun 25, 2011 at 12:05:54PM CEST:
> 
> I need to keep my source files (i.e. .c and .h ) in a different directory 
> tree 
> from the one that contains the autotools control files and the build 
> directory.

That's supported by default: see
  info Automake "VPATH Builds"

Apart from that feature, you can also specify source file names with
subdir components, e.g.,
  foo_SOURCES = a.c sub/b.c sub/subsub/c.c

and depending on whether you use the subdir-objects Automake option,
objects will land in the current directory or in relative subdirectories.

Hope that helps.

Cheers,
Ralf

Re: side stepping automake limitations with custom dependencies

[Ralf Wildenhues]

Hi Jie,

* Jie Zhang wrote on Thu, Jun 23, 2011 at 03:30:43PM CEST:
> On Thu, Jun 23, 2011 at 1:52 AM, Ralf Wildenhues wrote:
> > The only thing that keeps me from setting current behavior in stone is
> > that it is sooo unintuitive and ugly.
> >
> There is another ugly way to achieve the same purpose, i.e. redefine
> the entire, like this:
> 
> vhdl_flex.lo: vhdl_flex.c vhdl_bison.h
> if am__fastdepCC
> $(AM_V_CC)$(LTCOMPILE) -Wno-error -MT $@ -MD -MP -MF
> $(DEPDIR)/$*.Tpo -c -o $@ `test -f '$<' || echo '$(srcdir)/'`$<
> $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo
> else
> $(AM_V_CC) @AM_BACKSLASH@
> if AMDEP
> source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@
> DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
> endif
> $(LTCOMPILE) -Wno-error -c -o $@ `test -f '$<' || echo '$(srcdir)/'`$<
> endif
> 
> This way also references several undocumented internal variables of
> Automake. Which way is preferred? or less ugly?

Well, this is waay more verbose, prone to bitrot as the automake-emitted
rules are improved over time (which is more likely than anything else),
and in this special case also unportable, as $< in non-inference rules
requires GNU make.

> The root cause of this issue is that Automake does not distinguish
> rules with commands and rules that only specify dependencies. I'm
> curious that why Automake does not distinguish them. Is it too
> difficult to implement or it is also a feature or just because no one
> does it?

With Posix make Makefiles, it would be doable with some more work, but
automake doesn't.  With GNU make extensions used in the Makefile.am, it
is almost impossible without implementing a large set of GNU make
extension parsing in automake as well.

At least that's what I think it was never tried.  This issue is from
before I knew what automake does.

Cheers,
Ralf

Re: side stepping automake limitations with custom dependencies

[Ralf Wildenhues]

Hi Mike,

* Mike Frysinger wrote on Thu, Jun 23, 2011 at 02:08:50AM CEST:
>   http://sources.redhat.com/automake/automake.html#Extending
>   [quote]
>   Note that Automake does not make any distinction between rules with
>   commands and rules that only specify dependencies. So it is not possible
>   to append new dependencies to an automake-defined target without
>   redefining the entire rule.
>   [/quote]
> 
> i happened to notice though that automake only parses exact semantic matches. 
>  
> it cannot handle going through variables.  so i tried out:
> $(libbsdl_la_OBJECTS): bsdl_bison.h vhdl_bison.h

> is this an undocumented feature ?  or a bug that'll get fixed at some point ? 
>  
> we've been using this for a few years now without problems, but past behavior 
> does not guarantee future behavior ...

I'd say there are two undocumented bits involved here: the name of the
_OBJECTS variable, and the fact that automake only parses exact matches.
Both are exploited in several user packages, esp. the latter one is,
because in many cases there is no better way.

I'd say that for any of those to change, it would require at least a
major version bump, a big announcement, and most importantly, a
documented replacement mechanism that works for all use cases.  And
maybe even an upgrade path that warns users about their current code.
I don't see that happening anytime soon.  The replacement mechanism
would obviously have to be the first thing, to allow smooth migration.

The only thing that keeps me from setting current behavior in stone is
that it is sooo unintuitive and ugly.

Cheers,
Ralf

Re: Choosing man section at configure time

[Ralf Wildenhues]

Hi Harlan,

* Harlan Stenn wrote on Tue, Jun 21, 2011 at 12:13:06PM CEST:
> For example, I have 'sntp.man.in' and 'sntp.mdoc.in' in the distribution
> tarball, and at configure time "stuff happens" where the decision is
> made as to which version (man or mdoc) of the manual is to be installed,
> and also the manual section it should go in to.
> 
> I AC_SUBST([SNTP_MS]), and SNTP_MS will have the value "1", "1m", or
> "8".
> 
> I also AC_SUBST([MANTAGFMT]), and MANTAGFMT is either "man" or "mdoc".

Try:

man1_MANS =
man8_MANS =
man_MANS = sntp.$(SNTP_MS)
...
sntp.$(SNTP_MS): $(srcdir)sntp.$(MANTAGFMT).in
   sed -f m4/mansec.sed $(srcdir)sntp.$(MANTAGFMT).in > sntp.$(SNTP_MS)

Untested, please report back if it doesn't work.  And yes, it's not too
intuitive, sorry.

Cheers,
Ralf

Re: [GSoC] Early design discussion for TAP/SubUnit support in automake.

[Ralf Wildenhues]

Hi Stefano, and sorry for the long delay,

* Stefano Lattarini wrote on Fri, Apr 29, 2011 at 11:21:06AM CEST:
> Now that my GSoC application "automake - Interfacing with a test protocol
> like TAP or subunit" has been officially accepted, I'd like to start
> discussing with the community some early, high-level design and interface
> decisions.


>  1. Reuse parallel-tests "framework"
>  ---
> 
>   The new TAP/SubUnit support should reuse as much of the current
>   parallel-tests implementation and semantics as possible.  In particular,
>   it should be able to run different test scripts in parallel, generate a
>   `.log' file for each test script and a "summarizing" `test-suite.log'
>   file, honour the make variables AM_TESTS_ENVIRONMENT, TESTS_ENVIRONMENT
>   and AM_COLOR_TESTS and the environment variable VERBOSE, and support
>   different extensions for the test scripts, with extension-specific "log
>   compilers" and flags (the stuff enabled by TESTS_EXTENSIONS,
>   _LOG_COMPILER, etc.).

Sounds all sane.

>   The XFAIL_TESTS variable might be still supported for the sake of
>   backward-compatibility (see below for the details), but it should be
>   deprecated, since TAP and SubUnit offer better and more granular ways
>   to express expected failures.

OK.

In another mail:
> Thinking again about this, it might be worth trying to be even more consistent
> with the existing parallel-tests functionality, and use an `ext_TEST_PROTOCOL'
> variable (or similar) instead of a global `tests-protocol' option.  With some
> tweaking to the post-processing of `.log' files done in `lib/am/check.am' (to
> generate `$(TEST_SUITE_LOG)'), this might allow greater code reuse and a more
> consistent API.
> 
> I've started experimenting with this idea, and I'm not seeing any obvious
> shortcoming right now.  I'm hoping I'll be able to post some experimental
> patches soon enough.

Allowing to specify that per-test is a good idea for transitioning test
suites.  I hope to look into the posted patches later today.

>  2. New automake option `tests-protocol'
>  ---
> 
>  The Tap/SubUnit support in the Automake-generated testsuite drivers
>  should be enabled by a new (argument-requiring) option `tests-protocol',
>  that will be used to specify the level of support for, detection of, and
>  enforcing of SubUnit/TAP streams.
> 
>  The possible values for `tests-protocol' will be:
>   - tests-protocol=tap
> All test scripts are expected to use the TAP protocol.
>   - tests-protocol=subunit
> All test scripts are expected to use the SubUnit protocol.
>   - tests-protocol=adaptive

The way you describe "adaptive", it sounds like it should rather be
named something like "detect" or "detected" or so.

> Each test script is expected to print on its first line of output
> which protocol it uses (the exact format of this special line is
> still to be determined); if this line is unrecognized, the driver
> should assume that the test script uses no protocol.  Also, in
> this case, we should continue to honour XFAIL_TESTS.  All of this
> should help to maximize backward-compatibility.



>  3. Console output from the test driver
>  ---
> 
>   This output should remain as close as possible to the one already
>   provided by the current parallel-tests driver.  The following example
>   should help to clarify what I mean.
> 
>   Assume we are using `tests-protocol=adaptive', and let TESTS be defined
>   to "pass.test skip.test subunit.test tap.test".  Here, `pass.test' and
>   `skip.test' are test scripts that use no protocol (and that exit with
>   status `0' and `77' respectively), `subunit.test' is a test script using
>   the SubUnit protocol and containing four testcases (one passing, one
>   failing, one skipped and one which incurs in an expected failure), and
>   `tap.test' is a test script using the TAP protocol which runs two
>   successful testcases, then encounters an internal error and bails out
>   (using the "Bail out!" directive).
> 
>   With such a setup, this is the output I'd expect from "make check":
> PASS: pass.test
> SKIP: skip.test
> PASS: subunit.test [testcase name/description]
> FAIL: subunit.test [testcase name/ description]
> SKIP: subunit.test [testcase name/description] [reason for skipping]
> XFAIL: subunit.test [testcase name/description] [failure reason]
> PASS: tap.test [testcase name/description]
> PASS: tap.test [testcase name/description]
> ERROR: tap.test [reason for the bailout]
> 
>   Of course, the `color-tests' option should make the above output properly
>   colorized; the attached html file shows what colors I'd expect.

Wrt. to the details of the output, I wouldn't fix things too early.
Two things two consider here:
- if tests are run in parallel, you want to avoid intermixing output
  from different tests as much as possible

Re: absolute or relative path for source files

[Ralf Wildenhues]

* Vincent Torri wrote on Sun, May 15, 2011 at 09:11:10PM CEST:
> Is it better to use absolute path:
> 
> mylib_SOURCES = $(top_builddir)/path1/path2/file.c
> 
> or relative path
> 
> mylib_SOURCES = ../path2/file.c

Both can be ok, but it is important that you don't mix both in the same
makefile, as make does not normalize file names.

Cheers,
Ralf

Re: Dependecy directory and EXTRA_PROGRAMS

[Ralf Wildenhues]

Hi Sergio,

* Sergio Belkin wrote on Fri, May 13, 2011 at 05:37:28AM CEST:
> I have a directory ./doc/tests of source files as documentation
> examples, you can compile optionally issuing "make tests". That files
> as marked with dist_ in order to go into distribution.

OK.  So is there anything wrong with this directory?
BTW, I'd always omit the leading ./ from such names.

> I've found that EXTRA_PROGRAMS *always* generate .deps directory
> regardless you didn't run "make tests". How can I avoid that?

How does this paragraph relate to the previous one?  And why would you
care whether some .deps directory is created or not?

I'm assuming the directory is duly removed upon 'make clean', right?
So everything should be fine, it seems to me.  If you still suspect a
bug, or some behavior that is unacceptable to you, please state the
Automake version you're using and show the Makefile.am.

Thanks,
Ralf

summer is saved

[Ralf Wildenhues]

Hi Stefano, all,

I have great news and not-so-great news: First, congratulations for
your successful Summer of Code application!  Second, my hope for review
time last weekend was over-optimistic, sorry about that.  I might have
connectivity and time this coming weekend, but no promises.  Things
should start to get better the weekend after that, however.

Thanks,
Ralf

Re: Implicitly specify compiler flags based on library dependencies

[Ralf Wildenhues]

Hello Daniel,

* Daniel Neuberger wrote on Wed, Apr 20, 2011 at 03:56:15PM CEST:
> Is there any way to make a dependent library implicitly use any
> compiler flags used by a library on which it depends?

Not automatically.  libtool has 'inherited_linker_flags', but it is not
appropriate for your use case (and of limited usefulness otherwise).

The typical way to solve issues like this is to write a macro AX_LIBFOO
or so, shipped with libfoo, that provides flags for users of libfoo.
That, or a pkg-config file with pretty much the same information.

Hope that helps.

Cheers,
Ralf

Re: Use partial relative path in LIBADD

[Ralf Wildenhues]

Hello Daniel,

* Daniel Neuberger wrote on Wed, Apr 20, 2011 at 03:31:25PM CEST:
> Is there a way to translate this:
> 
>libfoo_la_LIBADD = $(top_builddir)/some/path/libfoobar.la
> 
> to this:
> 
>libfoo_la_LIBADD = some/path/libfoobar.la
> 
> with Automake?

Unforteunately not; neither make, nor automake have good ways to
canonicalize paths (with the exception that GNU make will know that FILE
and ./FILE are identical for non-absolute FILE).

Why do you ever need to use the former?  (It can come in handy in
included Makefile.am fragments.)  The important bits are: however
you specify a file in one Makefile.am, you should always do it the
same way, or things may go wrong.  So a library built from the same
Makefile.am should have a relative name; but arguably, you could
also write, e.g.,

lib_LTLIBRARIES = ./libfoo.la
__libfoo_la_SOURCES = ...

if you really want to make your life hard; I'd always suggest

lib_LTLIBRARIES = libfoo.la
libfoo_la_SOURCES = ...

and using plain 'libfoo.la' in libbar_la_LIBADD.

Hope that helps.

Cheers,
Ralf

Re: Variable list modification

[Ralf Wildenhues]

Hello Justin,

* Too, Justin A. wrote on Wed, Apr 20, 2011 at 11:33:26PM CEST:
> FOO=1 2
> $(FOO:=bar) => 1bar 2bar.
> 
> Is there a way to prepend bar?

This is a GNU make question, or maybe a Posix make question, depending
on what you target.  In Posix make, the answer is no.  With GNU make,
you can use the patsubst function, see 'info make --index patsubst'.

Please note that automake won't really grok GNU make functions, so using
them in variables that are special to automake can lead to surprises.

Hope that helps.

Cheers,
Ralf

Re: can't send to list

[Ralf Wildenhues]

Hello Daniel,

* Neuberger, Daniel [USA] wrote on Wed, Apr 20, 2011 at 04:39:05PM CEST:
> Sorry that this is off topic (I couldn't find a more appropriate list
> though), but it seems that I cannot send to this list even though I'm
> subscribed.  My initial subscription was with gmail and I got the list
> welcome message, but my messages still aren't getting through.
> They're just being dropped.  I've tried sending directly from gmail
> and also using gmane.
> 
> Any ideas why my messages aren't going through?

Your mails are getting through.  But there is first-post moderation,
and the moderator(s) may not see your mail for a few hours or be busy.

Cheers,
Ralf

Re: Controlling libtool library link stage

[Ralf Wildenhues]

Hello Justin,

* Too, Justin A. wrote on Tue, Apr 19, 2011 at 06:52:33PM CEST:
> The libraries in LIBADD are the convenience libraries generated in
> subdirectories.  I would like to perform the link step of all of the
> intermediate libraries into libprogram.la on a ramdisk.

Why not put the whole build tree on the ram disk?
I might be spoiled there.

> Is there a
> hook to override libprogram.la's link step such that I can copy all of
> the intermediate libraries to the ramdisk and then link them into
> libprogram.la on the ramdisk?

Not really.

> Or would I need to create a wrapper for
> the linker and set the top-level Makefile.am's linker to this
> wrapper-script?

You could probably cope without, with something like

# AC_SUBST(ramdisk) adequately from configure.
libfoo_la_LIBADD = $(ramdisk)/libconv1.la $(ramdisk)/libconv2.la ...

# Could use EXTRA_..._DEPENDENCIES with Automake 1.12.
libfoo_la_DEPENDENCIES = $(libfoo_la_LIBADD)
$(ramdisk)/libconv1.la: sub/libconv1.la
cp -p sub/libconv1.la $@
$(ramdisk)/libconv2.la: ...
...

Cheers,
Ralf

Re: Recursive initialization of autotools project?

[Ralf Wildenhues]

Hi Trevor,

* Trevor Bain wrote on Fri, Apr 15, 2011 at 08:55:06PM CEST:
> I have a 3 deep project (as follows below). If I run "automake
> --add-missing" it only adds the files for the current level, hence to
> start out the project I need to hit each branch with the
> "--add-missing" flag (and also run aclocal etc.). After this, from the
> root, I can run "autoreconf", "./configure", and "make" just fine.
> 
> Is there a way to recursively run all of the initialization or some
> other method to prevent me needing to leave the root to initialize the
> whole project (hopefully w/o a shell script that requires me to
> respecify the project layout)?

autoreconf --verbose --install

Cheers,
Ralf

Re: Sub-make environment

[Ralf Wildenhues]

Hello Justin,

* Too, Justin A. wrote on Fri, Apr 15, 2011 at 11:54:47PM CEST:
> I'm trying to communicate environment variables to a sub-make.

The answer to this question is most similar to the answer to a recent
question of yours:
http://thread.gmane.org/gmane.comp.sysutils.automake.general/12808/focus=12809

Cheers,
Ralf

Re: Passing arguments to test programs during test suite

[Ralf Wildenhues]

Hi Jeff,

* Daily, Jeff A wrote on Fri, Apr 15, 2011 at 08:06:56PM CEST:
> I'm using automake 1.11.1's 'parallel-tests' test suite capabilities.  It's
> been fine so far.  I even hacked it a little to support launching my MPI
> programs.  I have some tests that run serially (without MPI) and some that
> do:
> 
> TESTS = $(SERIAL_TESTS) $(PARALLEL_TESTS)
> LOG_COMPILER = \
> maybe_mpi=`if echo "$(SERIAL_TESTS)" | $(GREP) "$$p" > /dev/null; then echo
> ""; else echo "$(MPIEXEC)"; fi`; \
> $$maybe_mpi
> 
> Users run the test suite using something like
> 
> make check MPIEXEC="mpiexec -np 4"
> 
> On one particular platform we don't use MPI but instead have our own process
> manager built into each of our programs.  This means each program must be
> launched by passing command-line arguments such as "-np 4".  How can I do
> this?  I tried:
> 
> LOG_COMPILER = \
> maybe_mpi=`if echo "$(SERIAL_TESTS)" | $(GREP) "$$p" > /dev/null; then echo
> ""; else echo "$(MPIEXEC)"; fi`; \
> tst="$$dir$$f -np 4"; \
> $$maybe_mpi

Make that last line
  eval "$$maybe_mpi"

instead.

> To make things even harder, I have some test programs which take specific
> command-line arguments.  I've been leaving them out of the test suite since
> I can't figure out how to pass command-line arguments to any tests let alone
> individual tests.

We should add support for that.  Right now your best bet is a wrapper
script that transports its options to the programs options.  That
wrapper script can be part of LOG_COMPILER.


If your MPI and non-MPI programs have different suffixes, you could make
things simpler; for example:

TEST_SUFFIXES = .t .t-mpi
T_MPI_LOG_COMPILER = $(MPIEXEC)

FWIW, we still go a simpler way and have a wrapper script for each test.
It sources a defs scriptlet (generated from $(srcdir)/defs.in) that
contains a more-or-less portable shell function to start MPI jobs.
The shell function aims to work also with batch schedulers like qsub.

Hope that helps.

Cheers,
Ralf

Re: Modify $PATH for all subdirectories

[Ralf Wildenhues]

Hello Justin,

* Too, Justin A. wrote on Wed, Apr 06, 2011 at 10:45:06PM CEST:
> An executable is generated during 'make' in a directory called
> scripts/test/install/bin. This executable is used during the
> "check-local" rules in all of the Makefile.am's. In the top-level
> Makefile.am, how can I accomplish something like "export
> PATH=`pwd`/scripts/test/install/bin:$PATH" so the executable will be
> available to the "check-local" rules (where would I include this)? Or
> is there a better way to accomplish this?

There are several possible answers to this.

If your project requires GNU make, you can probably just write
  export PATH := $(shell echo `pwd`/scripts/test/install/bin:${PATH})

or
  export PATH := ${PWD}/scripts/test/install/bin:${PATH}

in the toplevel Makefile.am (untested).  But that won't work with
other makes.  With make 3.81 or newer, you can even limit that to
the check target by putting the above as a target-specific flag if
that is really necessary.

If all your tests use one of the Automake test drivers, you can use
  TESTS_ENVIRONMENT = PATH=$(top_builddir)/scripts/test/install/bin:$$PATH; 
export PATH;

but that won't help for check-local rules that invoke other test
drivers.  (Autotest has a similar feature, see 'info Autoconf --index
AUTOTEST_PATH' or the description for the atlocal file.)

It is also possible to change macros during the make recursion step
by messing with AM_MAKEFLAGS.  E.g., your toplevel Makefile.am could
have

 AM_MAKEFLAGS = PATH=`pwd`/scripts/test/install/bin:${PATH}

Hope that helps.

Cheers,
Ralf

Re: Changing build steps with automake

[Ralf Wildenhues]

Hi Jan,

* Honza wrote on Tue, Apr 05, 2011 at 03:29:37PM CEST:
> I'm new to autotools and have not found the answer to my question in
> documentation so I decided to ask here. I need to change/add compilation
> steps in opencore-amr project, which uses autotools to generate makefiles.
> 
> I'd like to have following procedure
> 1. run gcc with -S producing asm code
> 2. perform a sed editing on resulting asm files
> 3. compile edited asm files
> 
> Is there any way how to alter Makefile.am to reach demanded behaviour,
> please? It would be easy when using simple Makefile, but I have no idea in
> this situation.

Automake is surprisingly inflexible in this area.  Here's a couple of
ideas.

1) If your project doesn't use per-target compile flags, you can just
write

.c.o:
...
.c.obj:
...

and maybe
.c.lo:
...

rules that do this for you.  Look at what automake puts in Makefile.in,
use that as staring point in your Makefile.am and adjust.


2) If your project uses per-target compile flags, the most fool-proof
way would be to use a wrapper script for the compiler, and then near the
end of configure.ac

CC="\${top_srcdir}/path/to/wrapper $CC"
(or top_builddir if you need to generate the wrapper)

and let the wrapper do the steps for you.


Hope that helps.  If not, it can help to show what you've got, and
errors.

Cheers,
Ralf

Re: install to top_builddir

[Ralf Wildenhues]

Hello Reuben,

* Reuben Hawkins wrote on Tue, Apr 05, 2011 at 05:51:52AM CEST:
> In my build process, I need to have all the libraries copied to
> $(top_builddir)/lib and all the bins to $(top_builddir)/bin and headers,

Can you explain to us why you would need that to be done?

> etc...  Is there some de facto standard what to do this with automake?

Extending the 'all' target can be done with the 'all-local' target.
It (and any prerequisites it may have) are updated at 'make all' time
(but the order in which it is run wrt. other prerequisites of 'all' is
undefined in the parallel make case.

> What I have now at the end of each of my Makefile.am's (which isn't doing
> exactly what I want, but before I put more effort into it, I thought I'd ask
> here) is...
> 
> publish:
> $(MAKE) install DESTDIR="$(abs_builddir)/$(top_builddir)"
> 
> all: am-all publish

You shouldn't override all.  Use something like

all-local: publish

but also make 'publish' depend upon all files you are about to install,
because otherwise parallel make may introduce ugly races with your two
make instances competing for each other in updating them.

Your publish rule commands have problems too: you don't override prefix,
so that stuff ends up below $(abs_builddir)/$(top_builddir)$(prefix).
That doesn't make sense to me (but since you don't explain what you
actually need to do, I'm not sure).  I'd have expected something like
  $(MAKE) $(AM_MAKEFLAGS) install DESTDIR= prefix="$(abs_top_builddir)"

As to motivation: I suspect you want to reuse code for finding
installed stuff similarly when looking for uninstalled stuff.
It is often possible to make this easier by laying out the build
tree similarly to the install tree (or multiple install trees,
for large projects).  But without more details, it's hard to give
better advice.

Hope that helps.

Cheers,
Ralf

Re: not breaking "make" after m4 macros and source files changed

[Ralf Wildenhues]

Hi Bruno,

* Bruno Haible wrote on Sat, Apr 02, 2011 at 07:08:14PM CEST:
> OK, let me get more constructive. For which of the following do you have
> test cases in the Automake test suite already, that verify that after
> "./configure" and "make" and some edits in the infrastructure, "make"
> still works as well as "make -k distclean; ./configure; make" would
> have worked:
> 
>   - A renamed m4 macro.
> Before: m4/foo.m4 defines FOO1, configure.ac depends on m4/macros.m4 that
> invokes FOO1.
> After:  m4/foo.m4 defines FOO2, configure.ac depends on m4/macros.m4 that
> invokes FOO2.

automake/tests/aclocal5.test and
automake/tests/aclocal6.test come close.

>   - A removed m4 file.
> Before: m4/foo.m4 defines FOO, configure.ac depends on m4/macros.m4 that
> invokes FOO.
> After:  m4/foo.m4 is gone, configure.ac depends on m4/macros.m4 that
> does not invoke FOO any more.

automake/tests/acloca22.test

>   - A renamed m4 file.
> Before: m4/foo1.m4 defines FOO, configure.ac depends on m4/macros.m4 that
> invokes FOO.
> After:  m4/foo2.m4 defines FOO, configure.ac depends on m4/macros.m4 that
> invokes FOO.

>   - A renamed m4 macro in a renamed m4 file.
> Before: m4/foo1.m4 defines FOO1, configure.ac depends on m4/macros.m4 that
> invokes FOO1.
> After:  m4/foo2.m4 defines FOO2, configure.ac depends on m4/macros.m4 that
> invokes FOO2.

These two could use a test.

>   - An added AC_SUBST.
> Before: configure.ac depends on m4/macros.m4 that AC_SUBSTs FOO.
> lib/foo.h is generated through lib/Makefile, substituting FOO.
> lib/foo.c includes foo.h and needs to be compiled for 'all'.
> After:  configure.ac depends on m4/macros.m4 that AC_SUBSTs FOO and BAR.
> lib/foo.h is generated through lib/Makefile, substituting FOO and 
> BAR.
> lib/foo.c includes foo.h and needs to be compiled for 'all'.

This is pretty much an issue with the rule generating lib/foo.h needing
to depend on config.status, so not primarily an Automake issue.  The
fact that config.status is regenerated when a macro is updated in some
way, is exercised in several tests in the Automake testsuite.

>   - An added .h file.
> Before: configure.ac depends on m4/macros.m4 that AC_SUBSTs STDIO_H to 
> empty.
> lib/stdio.h is generated through lib/Makefile if STDIO_H is 
> non-empty.
> lib/foo.c includes stdio.h and needs to be compiled for 'all'.
> After:  configure.ac depends on m4/macros.m4 that AC_SUBSTs STDIO_H to 
> stdio.h.
> lib/stdio.h is generated through lib/Makefile if STDIO_H is 
> non-empty.
> lib/foo.c includes stdio.h and needs to be compiled for 'all'.
> Here you need to check that after "make", lib/stdio.h is present.

Likewise.  Since here, automake's dependency tracking is involved, a
test would be good.

>   - A removed .h file.
> Before: configure.ac depends on m4/macros.m4 that AC_SUBSTs STDIO_H to 
> stdio.h.
> lib/stdio.h is generated through lib/Makefile if STDIO_H is 
> non-empty.
> lib/foo.c includes stdio.h and needs to be compiled for 'all'.
> After:  configure.ac depends on m4/macros.m4 that AC_SUBSTs STDIO_H to 
> empty.
> lib/stdio.h is generated through lib/Makefile if STDIO_H is 
> non-empty.
> lib/foo.c includes stdio.h and needs to be compiled for 'all'.
> Here you need to check that after "make", lib/stdio.h is gone.

This is a case that doesn't currently work, as nothing causes
lib/stdio.h to be removed.  It is not yet clear to me where to
stick those semantics sanely.

Thanks,
Ralf

Re: Example in libtool manual gives wrong dependencies w/ automake.

[Ralf Wildenhues]

Hello Nick,

* Nick Bowler wrote on Fri, Apr 01, 2011 at 04:04:21PM CEST:
> I'm working on a project which uses libltdl to load modules, and I've
> set it up in a manner pretty similar to what's described in the libtool
> manual (10.3 Linking with dlopened modules, http://xrl.us/bjk9e5).
> In that section, the manual recommends to use a weak library interface.
> 
> However, the example given in the manual does not work because
> the generated makefile lacks dependencies from libloader.la to
> libinterface.la's objects.

Indeed.  The reference to the internal *_OBJECTS variable is surprising.
It would be cleaner to have a libfoo_conv.la convenience
(noinst_LTLIBRARIES) archive that had the same sources as libfoo.la and
add that to libbar_la_LIBADD.

A patch to fix libtool.texi to this end would be appreciated.

>  Here's a simplified and complete
> version of the example in the libtool manual:
> 
>   % cat >Makefile.am <<'EOF'
>   lib_LTLIBRARIES = libbar.la libfoo.la
> 
>   libfoo_la_SOURCES = foo.c
> 
>   libbar_la_SOURCES = bar.c
>   libbar_la_LDFLAGS = -weak libfoo.la
>   libbar_la_LIBADD  = $(libfoo_la_OBJECTS)
>   EOF

>   % ./configure -q && make
> CC bar.lo
> CCLD   libbar.la
>   libtool: link: `foo.lo' is not a valid libtool object
>   make: *** [libbar.la] Error 1
> 
> Looking at the generated Makefile.in, we see this is because libbar.la
> doesn't have any dependency on foo.lo.  We can add it to the
> Makefile.am:
> 
>   libbar_la = libbar.la
>   $(libbar_la): $(libfoo_la_OBJECTS)
> 
> and now it will build.  So, should automake support the example in the
> libtool manual, or does the libtool manual need to be fixed?  If the
> latter, is the above workaround a good one?

I see several workarounds.  The one I described above is clean.
Another would be to compute libbar_la_DEPENDENCIES manually, e.g.:
  libbar_la_DEPENDENCIES = $(libfoo_la_OBJECTS)

In a real-life situation, there are other dependencies, so you'd prefer
to use
  EXTRA_libbar_la_DEPENDENCIES = $(libfoo_la_OBJECTS)

so automake will still compute libbar_la_DEPENDENCIES
(available only in git Automake, will be in 1.12; sorry).

One thing that I wonder about is why you actually need the foo code to
be both in libfoo and libbar.  If possible, then code should only ever
exist in one shared library.  (I may be missing details from the
dlpreopen machinery here now, it's been a while ... feedback as to what
worked for you appreciated.)


As to whether automake should be able to look through $(*_OBJECTS)
references in _LIBADD and add them to _DEPENDENCIES: yes that too
seems like a viable idea.  'info Automake "Program and Library
Variables"' is currently vague about this case (but it can be, since
_OBJECTS are supposed to be internal details).

So there really are two issues here, one in libtool.texi and one in
Automake.  Thanks for reporting them.

Cheers,
Ralf

Re: subdir-objects, dirstamps and --disable-dependency-tracking (bug!)

[Ralf Wildenhues]

* Ignacy Gawedzki wrote on Wed, Mar 30, 2011 at 10:13:32AM CEST:
> On Wed, Mar 30, 2011 at 07:51:40AM +0200, thus spake Ralf Wildenhues:
> > * Ignacy Gawedzki wrote on Wed, Mar 30, 2011 at 12:03:09AM CEST:
> > > I suspect I found a bug in automake 1.11.1 .
> > 
> > Yes, you have.  Thanks for reporting it!
> 
> Shall I post that to bug-autom...@gnu.org then?

That'd be prudent if we'd otherwise forget the bug.  Shouldn't be the
case here.

> > > Does anybody know of a clean fix/workaround?
> > 
> > Well, a workaround is to add the above line to your Makefile.am.
> 
> Can I reasonably assume that it won't break again when the bug is fixed at
> some point in the future?

You can assume that the workaround remains working.  It is very unlikely
that the dirstamp mechanism is changed.

Below is a proposed patch for maint.  It's not completely pretty, as it
adds some more code duplication in handle_single_transform, and it also
doesn't cause the most compact Makefile.in I can think of in all cases:
if the derived-sources rule happens to not be an inference rule, then
the dependency could be tacked right onto the rule.  That seems very
minor though.

The issue happens both with and without per-target compile flags, it's
just a lot less likely to be visible without them unless you explicitly
run 'make sub/foo.cc'.

Review appreciated.  Ignacy, OK to add your name to THANKS?
(make check is still running.)

Thanks,
Ralf

Create subdirs for generated sources even when not dep tracking.

* automake.in (handle_single_transform): If $object is derived
and lands in subdir, be sure to output a dirstamp dependency.
* tests/lex-subobj-nodep.test: New test.
* tests/Makefile.am (TESTS): Update.
* THANKS: Update.
Report and initial test case by Ignacy Gawedzki.

diff --git a/automake.in b/automake.in
index d56fbf7..d74730d 100755
--- a/automake.in
+++ b/automake.in
@@ -2053,6 +2053,15 @@ sub handle_single_transform ($%)
# derived from is not.
&push_dist_common ($object)
  unless ($topparent =~ /^(?:nobase_)?nodist_/);
+
+   # If resulting derived source is in a subdir, we need to make
+   # sure the subdir exists at build time.
+   if ($object =~ /\//)
+ {
+   my $dirstamp = require_build_directory_maybe ($object);
+   depend ($object, $dirstamp)
+ if ($dirstamp);
+ }
next;
  }
 
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 4becdbb..f095dd8 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -428,6 +428,7 @@ lex2.test \
 lex3.test \
 lex4.test \
 lex5.test \
+lex-subobj-nodep.test \
 lflags.test \
 lflags2.test \
 libexec.test \
diff --git a/tests/lex-subobj-nodep.test b/tests/lex-subobj-nodep.test
new file mode 100755
index 000..c4c49ea
--- /dev/null
+++ b/tests/lex-subobj-nodep.test
@@ -0,0 +1,72 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Ensure subdirs for subdir scanners are generated when subdir-objects
+# are used, even when dependency tracking is disabled.
+
+required=flex
+. ./defs || Exit 1
+
+set -e
+
+cat >>configure.in <<\END
+AC_PROG_CC
+AM_PROG_CC_C_O
+AC_PROG_LEX
+AC_OUTPUT
+END
+
+cat >Makefile.am <<\END
+AUTOMAKE_OPTIONS = subdir-objects
+bin_PROGRAMS = p1 p2
+p1_SOURCES = sub1/s1.l
+p2_SOURCES = sub2/s2.l
+p2_CPPFLAGS = -DWHATEVER
+END
+
+mkdir sub1 sub2
+
+cat >sub1/s1.l <<\END
+%%
+"END"   return EOF;
+.
+%%
+int
+main ()
+{
+  while (yylex () != EOF)
+;
+
+  return 0;
+}
+
+int yywrap() { return 0; }
+END
+
+cp sub1/s1.l sub2/s2.l
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+mkdir build
+cd build
+../configure --disable-dependency-tracking
+$MAKE sub1/s1.c
+$MAKE sub2/s2.c
+rm -rf sub1 sub2
+$MAKE
+
+:

Re: Removing Mac OS X resource forks from distribution tarballs

[Ralf Wildenhues]

* Dave Hart wrote on Thu, Mar 31, 2011 at 08:46:56AM CEST:
> Apparently,
> there is no command-line switch to disable generation of ._
> psuedofiles in tarballs, but there is an undocumented environment
> variable that so modifies tar behavior,
> COPY_EXTENDED_ATTRIBUTES_DISABLE=true. For reasons unknown outside the
> bowels of One Infinite Loop, this was renamed to the
> far-less-appropriate COPYFILE_DISABLE as of the Leopard release of Mac
> OS X.  [1]
> 
> When the build triplet matches *-apple-darwin*, Automake could include
> in the distdir target code to set both of the undocumented environment
> variables to true.  That presumes no Automake clients find the
> extended attributes and resource forks in tarballs valuable.

Right.  And since we don't know that (well, at least I don't), it would
be prudent to make it configurable.  Well, we can just say that the poor
Mac users can configure it by setting COPY_EXTENDED_ATTRIBUTES_DISABLE
and/or COPYFILE_DISABLE in their environment in their shell startup
script, and stop worrying about any changes to Automake.  ;-)

Thanks,
Ralf

> [1] http://norman.walsh.name/2008/02/22/tar

Re: c file used as a make file

[Ralf Wildenhues]

Hello Paul,

* Paul Elliott wrote on Tue, Mar 22, 2011 at 03:01:49PM CET:
> On Tuesday, March 22, 2011 01:54:33 am Paul Elliott wrote:
> > One .c file is used as a .h file. That is, it is included by another .c
> > file and it should not be itself compiled. Why the author did this I do
> > not know, but I do not want to change the author's source or change or
> > rename any files.
> > 
> > How do I tell auto*tools to treat this file as a .h file? That is, do not
> > try to compile this file itself, and do include it in the files to be
> > distributed.

Instead of
  foo_SOURCES += treat-as-header.c

write
  EXTRA_DIST += treat-as-header.c

and be done with it.

> Can I write a rule for the .lo that has to be compiled with the wierd
> include that would explictly list the .c file as a dependancy? How would I do 
> that?

Is the .c file generated?  If yes, then try adding it to BUILT_SOURCES,
if not, then the automatic dependency tracking should pick up the
dependency for you when the object is built for the first time.

Hope that helps.

Cheers,
Ralf

Re: documentation on using automake to produce documentation

[Ralf Wildenhues]

Hello Paul,

* Paul Elliott wrote on Thu, Mar 31, 2011 at 02:25:13AM CEST:
> Is it documented any where how to do the comparatively simpler task of using 
> automake to produce and distribute documentation?
> 
> Just simple things like specifying simple rules to build. And how to specify 
> distribution directories for documentation.

I'm not quite sure where to start, your request is fairly vague.

Generally, a Makefile.am is also a Makefile.  Or, more precisely, most
of the stuff you write in Makefile.am gets copied verbatim into the
resulting Makefile.  So you can just write plain make rules for anything
not directly supported by Automake.  See 'info Automake Extending'.

In order to hook your rules to the standard targets, use *-local and
maybe *-hook targets.  So, to extend 'all', specify your own stuff as
prerequisite of all-local.  Similarly for 'dvi', 'pdf', 'html', 'ps'.
And of course 'clean' and 'distclean'.  To specify distribution
semantics, you can either use dist_ or nodist_ prefixes to existing
special variables, or use EXTRA_DIST for files or directories.
Installation semantics can be specified with noinst_ or with directory
prefix (foodir = ..., foo_DATA = ...), or hand-written
install-data-local rules.

If you have Texinfo documentation or man pages, see 'info Automake
Documentation'.

For LaTeX sources, there's currently a bug open on our debbugs where
Reuben is looking into better interfacing with latexmk and/or rubber.

Does this help?  If not, please be more specific.

Thanks,
Ralf

Re: Removing Mac OS X resource forks from distribution tarballs

[Ralf Wildenhues]

Hello Dave,

* Dave Hart wrote on Wed, Mar 30, 2011 at 11:06:02PM CEST:
> Right, one approach would be to run a dist-hook which strips all
> resource forks from distdir files.
> 
> An equally effective approach which Automake could potentially handle
> generally on Darwin would be to instruct tar to ignore resource forks
> when creating the tar file.

Please show how that would work.

> Or, if Darwin's tar can't do that but can
> extract resource forks as if your filesystem didn't support them,
> Automake could round-trip to remove them on Darwin (tar, extract with
> resource forks named ._filename, find | xargs to rm the ._* files, tar
> again).

That would incur additional cost the size of the whole source tree, not
just the size of the resource forks.  To me, that seems prohibitive for
large source trees, and moreso for all trees not affected, or not on
Darwin.

Thanks,
Ralf

bug#8365: 3 of 657 tests failed

[Ralf Wildenhues]

Hi Stefano,

* Stefano Lattarini wrote on Wed, Mar 30, 2011 at 06:47:20PM CEST:
> --- a/tests/aclocal6.test
> +++ b/tests/aclocal6.test

> @@ -57,6 +57,8 @@ cd build
>  ../configure
>  $MAKE
>  
> +# Modified configure dependencies must be newer than config.status.
> +$sleep
>  # Update an aclocal.m4 dependency, then make sure all Makefiles
>  # are updated, even from a sub-directory.
>  echo 'AC_DEFUN([SOME_DEFS], [MORE_DEFS])' > ../m4/somedefs.m4

> --- a/tests/subdir5.test
> +++ b/tests/subdir5.test

> @@ -67,6 +67,8 @@ $MAKE
>  # does it in the other way: it updates confiles.m4 (which is m4_included
>  # by configure.in there) after Makefile.am.
>  
> +# Modified configure dependencies must be newer than config.status.
> +$sleep

This hunk makes perfect sense to me, and is fine for maint,
but the other two do not make sense to me yet:
configure ensures that config.status is newer than itself,
thus configure must be older than any .m4 files that are
edited afterwards even without $sleep.

I've read the longish explanation of yours superficially;
if you think it addresses my concern please say so and I will
scrutinize it.

>  sed configure.tmp -e '/^AC_OUTPUT$/i\
>  AC_CONFIG_FILES([maude/Makefile])\
>  m4_include([confile.m4])\
> @@ -94,6 +96,8 @@ test -f maude/Makefile
>  # Then we add a new directory by modifying a file included (through
>  # `m4_include') by configure.in.
>  mkdir maude2
> +# Modified configure dependencies must be newer than config.status.
> +$sleep
>  cat >> confile.m4 << 'END'
>  AC_CONFIG_FILES([maude2/Makefile])
>  AC_SUBST([GREPME])

Thanks for looking into this,
Ralf

bug#8365: 3 of 657 tests failed

[Ralf Wildenhues]

* Ralf Wildenhues wrote on Wed, Mar 30, 2011 at 06:59:07PM CEST:
> * Stefano Lattarini wrote on Wed, Mar 30, 2011 at 06:47:20PM CEST:
> > --- a/tests/subdir5.test
> > +++ b/tests/subdir5.test
> 
> > @@ -67,6 +67,8 @@ $MAKE
> >  # does it in the other way: it updates confiles.m4 (which is m4_included
> >  # by configure.in there) after Makefile.am.
> >  
> > +# Modified configure dependencies must be newer than config.status.
> > +$sleep
> 
> This hunk makes perfect sense to me, and is fine for maint,

Never mind; this hunk has the same problem as the other two.
Cf. also the _long_ comment above it explaining why it should
be sufficient.

This comment would need fixing if it were wrong.

> but the other two do not make sense to me yet:
> configure ensures that config.status is newer than itself,
> thus configure must be older than any .m4 files that are
> edited afterwards even without $sleep.

Cheers,
Ralf

Re: Removing Mac OS X resource forks from distribution tarballs

[Ralf Wildenhues]

Hi Adam,

* Adam Mercer wrote on Wed, Mar 30, 2011 at 03:50:40PM CEST:
> When I build distribution tarballs on my Mac sometimes resource forks
> find their way into the tarball and then when they are extracted on
> another platform, Debian for example, these resource forks show up as
> separate files of the for ._*, e.g.:
> 
> ldg-cert-util-2.5/debian# ls -la
> total 68
> drwxr-xr-x 2 501 staff 4096 Mar 30 09:13 .
> drwxr-xr-x 5 501 staff 4096 Mar 29 20:54 ..
> -rw-r--r-- 1 501 staff  240 May 13  2010 ._compat
> -rw-r--r-- 1 501 staff  240 May 13  2010 ._control
> -rw-r--r-- 1 501 staff  240 May 13  2010 ._copyright
> -rw-r--r-- 1 501 staff  240 May 13  2010 ._install
> -rwxr-xr-x 1 501 staff  240 May 13  2010 ._rules
[...]
> Is there some rule that I can add that will strip any resource forks
> from files before adding them to the distribution?

dist-hook: remove-mac-resource-forks
remove-mac-resource-forks:
rm -f $(DISTDIR)/._*

?  I'm not sure what the exact glob for those resource forks is.

Cheers,
Ralf

Re: subdir-objects, dirstamps and --disable-dependency-tracking (bug?)

[Ralf Wildenhues]

Hi Ignacy,

* Ignacy Gawedzki wrote on Wed, Mar 30, 2011 at 12:03:09AM CEST:
> I suspect I found a bug in automake 1.11.1 .

Yes, you have.  Thanks for reporting it!

> I have a project in which some program has .ll and .yy sources that must be
> generated by flex and bison.  It also happens I need to have specific
> preprocessor options passed to that program using a _CPPFLAGS variable.  This
> makes everything produced for that program prefixed with the program name.
> Besides, I need to use the subdir-objects automake option.
> 
> Now the problem is that some subdirectories are not created automatically in
> $(builddir) as expected and I think I have found why.  It happens when
> configure is called with --disable-dependency-tracking, as is usually the case
> when called through debhelper.
[...]

> Whereas there is *no* rule of the kind :
> 
>   src/scanner.cc: src/$(am__dirstamp)

> Does anybody know of a clean fix/workaround?

Well, a workaround is to add the above line to your Makefile.am.

Cheers,
Ralf

Re: link dependency failed with make -j*

[Ralf Wildenhues]

Hello Vincent,

* Vincent Torri wrote on Mon, Mar 28, 2011 at 06:40:51AM CEST:
> On Mon, Mar 28, 2011 at 1:10 AM, Bob Friesenhahn <
> bfrie...@simple.dallas.tx.us> wrote:
> > On Sun, 27 Mar 2011, Vincent Torri wrote:
> >> I have a library  that depends on several libraries, these latter ones
> >> must be built first:
> >>
> >> pkg_LTLIBRARIES = pdf.la
> >>
> >> noinst_LTLIBRARIES = libfitz.la libdraw.la libcmaps.la libfonts.la
> >> libmupdf.la
> >>
> >> Does someone know what i should do to enable parallel compilation ?
> >
> > Maybe you should make pdf.la dependent on ibfitz.la libdraw.la ...?
> you mean:
> 
> pdf_la_DEPENDENCIES = libfitz.la libdraw.la ... ?
> 
> I thought that adding libfitz.la libdraw.la to pdf_la_LIBADD was sufficient

It is sufficient, but you may not spell the dependency as
  pdf_la_LIBADD = $(top_builddir)/.../foo.la ...

but should spell it as
  pdf_la_LIBADD = foo.la ...

because 'make' does not flatten or normalize names when comparing
targets or dependencies.

Cheers,
Ralf

Re: [GSoC Proposal] automake - Interfacing with a test protocol like TAP or subunit

[Ralf Wildenhues]

* Stefano Lattarini wrote on Sun, Mar 20, 2011 at 01:41:07PM CET:
> On Sunday 20 March 2011, Ralf Wildenhues wrote:
> > Or add a subunit parser and a quick tap2subunit perl module today
> ["perl module"? what about portability?]

awk should be sufficient, for text-mode output at least.

> > and have the best of both worlds?  (This is meant as an honest question,
> > even if it looks like a rhetoric one.)
> >
> I'd rather add a SubUnit parser only when *and if* the need arise;

But look, going the other way round, there is little change you'll ever
need to introduce an incompatible change later.

> I have to admit that, by reading more carefully the README of subunit,
> I'm intrigued by the fact that there seem to already be producers for
> C, C++, Python, Perl and the shell... Still, I'm not confartable with
> not being able to find documentation and examples that are clear enough
> to allow me to define proper goals and progress estimation.

This is fixable though, no?

As far as I can see it, so far the arguments pro TAP are/were:
- perl, rather than only python, support exists (but cf. your statement
  above),
- simplicity,
- well-definedness of the protocol.

I don't see how subunit is not well-defined, looking at the EBNF in the
README.  I understand your desire to tackle simple first, and have
something you can already overlook mostly.  But summer is long, and even
if it turns out too short: it's not necessary to finish exactly in time.

I'm a bit intrigued by the fact that subunit appears to address a few
deficiencies in TAP, and wonder whether we'd regret choosing TAP later.
That said, I guess I'm fine with a project proposal along the lines of
"try subunit support; and if that turns out too hard (e.g., for
portability issues), then fall back to TAP", if you prefer a safety net.

Cheers,
Ralf

Re: [GSoC Proposal] automake - Interfacing with a test protocol like TAP or subunit

[Ralf Wildenhues]

* Stefano Lattarini wrote on Sun, Mar 20, 2011 at 11:40:39AM CET:
> On Sunday 20 March 2011, Robert Collins wrote:
> > On Sun, Mar 20, 2011 at 8:53 PM, Ralf Wildenhues wrote:
> > > Are TAP and subunit compatible on their common subset?  If not, why not?
> > 
> > You can convert TAP to subunit, and you can convert the things TAP can
> > represent from subunit to TAP. subunit's core is more structured than
> > TAP, so the two protocols don't pun as each other at all.
> >
> If I'm not misreading the class TAP2SubUnit in python/subunit/__init__.py,
> converting from TAP into subunit shouldn't be much difficult, even with
> just the POSIX tools available.  This means that we could add support for
> TAP "today", and if "tomorrow" we decide (or our users tell us) to start
> supporting also SubUnit, we could add a SubUnit parser, and then continue
> to support also TAP by writing a TAP->SubUnit converter and filtering TAP
> output into it -- which would allow us to drop the TAP parser (less code
> to maintain) without losing backward-compatibility.

Or add a subunit parser and a quick tap2subunit perl module today
and have the best of both worlds?  (This is meant as an honest question,
even if it looks like a rhetoric one.)

Thanks,
Ralf

Re: [GSoC Proposal] automake - Interfacing with a test protocol like TAP or subunit

[Ralf Wildenhues]

* Robert Collins wrote on Sun, Mar 20, 2011 at 10:03:10AM CET:
> On Sun, Mar 20, 2011 at 8:53 PM, Ralf Wildenhues wrote:
> > * Robert Collins wrote on Sun, Mar 20, 2011 at 05:10:16AM CET:
> >> TAP is an extremely simple protocol, and the extensions to it to
> >> support things like not needing to maintain the count of tests,
> >> additional debug data and so on are pretty rudimentary. subunit, which
> >> I've mentioned before was written after TAP, to solve similar problems
> >> and address the issues in TAP itself.
> >
> > Are TAP and subunit compatible on their common subset?  If not, why not?
> 
> You can convert TAP to subunit, and you can convert the things TAP can
> represent from subunit to TAP. subunit's core is more structured than
> TAP, so the two protocols don't pun as each other at all.

Are there existing converters?

> >> Unlike TAP subunit supports attachments (binary and text) to tests,
> >
> > Handling of binary data may end up being quite tricky within a
> > restricted Posix environment with only a few tools available.
> > What if a consumer cannot handle them?  Is there possibility
> > for graceful fallback?
> 
> I didn't think posix had 7 bit limits?

Well, I meant generally handling binary data with shell tools only.
Analyzing binary data with portable shell tools can be tricky.

> Anyhow, there is a baseline
> profile which assumes just a single description of the error in a test
> - it uses \r] to delimit a traceback. Alternatively, a C parser - on
> my 'sometime' TODO list - will probably clock in small enough to
> bundle for projects very low in the dependency stack.

It would be nice if autotools could cope without compiled code.  That
leads to complications when $CC is a cross-compiler.  But I see in the
link that seemingly a shell consumer already exists.  So maybe my
worries are moot.

Is there any way we can measure existing usage of TAP or subunit that
doesn't rely on hearsay?

Thanks,
Ralf

Re: GSoC project idea: non-recursive automake project

[Ralf Wildenhues]

Hello Pippijn,

* Pippijn van Steenhoven wrote on Sat, Mar 19, 2011 at 10:47:35AM CET:
> On Sat, Mar 19, 2011 at 10:38:39AM +0100, Pippijn van Steenhoven wrote:
> > On Fri, Mar 18, 2011 at 05:26:58PM -0700, Harlan Stenn wrote:
> > > If there was a student interested in showing how "easy" it was to use
> > > automake to do non-recursive Makefiles for a project, I'd be willing to
> > > co-mentor and work with them to convert NTP to that sort of operation.
> > 
> > It's mostly trivial. How hard are GSoC projects supposed to be?
> 
> Being a student, I'd be willing to prove it ;)

If you would like to apply as student for this GSoC project, can you
please formulate a project proposal (as is needed for a GSoC application
anyway) and post it here?  Please take into account what else was said
in this thread about having enough work for a SoC term.  (We can help
with more ideas about how one could extend Automake if that's unclear.)

Harlan's description of the task will soon appear in the archives of
http://lists.gnu.org/archive/html/summer-of-code/2011-03/threads.html
and eventually also on
http://www.gnu.org/software/soc-projects/ideas-2011.html
but it is still fairly vague.

Then, I remember reading you on this list before, but not yet much in a
development role.  I have no idea whether you know the autotools code or
NTP well.  Now, I don't want to put you on the spot, and prior
development experience is not a requirement for GSoC applications, but
if your proposal is going to encompass hacking on NTP and/or Autotools,
it would help to see or be able to judge your coding and working
together skills in some way or other.

You could help us with something like looking at the debbugs for
Automake and working on some bug (or even just outlining strategies to
do so); or addressing an issue that somebody reported on the mailing
list; or enhancing the documentation in some way; or similar for NTP.
There's no need for you to find out everything (or even much) yourself.
In fact, if you ask questions on the way, that will probably make things
easier for us.  If you don't see something feasible to do, we can
probably also come up with a small (maybe 1-2 hr) task.

Finally, I should add that I'm off-list for the second half of this week
(starting Wednesday; but I should be back sometime next weekend) and
otherwise usually read mails twice a day, so expect some latency.

Thanks,
Ralf

Re: [GSoC Proposal] automake - Interfacing with a test protocol like TAP or subunit

[Ralf Wildenhues]

Hi Robert,

thanks for the feedback.  I have a couple of questions:

* Robert Collins wrote on Sun, Mar 20, 2011 at 05:10:16AM CET:
> TAP is an extremely simple protocol, and the extensions to it to
> support things like not needing to maintain the count of tests,
> additional debug data and so on are pretty rudimentary. subunit, which
> I've mentioned before was written after TAP, to solve similar problems
> and address the issues in TAP itself.

Are TAP and subunit compatible on their common subset?  If not, why not?

> Unlike TAP subunit supports attachments (binary and text) to tests,

Handling of binary data may end up being quite tricky within a
restricted Posix environment with only a few tools available.
What if a consumer cannot handle them?  Is there possibility
for graceful fallback?

> test naming, tagging, timestamping (permitting robust timing data even
> in parallelised or distributed testing).
> 
> Integrating with TAP is basically uninteresting to anyone working in a
> high level language: Python, Ruby,Java, C++ etc.

Well, Perl is a high-level language too, I'd say.  ;-)

Thanks,
Ralf

Re: GSoC project idea: non-recursive automake project

[Ralf Wildenhues]

Hi Harlan,

* Harlan Stenn wrote on Sat, Mar 19, 2011 at 01:26:58AM CET:
> If there was a student interested in showing how "easy" it was to use
> automake to do non-recursive Makefiles for a project, I'd be willing to
> co-mentor and work with them to convert NTP to that sort of operation.

Thanks for the co-mentoring offer and the SoC idea!

I have a question though: how much work do you expect this to be?
Haven't looked at NTP in a long time, but typically, turning a project
into non-recursive was either a straightforward to trivial task of
maybe 1-2 days for somebody experienced with autotools, or something
difficult to impossible due to limitations in either of Make, Automake,
or third-party bits.

Maybe such a proposal could be enhanced to avoid having not enough work:
For example, while converting NTP, the student could start a document
with a general recipe for this conversion.  And then maybe try it out on
a couple more projects, and possibly refine the recipe along the way.
For students that get very far, they could also try working on
limitations in other tools should they come across them.

This way the student will not get bored.  However, it might be harder
to define specific goals to achieve, or to define success in the end.

Would you be willing to formulate this as a proposal for the GNU
proposals wiki page?

I should note that I certainly don't have unlimited mentoring time.
I expect to be able to mentor one student, and I'm sure co- or backup-
mentoring beside that should be possible, but if we can find another
person to help that would only be good.

Cheers,
Ralf

Re: Do convenience libraries propagate -R linker flags?

[Ralf Wildenhues]

Hello Rhys,

* Rhys Ulerich wrote on Mon, Mar 14, 2011 at 05:34:04PM CET:
> I've got a convenience library where the LDFLAGS includes -R:
> noinst_LTLIBRARIES = libsomething.la
> ...
> libsomething_la_LDFLAGS = -Rsomewhere
> and I indeed see "-Rsomewhere" appear within libsomething.la's dependency_libs
> line the way I expect.
> 
> In another directory, I build a program depending on libsomething.la
> bin_PROGRAMS = myprog
> myprog_LDADD = ../lib/libsomething.la
> and then issue 'make install' to install it.
> 
> After installation, I'm not seeing '-Rsomething' appear within the installed
> binary's rpath according to 'readelf -d myprog'. 'ldd myprog' issued after
> emptying my LD_LIBRARY_PATH confirms that -Rsomething isn't present.
> 
> Should I expect -R flags provided at convenience library creation time to be
> propagated to an installed binary which uses that convenience library?  If
> not, I'll stop trying.  If so, I'll see if I can get a recreate for what
> I'm observing.

I think they are not propagated currently for convenience archives.  As
to whether they should, I'm not sure.  I remember that this has been
reported before.  I also remember that the issue has not been entirely
noncontroversial.

Somebody would have to dig in mailing list and commit history (of the
Libtool project) to find out for sure.

Cheers,
Ralf

Re: ACLOCAL_PATH patch?

[Ralf Wildenhues]

* Stefano Lattarini wrote on Tue, Mar 15, 2011 at 10:36:28PM CET:
> On Tuesday 15 March 2011, Harlan Stenn wrote:
> > http://www.mail-archive.com/automake-patches@gnu.org/msg00873.html
> > describes a patch I once asked for.  I did a search in the
> > automake-1.11.1 codebase and I don't see any mention of ACLOCAL_PATH in
> > there at all.
> >
> Correct: the patch has never been applied.
> 
> BTW, that old patch is supersed by this newer patch series:
>  
> which is unfortunately still pending review (or at least aknowledgement).

I'll try to look at it soonish.

Sorry for the long delays,
Ralf

Re: Mucking with ltmain.sh

[Ralf Wildenhues]

Hello Rhys,

* Rhys Ulerich wrote on Fri, Mar 11, 2011 at 09:03:06PM CET:
> Can anyone recommend either a clean way to perform the ltmain.sh
> surgery at bootstrap?

Most needed surgery at bootstrap can be done by having a ./bootstrap
script that contains something like

#! /bin/sh

autoreconf -vi

patch -p0 <<\EOF
--- ltmain.sh~
+++ ltmain.sh
... unidiff ...
EOF

because it will stop modifying code once it is fixed.  Be sure to check
exit status semantics of patch.

The actual issue should be fixed in Libtool too, of course.

Thanks,
Ralf

Re: Automake GSoC idea

[Ralf Wildenhues]

[ adding the SoC list back in Cc: ]

Hello,

* Daniel Herring wrote on Wed, Mar 09, 2011 at 08:06:45AM CET:
> Refactor Automake so it can easily be extended for new file types.

Yes, this is a good idea, too.  Actually, there is a pending patch from
Valentin that does an important first step in this area; I wasn't sure
whether he'd pick up work on this again eventually.  Valentin?

> Examples:
> - Qt preprocessed files (e.g. moc and linguist generate sources to link)
> - IDL files (may generate headers and several sources to link)
> - NVidia's CUDA

Yep.  Actually, we even have proposed patches for QT and three for CUDA.
The QT ones would need some love (more importantly, review), none of the
CUDA ones can go in because of copyright assignment issues.  :-(

> P.S.  I'm employed, but not with enough free time to mentor this.  I
> could help by providing case studies of tasks that aren't easy with
> automake.

Unfortunately, I still lack final confirmation that I can be mentor for
this.  In the case that I can't, we'd still need somebody for this role.
Eric has agreed to be my backup mentor.

Cheers,
Ralf

Re: PKG_CHECK_MODULES on system without pkg-config installed?

[Ralf Wildenhues]

Hello,

* Roger Leigh wrote on Thu, Mar 10, 2011 at 01:03:16PM CET:
> This is not meant to sound like a troll, but: is anyone really
> *really* using static linking in 2011?

I'd love to answer no, but at least parts of the HPC crowd will do
almost anything to get a couple percent more performance out of their
code plus libraries.  Static linking is a real low-hanging fruit there.

Cheers,
Ralf

Re: Autotools GSoC ideas

[Ralf Wildenhues]

* Stefano Lattarini wrote on Tue, Mar 08, 2011 at 10:50:47PM CET:
> On Tuesday 08 March 2011, Robert Collins wrote:
> > On Wed, Mar 9, 2011 at 8:39 AM, Stefano Lattarini wrote:
> > >  ``Interfacing with the Test Anything Protocol (TAP).  If possible, try
> > >   to write an implementation that will allow future extensions to
> > >   similar but more advanced advanced protocols (e.g., subunit, which
> > >   is similar to TAP but slightly more structured, capable of handling
> > >   binary attachments, and so on).''
> > 
> > You could - or you could just write to the most capable and let folk
> > insert a filter (e.g. tap2subunit, included in the subunit package) if
> > they are using a different protocol themselves.
> >
> This seems a good approach from a design point of view; unfortunately, the
> existing filters in the `subunit' distribution all require python, which
> hampers portability in a way unacceptable for automake.

I don't think we need to do this kind of evaluation or decision at
this point yet.  It could be done as part of the project, or maybe a
project proposal.  Also, just because there may be only python filters
ATM, doesn't mean that needs to remain this way.

I've added the proposals to the wiki now.

Thanks,
Ralf

Re: Autotools GSoC ideas

[Ralf Wildenhues]

Hi Robert,

* Robert Collins wrote on Tue, Mar 08, 2011 at 02:57:44AM CET:
> On Tue, Mar 8, 2011 at 1:10 AM, Ralf Wildenhues wrote:
> > I'll throw a couple of suggestions for Autotools out there:
> >
> > 1) Interfacing with the Test Anything Protocol (TAP) (or maybe another
> > test protocol?).

> I've mentioned subunit before - its similar to TAP but slightly more
> structured and capable of handling binary attachments and so on.

Right, I forgot about the name, _again_.  Thanks for reminding me.

So let's extend the suggestion to include evaluating the best protocol
(or set of protocols) to use/interface to.

Cheers,
Ralf

Autotools GSoC ideas

[Ralf Wildenhues]

Hello,

I'll throw a couple of suggestions for Autotools out there:

1) Interfacing with the Test Anything Protocol (TAP) (or maybe another
test protocol?).  Automake-generated Makefiles could be consumers of the
protocol for third-party testsuites.  Automake-generated simple
testsuites (TESTS) and Autoconf-generated Autotest testsuites could be
producers.

This project would entail both investigating the possible advantages
of either step, prototyping an implementation, and getting it ready
to merge into the code.  Each step on its own would already be useful
on its own, and small enough to be completed within the given time
frame.

2) Automake can (optionally) exploit more features of GNU make,
with an eventual goal of reimplementing some of the functionality
in GNU make, both for efficiency and for more ease of use.  When
a project using Automake requires GNU make anyway, it would be nice
if Automake could play along better with GNU make-specific notation
in the Automake special variables (like foo_SOURCES etc).

I have some preliminary patches to get this started, which could act
as a good starting point for further developments.


There are certainly more projects I could think of ...

Thanks,
Ralf

Re: about what goes into a distribution file

[Ralf Wildenhues]

Hello Reuben,

* Reuben Hawkins wrote on Thu, Feb 24, 2011 at 05:33:07AM CET:
> The attached project (a trivial flex scanner) built with "make dist"
> includes scanner.c (generated by flex from scanner.l).  Why is scanner.c
> included?

Quoting automake.info:

|You should never explicitly mention the intermediate (C or C++) file
| in any `SOURCES' variable; only list the source file.
| 
|The intermediate files generated by `yacc' (or `lex') will be
| included in any distribution that is made.  That way the user doesn't
| need to have `yacc' or `lex'.

>  Can I somehow prevent it from being include without setting up a
> dist-hook rule?

Not with the current version of Automake, sorry.  I recommend that you
document the Lex requirement for your users, and only require your users
to have Lex if there is no other way.

If you like, you can file an enhancement bug on the bug-automake list
that allows for this; please include a rationale, so we can judge
importance of this feature.  I'm not sure if automake should detect non
distribution from scanning CLEANFILES.  Stefano is more into the lex and
yacc business ATM, he might have some input on this.

Thanks,
Ralf

Re: dynamic executables for check_PROGRAMS?

[Ralf Wildenhues]

Hi Jeff,

* Daily, Jeff A wrote on Fri, Feb 25, 2011 at 07:04:32PM CET:
> I've attached a tiny example.  It consists of three programs:
> main_bin, main_check, main_nofoo_bin.  Here's the Makefile.am:

> bin_PROGRAMS = main_bin main_nofoo_bin
> check_PROGRAMS = main_check
> lib_LTLIBRARIES = libfoo.la

> main_nofoo_bin_LDADD =
> LDADD = libfoo.la
> 
> Here are the relevant pieces of configure.ac.  I know some of the
> automake settings don't really apply to this simple test, but they're
> the ones I use in my software package so I hope to reproduce the
> conditions as completely as possible:

> When I mkdir bld && cd bld && ../configure --disable-static
> --enable-shared --prefix=`pwd` && make && make check && make install,
> I get a main_bin, main_check, and main_nofoo_bin in my build
> directory.  When I run ldd main_bin or ldd main_check I get "not a
> dynamic executable".  When I run "ldd main_nofoo_bin" or when I run
> "ldd bin/main_bin" I get the listing of shared libraries as expected,
> so somehow I get a dynamic executable if I don't link to libfoo.la or
> if I install the program.  But if I link to libfoo.la, prior to
> installing, my binaries are statically linked?

Ahh, now I get what you mean.

No, your binaries are not linked statically before installation.
But they are hidden in the .libs directory, and what you see is a
(libtool-generated) shell wrapper that invokes the hidden programs
(after setting some environment variables so the uninstalled libraries
are found and used).  Also, it may be that they are relinked at 'make
install' time by libtool.

Hope that helps.  If this explanation doesn't suffice, then please
explain what you aim to do that currently doesn't work.

Thanks,
Ralf

Re: debbugs, and a FAQ, for Autotools

[Ralf Wildenhues]

* Eric Blake wrote on Thu, Feb 24, 2011 at 01:04:31AM CET:
> On 02/23/2011 05:02 PM, Ben Pfaff wrote:
> > The question "libtool reorders link fags" seems to have a
> > spelling error in the last word.  It's not obvious to me what
> > word is meant.
> 
> flags

Fixed now, thanks for the report.
Ralf

Re: debbugs, and a FAQ, for Autotools

[Ralf Wildenhues]

Hello,

* Miles Bader wrote on Wed, Feb 23, 2011 at 05:17:29AM CET:
> Ralf Hemmecke writes:
> > Sure. But it is also relevant if one developer adds a macro which is
> > only available in some recent version of automake, say. Another
> > developer might not yet have that automake version.
> 
> It doesn't really seem any worse than _any_ potential tool
> incompatibility problem -- compiler version, library version, etc --
> though...
> 
> Usually those issues aren't such a huge deal, because most project try
> to be relatively portable, and when version dependencies do crop up,
> they can be dealt with relatively well using simple checks in the
> configure script.
> 
> Isn't that what people usually do about autoconf versions too (declare a
> minimum version in configure.ac)?

Yes.  You can even declare a maximum (or exact) version, too, if that is
what you need.  We did that in GCC, see the _GCC_AUTOCONF_VERSION_CHECK
macro in http://gcc.gnu.org/viewcvs/trunk/config/override.m4?view=markup
and with Automake, it should work at least to test $am__api_version at
configure time.  I guess we should make it possible to test easily at
m4 run time too ...

(and I guess this is worth another FAQ entry ...)

Cheers,
Ralf

Re: debbugs, and a FAQ, for Autotools

[Ralf Wildenhues]

Hello Ralf,

* Ralf Hemmecke wrote on Tue, Feb 22, 2011 at 10:30:21PM CET:
> I'm just reading the current FAQ under
> 
> 1.3 Where can I get the latest versions of these tools?
> 
> http://www.gnu.org/software/automake/faq/autotools-faq.html#Where-can-I-get-the-latest-versions-of-these-tools_003f
> 
> Wouldn't it be useful to give a little script that installs
> know-good combinations (the latest version?) of all the autotools
> components in the users (developers) HOME?
> 
> You probably can do
> 
> https://github.com/hemmecke/flint2/blob/autotools/autotools.mk
> 
> better than me, but it might be a first start.

Well,
http://www.gnu.org/software/automake/faq/autotools-faq.html#How-do-I-install-the-Autotools-_0028as-user_0029_003f
is not as cool as your version, but provides roughly the same
functionality AFAICS.

I think either of these two improvements would be nice:

- having a list of known-good combinations of tool versions; having
  tested them extensively would be great, otherwise we might at least
  document some sort of minimal-version-requirement-table.
- a change that lets the script automatically install the most recent
  stable releases of each tool.

Cheers,
Ralf

Re: debbugs, and a FAQ, for Autotools

[Ralf Wildenhues]

* Ralf Hemmecke wrote on Tue, Feb 22, 2011 at 10:41:20PM CET:
> http://www.gnu.org/software/automake/faq/autotools-faq.html#How-do-I-add-a-question-to-this-FAQ_003f
> 
> Do you think, it would be a good idea to just open up a git repo (on
> github.com, for example) and put the autotools-faq.texi file there?
> Or is there already a git repo for this?

Maybe.  It is currently in the web pages CVS, so an export from there
wouldn't be hard either.  The savannah page for automake shows how to
access the CVS (a patch to add that bit of info to the FAQ would be
welcome ;-)

I'm not sure if git is needed here (other than that it of course would
be nice if all webpage stuff was in git anyway), since I hope we don't
need diverging or long-time developments in the FAQ.  ;-)

Thanks,
Ralf

Re: debbugs, and a FAQ, for Autotools

[Ralf Wildenhues]

Hi Eric,

* Eric Blake wrote on Mon, Feb 21, 2011 at 03:24:01PM CET:
> On 02/13/2011 11:12 AM, Ralf Wildenhues wrote:
> > 1) Autoconf and Libtool should also use debbugs.
> > 
> > bug-automake has switched a few months ago, and I find it helpful to
> > avoid losing reports.  Given that we never have enough time on our
> > hands, it becomes more important to not lose track.
> > 
> > See http://debbugs.gnu.org/ and linked pages for details.
> 
> Seems like it might be reasonable for autoconf.  However, my biggest
> concern is that right now, I filter both autoconf and automake messages
> into the same mail folder, but debbugs anonymizes which list a bug is
> being reported against (that is, the To: is rewritten as
> ###@debbugs.gnu.org, so there is no longer any mention of 'automake' in
> any of the normal mail headers).  While I could probably force filtering
> to take place on X-Debbugs-Original-To:, that still doesn't solve the
> problem of no visual indication on which list a bug is reported against.

Can you show Sender: information by your mail program?
That's what I use both for show and filter.

This seems like a problem that is more general than just the question of
whether Autoconf or Libtool should use debbugs.  IOW, if the above is
not sufficient, then debbugs should provide help.  (Or we could also
patch mail user agents.  ;-)  I'm adding Glenn in Cc:, in case he has
further input on this.

Thanks,
Ralf

Re: dynamic executables for check_PROGRAMS?

[Ralf Wildenhues]

Hello Jeff,

* Daily, Jeff A wrote on Fri, Feb 18, 2011 at 12:06:13AM CET:
> I wrote a profiling layer for my library utilizing weak symbols.  I
> thought for starters it would be nice to profile some of my test
> programs, to make sure things are working okay.  I'm using autoconf,
> automake, and libtool, so I configured using --enable-shared
> --disable-static, however, my test programs are not created as dynamic
> executables.

I'm not sure I understand this.  Programs are typically not created
differently based on whether they are check_PROGRAMS or bin_PROGRAMS.
Maybe you are thinking of uninstalled libraries which those programs
depend on and which are noinst_LTLIBRARIES thus only generated as
convenience archives?  Here's how you can let them be shared libraries:

noinst_LTLIBRARIES = libtesthelper.la
libtesthelper_la_LDFLAGS = -rpath /nowhere

The actual path you specify is not important, except that it should not
be world-accessible because the system might try to find files there.

If I have misunderstood your question, the maybe it helps to show a
small example?

Thanks,
Ralf

Re: about what goes into a distribution file

[Ralf Wildenhues]

Hi Vincent,

* Vincent Torri wrote on Fri, Feb 18, 2011 at 08:33:01AM CET:
> from the automake documentation, all the files read by AC_CONFIG_FILES are
> added in the distribution. But why aren't the generated files not in the
> tarball (the .pc.in are in but not the .pc for example) ?

Just to answer the question for whoever else stumbles over this:
because the point of generating them is that they may look differently
on different systems.  Thus it makes no sense to distribute them.

And actually, the rules are a bit more complicated: the .in files are
distributed only if they are mentioned literally in configure.ac (i.e.,
not as shell variable), and when they are not the output file of another
AC_CONFIG_FILES command (e.g., foo.in.in -> foo.in -> foo).  The gory
details are documented here:
http://www.gnu.org/software/automake/manual/html_node/Requirements.html

Cheers,
Ralf

Re: debbugs, and a FAQ, for Autotools

[Ralf Wildenhues]

Hi Russell,

* Russell Shaw wrote on Mon, Feb 14, 2011 at 12:00:14AM CET:
> I'd ask more about how the internals of ./configure and autoconf works.

Can you formulate more specific questions?

> And questions on how to make bison get handled without being forced to
> mimic standard yacc.

I've added that question there now (without answer so far).

Thanks,
Ralf

Re: debbugs, and a FAQ, for Autotools

[Ralf Wildenhues]

Hello everyone,

* Ralf Wildenhues wrote on Sun, Feb 13, 2011 at 07:12:02PM CET:
> 2) Autotools should have a FAQ document.

I've done a brain dump now, here's a rough initial version:
http://www.gnu.org/software/automake/faq/autotools-faq.html and
http://www.gnu.org/software/automake/faq/autotools-faq.texi
(the former is generated from the latter via the Makefile).
It is all incomplete, buggy, partly ugly, probably missing
gnulib and gettext references in places, and everything, but
I would like something to point users to now.  Should all be
reformulated nicely eventually ...

I'd be happy about feedback, help, anything.  For now there
are several more questions than answers, and there are
certainly many more questions that you can think of, so
please write them.

I don't actually care about the license, as long as we can
move text between FAQ and manuals without too much hassle.

Thanks,
Ralf

Re: debbugs and (was: slow "make clean")

[Ralf Wildenhues]

* Glenn Morris wrote on Sun, Feb 13, 2011 at 09:01:24PM CET:
> 
> Ralf Wildenhues wrote (on Sun, 13 Feb 2011 at 19:01 +0100):
> 
> > I glanced over the debbugs documentation at debbugs.gnu.org now, but
> > couldn't find a place that said something about subscribing to
> > individual bugs.  Glen, do you happen to know more about this?
> 
> I know that it doesn't work, and personally I don't know how to
> implement it. :(
> 
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5439

Here's a Debian PR with discussion, and a patch and description:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=34071#62
Does that help any (haven't looked closely)?

Thanks,
Ralf

Re: Test support for automake

[Ralf Wildenhues]

* NightStrike wrote on Sun, Feb 13, 2011 at 02:10:32PM CET:
> On Sat, Feb 12, 2011 at 4:51 PM, Daily, Jeff A wrote:
> >> I have a non-recursive Makefile.am with which I build a library foo. I
> >> list all my tests (quite a lot) in check_PROGRAMS.
> >>
> >> All those tests should (of course) get libfoo.la as LDADD, but how
> >> exactly do I do that _without_ doing a per-target *_LDADD specification.
> >> Per target specification sounds like a lot of redundancy since all my
> >> _LDADD specification would just be libfoo.la.
> >> I'd like to add libfoo.la in only _one_ place.

> > Use the LDADD variable.  It's similar to the AM_* variables e.g.
> > AM_LDFLAGS, AM_CPPFLAGS.  All programs use LDADD for linking unless
> > it is overridden by a maude_LDADD per-target variable.

> It sounds like this is a good use case for support of check_LDADD,
> which of course is going to imply a request for generic x_Y support,
> where x is any prefix (bin, lib, etc) and Y is any suffix (cflags,
> etc).

Such flags are a big problem to get right.

What if
  check_PROGRAMS = foo
  bin_PROGRAMS = foo

happens?  Which rule should win?  I don't think we even forbid this kind
of setup, just like we don't forbid sources being listed for more than
one program or library.

Sticking special flags or LDADDs to specific targets OTOH is always
well-defined.

Cheers,
Ralf

debbugs, and a FAQ, for Autotools

[Ralf Wildenhues]

[ Cross post; Reply-To and Mail-Followup-To set.  Please followup to
  the automake list only, to avoid excessive spammage.  Thank you.  ]

Hello everyone,

I've been advertising debbugs before, I think we should be a good
example.  So, two proposals:

1) Autoconf and Libtool should also use debbugs.

bug-automake has switched a few months ago, and I find it helpful to
avoid losing reports.  Given that we never have enough time on our
hands, it becomes more important to not lose track.

See http://debbugs.gnu.org/ and linked pages for details.

2) Autotools should have a FAQ document.  Not of the sort of the FAQ
chapter that answers seven random questions and that has a 1 year+
latency until it is updated, but one that answers both totally-newbie
kinds of questions that get asked over and over again, or cross-tool
bug questions like the infamous libtool echo problem (which was due to
an incompatible m4sugar change).  A document that, ideally, eventually
obsoletes many of the third-party "here's how autotools work, in quick"
kinds of pages.

See e.g., this most recent thread which made the need so clear again:
http://thread.gmane.org/gmane.comp.sysutils.automake.patches/5672


For (2) I'd suggest a wiki if we GNU the infrastructure, but something
like a new page http://www.gnu.org/software/automake/Autotools-FAQ.html
would certainly be good.  (And yes, I've been arguing against wikis in
the past.  I was wrong.  Sue me.)


Now, I have very little spare time on my hands.  Any volunteers on
managing such a document?  Any people interested in contributing answers
or even only questions?  I wouldn't mind handing out commit privs to any
of the regulars on these lists.

Thanks for feedback,
Ralf

debbugs and (was: slow "make clean")

[Ralf Wildenhues]

* Ralf Hemmecke wrote on Sun, Feb 13, 2011 at 06:46:12PM CET:
> >We have a bug tracker.  Just write to the bug-automake list.
> >This is documented at
> >http://www.gnu.org/software/automake/manual/html_node/Reporting-Bugs.html
> 
> Oh yes, but it's not completely clear to me whether I get only an
> email for a bug that I submitted or whether I can explicitly can
> subscribed to some specific bug-thread.

First off, you do not need to be subscribed to bug-automake in order to
post there.  We have a habit of Cc:ing the author (and rest of people
already in Cc:) when replying, so non-subscribers can follow the
discussion.  We don't always get this completely right, and people
replying via web interfaces a la gmane can't easily follow this, but
usually it works OK.

I glanced over the debbugs documentation at debbugs.gnu.org now, but
couldn't find a place that said something about subscribing to
individual bugs.  Glen, do you happen to know more about this?

> ...similar to what for example the roundup-tracker can do. The
> python people use it. The "Nosy list" is what I am referring to.
> 
> http://bugs.python.org/issue1294032
> 
> I think I would get too many (to me irrelevant) mails if I subscribe
> to the bug-automake mailing list.

Sure.  It's not too high traffic though.

Cheers,
Ralf

Re: slow "make clean"

[Ralf Wildenhues]

* Ralf Hemmecke wrote on Sun, Feb 13, 2011 at 03:10:39PM CET:
> >You mean, as a workaround until Automake is improved?  You could
> >overwrite the rule in your Makefile.am:
> >
> >mostlyclean-compile:
> > -find . -name \*.lo -o -name \*.$(OBJEXT) -print | xargs rm -f
> >
> >But note that automake may generate other files that would need to be
> >mostlyclean'ed too, so the above could be insufficient for your project.
> 
> Thanks, that seems to work as a workaround.

(BTW, just in case anyone is wondering now: the above rule is not
suitable for use in Automake because it can be severely suboptimal
if, say, only a couple of objects are built in the current directory
but there is a huge build directory tree which find would traverse.)

> But, of course, it would be
> good, if I knew what will be generated, so that I can clean everything even
> when Makefile.am changes slightly. Is there a variable that already contains
> every name of MAKE-generated files?

Nope, sorry.

> >Thanks for the report,
> 
> No problem. But I was a bit surprised that I haven't found any proper
> bugtracker. Now, I have to subscribe to automake-patches and try to figure
> out when automake will have improvements that fix the slow "make clean"
> problem instead of just subscribing to the specific bug report.

We have a bug tracker.  Just write to the bug-automake list.
This is documented at
http://www.gnu.org/software/automake/manual/html_node/Reporting-Bugs.html
which is linked from the Automake home page
http://www.gnu.org/software/automake/

Cheers,
Ralf

Re: turning off dependency tracking when building fat binaries on Mac OSX

[Ralf Wildenhues]

Hello Peter,

* Peter Johansson wrote on Sat, Jan 22, 2011 at 05:18:10PM CET:
> The `INSTALL' file provided by autotools has the following blurp
> 
> >   On MacOS X 10.5 and later systems, you can create libraries and
> >executables that work on multiple system types--known as "fat" or
> >"universal" binaries--by specifying multiple `-arch' options to the
> >compiler but only a single `-arch' option to the preprocessor.  Like
> >this:
> >
> > ./configure CC="gcc -arch i386 -arch x86_64 -arch ppc -arch ppc64" \
> > CXX="g++ -arch i386 -arch x86_64 -arch ppc -arch ppc64" \
> > CPP="gcc -E" CXXCPP="g++ -E"
> >
> I noticed that when configuring like that automake automagically
> turns off dependency-tracking, for good reasons.
> 
> However, it is not uncommon that people instead issue
> 
> ./configure CFLAGS="-arch i386 -arch x86_64 -arch ppc -arch ppc64" \
> CXXFLAGS="-arch i386 -arch x86_64 -arch ppc -arch ppc64" \
> LDFLAGS="-arch i386 -arch x86_64 -arch ppc -arch ppc64"
> 
> which will not works. See for example
> 
> http://www.mail-archive.com/help-gsl@gnu.org/msg03121.html

Yep.  I think that was one reason we explicitly documented the thing in
INSTALL.  With this configuration, your preprocessor will be wrong
(because CFLAGS are not passed to it) which means macro values may be
computed wrongly.

That doesn't mean we should break automake dependency tracking in this
configuration, but it does mean you shouldn't do it when it hurts.  ;-)

> You need to explicitly turn off dependency tracking. My question is
> simply is there a good reason why automake does not turn off the
> dependency tracking automagically in this case, or should I consider
> it a missing feature.

I suppose I'll accept a patch that enhances automake/m4/depend.m4 to
address this.  Are you motivated?

Cheers, and thanks for the report,
Ralf

Re: $(srcdir)/foo is not generated because $(srcdir)/$(srcdir)/foo exists

[Ralf Wildenhues]

Hello Peter,

* Peter Johansson wrote on Sat, Feb 12, 2011 at 09:56:41PM CET:
> I got bitten by how VPATH builds work and wondered if there is a
> good way to avoid getting bitten again.

Not an easy question.

> I have an Automake snippet (see below) that creates a file,
> `.revision', with current revision number and from this is a C
> header file `revision.h' created. Then to test that things work as
> I'd like I have target similar to distcheck that essentially does
> 
> svn export $(srcdir) _exported
> cd _exported
> ./bootstrap
> mkdir _build
> cd _build
> ../configure
> make
> make check
> 
> When doing that here I get the following error:
> 
> revision=$(cat ../.revision) \
> && sed "s|@SVN_REVISION@|$revision|g" < ../svn_revision.h.in \
> > svn_revision.h-t && mv svn_revision.h-t svn_revision.h
> cat: ../.revision: No such file or directory
> 
> and when I look into why there is no `../.revision' and I try
> 
> make `../.revision' I get
> make: `../../.revision' is up to date.
> 
> That is make finds the file in my original working copy and not in
> my current structure. Is there a good way to avoid this to happen?

I'd try to ensure the VPATH structure of your "distcheck" is different
from the one above that.  I.e,. either use two levels of _build/_build
or have the upper build directory be a sibling, rather than a subdir,
of the source tree.

> --- Makefile.am ---
> 
> EXTRA_DIST = .revision svn_revision.h.in

I'd use $(srcdir)/.revision here.  That helps non-GNU makes understand
that it's the same thing that there is a rule for below.  But please see
'info Autoconf "VPATH and Make"' for all the problems VPATH can make
(sorry for the pun).

> BUILT_SOURCES = $(srcdir)/.revision $(builddir)/svn_revision.h
> 
> if HAVE_SVN_WC
> YAT_SVN_REVISION_FORCE:
> $(srcdir)/.revision: YAT_SVN_REVISION_FORCE
> @$(SVNVERSION) $(srcdir) > $@-t \
> && $(MOVE_IF_CHANGE) $@-t $@

I would prepend all lines of a rule with a TAB, not just those not
following a backslash-escaped newline.  I'm actually not totally sure
whether that was for portability to non-Posix make or so automake would
parse things reliably.

> else
> ## this is needed in 'svn export' build
> $(srcdir)/.revision:
> echo "" > $@

I don't see why you should need this rule here.  The .revision file is
distributed, so it should already exist.  At most I'd add a stub target

  $(srcdir)/.revision:

here.  Maybe I'm missing something?

> endif

> $(builddir)/svn_revision.h: $(srcdir)/svn_revision.h.in $(srcdir)/.revision
> revision=$$(cat $(srcdir)/.revision) \
> && sed "s|@SVN_REVISION@|$$revision|g" < $(srcdir)/svn_revision.h.in \
> > $@-t && mv $@-t $@

Hope that helps.

Cheers,
Ralf

Re: slow "make clean"

[Ralf Wildenhues]

Hello Ralf,

* Ralf Hemmecke wrote on Sat, Feb 12, 2011 at 11:46:55PM CET:
> I've a project that must use subdir-objects, because of filename
> clashes in subdirectories (non-recursive build).

> The generated Makefile contains a target "mostlyclean-compile" which
> has about 1500 lines that look like
> 
>   -rm -f src/foo/foo.$(OBJEXT)
>   -rm -f src/foo/foo.lo
>   ...
> 
> Of course it takes ages to say "make clean". About 10 sec.
> 
> Wouldn't it be faster if there were just one call to rm or at least
> block the number of arguments to rm so that the maximal argument
> lenght is not exceeded?

Yes, we could do the latter, I suppose.

> Any idea how I can speed that up?

You mean, as a workaround until Automake is improved?  You could
overwrite the rule in your Makefile.am:

mostlyclean-compile:
-find . -name \*.lo -o -name \*.$(OBJEXT) -print | xargs rm -f

But note that automake may generate other files that would need to be
mostlyclean'ed too, so the above could be insufficient for your project.

Thanks for the report,
Ralf

Re: Non-recursive automake vs. gettext

[Ralf Wildenhues]

Hi John,

* John Darrington wrote on Wed, Feb 09, 2011 at 07:56:20PM CET:
> I like to use a non-recursive makefile structure for my projects.
> However, if the project also uses gettext, I end up having to
> constantly fight against both gettext and auto{conf,make}.
> 
> If AM_GNU_GETTEXT appears in the configure.ac file, then automake
> refuses to run, but gives the error:
> 
>  configure.ac:336: required file `po/Makefile.in' not found
>  Makefile.am:5: AM_GNU_GETTEXT used but `po' not in SUBDIRS
> 
> There are ways to circumvent this of course.  But I think that
> automake should not do this if  subdir-options is specified in
> AUTOMAKE_OPTIONS

Agreed.

It might be worth fixing this.  Even better would be if we got around to
implementing Bruno's proposal to move gettext's configury stuff to
Automake (as documented and tested in the pot-primary git branch).
That could fix this, and at the same time make several other issues go
away.

If somebody wants to beat me on this, I'd be delighted.

Thanks for the bug report,
Ralf

distcheck: cannot create directory `.am21666': Permission denied (was: creating a report that displays what has been installed)

[Ralf Wildenhues]

Hi Bruce,

* Bruce Korb wrote on Thu, Feb 03, 2011 at 09:48:25PM CET:
> P.S. in researching this, I found in my own logs a bunch of these:
> > rm -rf $backupdir; exit $rc
> > mkdir: cannot create directory `.am21666': Permission denied
> > make[3]: Leaving directory 
> > `/old-home/bkorb/ag/ag/autogen-5.11.6pre21/_build/doc'
> 
> It all has to do with backup directories for texi stuff.
> Only happens in "make distcheck".  Seems to be innocuous,
> but it is disconcerting.

On what system, which Automake version was used (look at the first line
of a Makefile.in).

Thanks,
Ralf

Re: AC_CONFIG_HEADERS

[Ralf Wildenhues]

Hi Jeff,

* Jeff Squyres wrote on Thu, Feb 03, 2011 at 04:30:24PM CET:
> We recently got bitten by what appears to be undocumented behavior in
> AC_CONFIG_HEADERS (meaning that I couldn't find this behavior
> described in the docs; I might well have missed it...?).  It seems
> that AM's version of AC_CONFIG_HEADERS automatically adds a -I to the
> generated file in the build tree.

Well, not completely undocumented, but quite hidden and misplaced in
info Automake "Program Variables":

 Automake already provides some `-I' options automatically, in a
 separate variable that is also passed to every compilation that
 invokes the C preprocessor.  In particular it generates `-I.',
 `-I$(srcdir)', and a `-I' pointing to the directory holding
 `config.h' (if you've used `AC_CONFIG_HEADERS' or
 `AM_CONFIG_HEADER').  You can disable the default `-I' options
 using the `nostdinc' option.

[...]
> The problem came recently when we added some other files in
> include/hwloc/*.h.  One of them had the same name as a system header
> file -- something like this:
> 
> include/hwloc.h
> include/hwloc/config.h<-- generated
> include/hwloc/conflicting-name.h  <-- /usr/include/conflicting-name.h 
> also exists
> 
> Some of the source files in our project have #include
> "conflicting-name.h" -- intending to get the system version.  But
> since Automake hard-codes -I$(top_builddir)/include/hwloc into
> Makefile.in, our source files are picking up
> include/hwloc/conflicting-name.h, not the system version.  Changing
> the source files to #include  doesn't seem to
> help.  And since the value is hard-coded in Makefile.in, we can't add
> any m4/shell scripting in configure.ac to filter that -I value out.

> I see the "nostdinc" Automake flag to inhibit adding all -I's, but
> that seems like a bit of overkill for our needs.  We *do* want the
> -I's for the AH-generated file, but not for any others.  Is there a
> way to do that?  Or is there another way to inhibit specific -I's?

Well, currently the only way to work around this is for you to use
the nostdinc option and put
  AC_SUBST([DEFAULT_INCLUDES], ['-I. -I$(srcdir) $(top_builddir)/include'])

in configure.ac.

We might want to consider adding options stdinc-ah stdinc-nohdr or so
for generating only include paths for autoheader-generated headers plus
the first two, or the first two only, respectively.  (Each with VPATH
simplification I suppose.)

Thanks,
Ralf

Re: creating a report that displays what has been installed

[Ralf Wildenhues]

Hello Vincent,

* Vincent Torri wrote on Thu, Feb 03, 2011 at 04:32:36PM CET:
> I'm trying to improve the autotools of he openjpeg project. The lead dev
> wants me to display in the terminal what has been installed. I tried to
> argue that looking at what is already in the terminal is sufficient, he
> still prefers to see it at the end of the process.
> 
> One possible way would be to add in a file some output strings (using a
> hook) each time something is installed, and in the top-level Makefile.am,
> displaying that report file.
> 
> Is there a smarter way to do this ?

Hmm.  Even what you suggest isn't easy to do, at least not portably.

Here's a fairly expensive way to list those files; it depends on DESTDIR
support, and assumes you do not have a file or directory ./DEST:

  make install DESTDIR=`pwd`/DEST
  echo "Installed files:"
  (cd DEST && find .)
  rm -rf DEST

Alternatively, you can override prefix=... at install time, but libtool
might complain a bit.

Hope that helps.

Cheers,
Ralf

Re: dashes in Makefile macro names

[Ralf Wildenhues]

Hello Stefano,

* Stefano Lattarini wrote on Fri, Jan 28, 2011 at 09:55:18PM CET:
> But there's a minor mistake in your testsuite addition, and I also
> have a couple small unrelated nits/suggestions ...

You're spot-on with your observations, thanks.  I've included all
elided comments in the patch below.

> > +required=cc
> >
> Here's the mistake: we don't have a meaningful 'cc' requirement yet :-(

No, we do.  There is a default case in the required loop which isn't lax
enough to allow any cc but good enough for any GCC installation to work.
;-)

I intend to push this patch and a followup one to remove the warnings
about recursive variable expansion (the 'portability-recursive' channel
stuff) once there is indication that it will be in the next Posix.

Thanks,
Ralf

Accept dashes in make variable names.

* lib/Automake/Variable.pm (Automake::Variable): Accept dashes
in Variable names.
* tests/varname-dash.test: New test.
* tests/Makefile.am: Update.
* NEWS: Update.
Suggestion from Bruno Haible.

diff --git a/NEWS b/NEWS
index a5dc93f..08d4955 100644
--- a/NEWS
+++ b/NEWS
@@ -7,6 +7,10 @@ New in 1.11a:
   - The deprecated options `--output-dir', `--Werror' and `--Wno-error'
 have been removed.
 
+  - Dashes (minus signs) in make variable names are accepted now, since they
+are seemingly portable.  The Unified naming scheme still flattens them to
+underscores as before.
+
 * New targets:
 
   - New `cscope' target to build a cscope database for the source tree.
diff --git a/lib/Automake/Variable.pm b/lib/Automake/Variable.pm
index 9b2c838..50caee1 100644
--- a/lib/Automake/Variable.pm
+++ b/lib/Automake/Variable.pm
@@ -129,10 +129,10 @@ non-object).
 
 =cut
 
-my $_VARIABLE_CHARACTERS = '[.A-Za-z0-9_@]+';
+my $_VARIABLE_CHARACTERS = '[.A-Za-z0-9_@-]+';
 my $_VARIABLE_PATTERN = '^' . $_VARIABLE_CHARACTERS . "\$";
 my $_VARIABLE_RECURSIVE_PATTERN =
-'^([.A-Za-z0-9_@]|\$[({]' . $_VARIABLE_CHARACTERS . '[})]?)+' . "\$";
+'^([.A-Za-z0-9_@-]|\$[({]' . $_VARIABLE_CHARACTERS . '[})]?)+' . "\$";
 
 # The order in which variables should be output.  (May contain
 # duplicates -- only the first occurrence matters.)
diff --git a/tests/Makefile.am b/tests/Makefile.am
index f53a1f3..1c1e247 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -932,6 +932,7 @@ vala2.test \
 vala3.test \
 vala4.test \
 vala5.test \
+varname-dash.test \
 vars.test \
 vars3.test \
 vartar.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index b78e5a5..fc3cc5b 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -1195,6 +1195,7 @@ vala2.test \
 vala3.test \
 vala4.test \
 vala5.test \
+varname-dash.test \
 vars.test \
 vars3.test \
 vartar.test \
diff --git a/tests/varname-dash.test b/tests/varname-dash.test
new file mode 100755
index 000..775c118
--- /dev/null
+++ b/tests/varname-dash.test
@@ -0,0 +1,62 @@
+#!/bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+
+# Ensure automake allows and understand '-' in variable names.
+
+required=cc
+. ./defs || Exit 1
+
+cat >>configure.in <<\END
+AC_PROG_CC
+AC_OUTPUT
+END
+
+cat >Makefile.am <<\END
+var- = 1
+-var = 1
+var2 = $(var-)
+var-3 = 3
+var4 = $(var-3)
+three = 3
+#var5 = $(var-$(three))
+#var6 = $(-var)
+prog-s = foo bar-s
+foo-bar = a-b
+foo_SOURCES = $(foo-bar:a-b=foo.c)
+bar_s_SOURCES = bar.c
+bin_PROGRAMS = $(prog-s)
+
+test:
+   test x'$(var-)$(var2)$(var-3)$(var4)$(var5)' = x11333
+.PHONY: test
+END
+
+cat >foo.c <<\END
+int main (void) { return 0; }
+END
+cp foo.c bar.c
+
+$ACLOCAL
+$AUTOMAKE
+sed 's/^#//' Makefile.am > t
+mv -f t Makefile.am
+$AUTOMAKE -Wno-portability
+$AUTOCONF
+./configure
+$MAKE test
+$MAKE
+
+:

Re: dashes in Makefile macro names

[Ralf Wildenhues]

* Eric Blake wrote on Fri, Jan 28, 2011 at 04:04:19PM CET:
> On 01/28/2011 04:38 AM, Bruno Haible wrote:
> > Can you use $(FOO-BAR) in Makefile? It's not portable in POSIX [1]
> 
> Then let's get that addressed in POSIX!

Thank you Eric for being thorough!

> We're already in the middle of
> getting POSIX to support ${var${v}} nested variable expansions, on the
> same arguments of multi-implementation portability and ease-of-use for
> automake:

Ugh, and I still need to provide some more feedback.  :-/

> http://austingroupbugs.net/view.php?id=336

> > doc: Don't discourage dashes in Makefile macro names.
> > * doc/autoconf.texi (Special Chars in Names): Mention that dashes in
> > Makefile macro names are supported.
> 
> Looks reasonable to me.  But I'll wait for any additional feedback
> before pushing, in case anyone else has input on the matter.

Well, so far, automake -Wsyntax won't understand these variable names
and will complain and die if you use them in Makefile.am files.  The fix
is a couple of changes only, but also I would like to hear some feedback
from Posix people first, they might have some input about systems we do
not have test access to.

Thanks,
Ralf

Accept dashes in make variable names.

* lib/Automake/Variable.pm (Automake::Variable): Accept dashes
in Variable names.
* tests/minusvar.test: New test.
* tests/Makefile.am: Update.
* NEWS: Update.
Suggestion from Bruno Haible.

diff --git a/NEWS b/NEWS
index dda6f8c..a02f609 100644
--- a/NEWS
+++ b/NEWS
@@ -12,6 +12,10 @@ New in 1.11a:
 syntax warnings.  For portable Makefiles, such code can be wrapped in
 the GNU_MAKE conditional.
 
+  - Dashes (minus signs) in make variable names are accepted now, since they
+are seemingly portable.  The Unified naming scheme still flattens them to
+underscores as before.
+
 * New targets:
 
   - New `cscope' target to build a cscope database for the source tree.
diff --git a/lib/Automake/Variable.pm b/lib/Automake/Variable.pm
index 419a162..134de99 100644
--- a/lib/Automake/Variable.pm
+++ b/lib/Automake/Variable.pm
@@ -130,10 +130,10 @@ non-object).
 
 =cut
 
-my $_VARIABLE_CHARACTERS = '[.A-Za-z0-9_@]+';
+my $_VARIABLE_CHARACTERS = '[.A-Za-z0-9_@-]+';
 my $_VARIABLE_PATTERN = '^' . $_VARIABLE_CHARACTERS . "\$";
 my $_VARIABLE_RECURSIVE_PATTERN =
-'^([.A-Za-z0-9_@]|\$[({]' . $_VARIABLE_CHARACTERS . '[})]?)+' . "\$";
+'^([.A-Za-z0-9_@-]|\$[({]' . $_VARIABLE_CHARACTERS . '[})]?)+' . "\$";
 
 # The order in which variables should be output.  (May contain
 # duplicates -- only the first occurrence matters.)
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 43c18b0..8f1087e 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -625,6 +625,7 @@ mdate3.test \
 mdate4.test \
 mdate5.test \
 mdate6.test \
+minusvar.test \
 missing.test \
 missing2.test \
 missing3.test \
diff --git a/tests/minusvar.test b/tests/minusvar.test
new file mode 100755
index 000..3ba803f
--- /dev/null
+++ b/tests/minusvar.test
@@ -0,0 +1,58 @@
+#!/bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+
+# Ensure automake allows and understand '-' in variable names.
+
+required=cc
+. ./defs || Exit 1
+
+cat >>configure.in <<\END
+AC_PROG_CC
+AC_OUTPUT
+END
+
+cat >Makefile.am <<\END
+var- = 1
+var2 = $(var-)
+var-3 = 3
+var4 = $(var-3)
+three = 3
+#var5 = $(var-$(three))
+prog-s = foo bar-s
+bar_s_SOURCES = bar.c
+bin_PROGRAMS = $(prog-s)
+
+test:
+   @echo $(var-)$(var2)$(var-3)$(var4)$(var5)
+END
+
+cat >foo.c <<\END
+int main (void) { return 0; }
+END
+cp foo.c bar.c
+
+$ACLOCAL
+$AUTOMAKE
+sed 's/^#//' Makefile.am > t
+mv -f t Makefile.am
+$AUTOMAKE -Wno-portability
+$AUTOCONF
+./configure
+$MAKE test >stdout || { cat stdout; Exit 1; }
+grep 11333 stdout
+$MAKE
+
+:

Re: make test with fortran program

[Ralf Wildenhues]

* Eve-Marie Devaliere wrote on Fri, Jan 28, 2011 at 08:29:31PM CET:
> TESTS = Example1_Simple
> check_PROGRAMS = Example1_Simple
> Example1_Simple_SOURCES = Example1_Simple.f90
> Example1_Simple_FCLAGS =

There's a typo here, should be _FCFLAGS not _FCLAGS.

> -I/home/pub/evemarie/projects/CRTM/autoconf/play/CRTM/libsrc/
> Example1_Simple_LDADD =
> /home/pub/evemarie/projects/CRTM/autoconf/play/CRTM/libsrc/.libs/libCRTM.la
> AM_FCFLAGS = $(Example1_Simple_FCLAGS)# not sure if
> Example1_Simple_FCLAGS is useful, definitely need the AM_FCFLAGS to have
> my include line...
> 
> Of course I will get rid of the hardcoding now that I got it to work...

The above should be simplifiable to something like below.
Please note that you should reference .la files outside the .libs
directory (so that dependencies on them will be produced correctly)
and you should use relative path names to things that are inside your
build tree (but you can use $(top_builddir) or similar).

So if all programs in this Makefile.am are built similarly, it should
work to something like this:

TESTS = $(check_PROGRAMS)
check_PROGRAMS = Example1_Simple
Example1_Simple_SOURCES = Example1_Simple.f90
AM_FCFLAGS = -I../../../play/CRTM/libsrc
LDADD = ../../../play/CRTM/libsrc/libCRTM.la


and if you can afford to require Automake 1.11 or newer, then
you wouldn't need a *_SOURCES line per test but could use

AUTOMAKE_OPTIONS = 1.11
AM_DEFAULT_SOURCE_EXT = .f90

instead.

Cheers,
Ralf

Re: make test with fortran program

[Ralf Wildenhues]

Hello Eve-Marie,

please do not top-post (writing your answer above what you refer to)
on these lists.  Thank you.

* Eve-Marie Devaliere wrote on Thu, Jan 27, 2011 at 07:55:49PM CET:
> AM_DEFAULT_SOURCE_EXT = .f90
> TESTS = forward/Example1_Simple/Example1_Simple \
> k_matrix/Example1_Simple/Example1_Simple
> check_PROGRAMS = $(TESTS)
> AUTOMAKE_OPTIONS=subdir-objects

> make test
> make: Nothing to be done for `test'.

'make check' is the thing to use; see 'info Automake --index check'.

> It hasn't created the *.o files either

It should create all the files you need.

> I haven't specified test: or all: but in the main Makefile.am I haven't
> either and it figures out how to compile what. I thought TESTS would do
> that for me??

TESTS only specifies what tests to run.  check_PROGRAMS specifies which
files are compiled and linked, and are needed before the 'check' target
can run.  AM_DEFAULT_SOURCE_EXT specifies the source file extension
that, for a program 'foo', the default foo_SOURCES entry has.

Cheers,
Ralf

Re: Conditional target

[Ralf Wildenhues]

Hello Sergio,

* Sergio Belkin wrote on Thu, Jan 27, 2011 at 01:15:45PM CET:
> 2011/1/27 Ralf Wildenhues :
> > * Sergio Belkin wrote on Thu, Jan 27, 2011 at 04:40:10AM CET:
> >> tests = $(EXTRA_PROGRAMS)
> >
> >> if forcestatic
> >> tests: LIBS += $(STATIC_RESOLV)
> >> end if
> >>
> >> The problem is that automake it complains that .
> >>
> >> tests was already defined in condition forcestatic, which is included
> >> in condition TRUE ...
> >> Makefile.am:150: ... `tests' previously defined here
[...]
> >  variable = foo
> >  if COND
> >  variable += bar
> >  endif
> 
> Thanks but i if I do that the static library is not appended at the
> end, and so it fails

Please show a small reproducible example that fails for you.
Maybe you did not set STATIC_RESOLV in this Makefile.am or did
not call AC_SUBST([STATIC_RESOLV]) in configure.ac?

Cheers,
Ralf

Re: make test with fortran program

[Ralf Wildenhues]

Hello Eve-Marie,

* Eve-Marie Devaliere wrote on Thu, Jan 27, 2011 at 07:07:51PM CET:
> AM_DEFAULT_SOURCE_EXT = .f90
> TESTS = forward/Example1_Simple/Example1_Simple \
> k_matrix/Example1_Simple/Example1_Simple
> check_PROGRAMS = $(TESTS)
> 
> when I run automake it outputs
> automake
> test/Makefile.am: object `Example1_Simple.$(OBJEXT)' created by
> `k_matrix/Example1_Simple/Example1_Simple.f90' and
> `forward/Example1_Simple/Example1_Simple.f90'

Use the 'subdir-objects' Automake option (see 'info Automake Options'
for how to do that).

> but nothing was created and if I try configure it complains that:
> 
> config.status: error: cannot find input file: `test/Makefile.in')

That looks like a followup error: if automake errors out, it won't
create a Makefile.in file.

Cheers,
Ralf

Re: Conditional target

[Ralf Wildenhues]

* Sergio Belkin wrote on Thu, Jan 27, 2011 at 04:40:10AM CET:
> tests = $(EXTRA_PROGRAMS)

> if forcestatic
> tests: LIBS += $(STATIC_RESOLV)
> end if
> 
> The problem is that automake it complains that .
> 
> tests was already defined in condition forcestatic, which is included
> in condition TRUE ...
> Makefile.am:150: ... `tests' previously defined here

(besides the typo that was already noted:)

Automake does not understand GNU make-style target-specific variable
settings
  target: variable = setting
  target: variable += setting

What it does interpret is Automake conditionals that are evaluated at
config.status time.  And it interprets += itself by flattening it:

  variable = foo
  if COND
  variable += bar
  endif

will be translated to something like
  variable = foo $(am__some_internal_helper_var1)
  @COND_TRUE@am__some_internal_helper_var1 = bar

and config.status substitutes @COND_TRUE@ with '#' or '' depending on
whether the condition is true at configure time.

Hope that helps.

Cheers,
Ralf

Re: automake-1.11: no `Makefile.am' found for any configure output

[Ralf Wildenhues]

Hi Brian,

* Brian J. Murrell wrote on Tue, Jan 25, 2011 at 03:26:36PM CET:
> Our project, which is admittedly built on older versions of the
> autotools seems to be exhibiting some issues when I try to use it with
> automake 1.11.
> 
> I have not doubt that this is due to our particular implementation just
> not being up to snuff with automake 1.11.  I'm just not sure where/why
> though.
> 
> The error I can automake, autoconf, etc. and get a valid working
> configuration that I can issue make in and all works.  At some point
> however "make clean" starts failing with:
> 
> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash 
> /home/brian/lustre-release/BUILD/lustre-2.0.59/missing --run aclocal-1.11 
>  cd . && /bin/bash /home/brian/lustre-release/BUILD/lustre-2.0.59/missing 
> --run automake-1.11 --foreign
> automake-1.11: no `Makefile.am' found for any configure output

Any chance that your normal bootstrap process passes arguments such as
-I m4 to aclocal, and the above rebuild rule needs to be fixed?

If you don't have a toplevel Makefile.am, you might need to add a stub
  am--refresh:

rule in the toplevel Makefile.in, to avoid some cases of rebuild
failures; more precisely, you should also have correct rules for
rebuilding aclocal.m4, configure, config.status, and Makefile in
the toplevel Makefile.in.  I don't think that is related to the
above error though, but I'm not sure.

Do you have a link to a tarball that exhibits the problem, and
instructions how to reproduce it?

Thanks,
Ralf

Re: texinfo files in sub directories

[Ralf Wildenhues]

Hello Thomas,

* Thomas Keller wrote on Tue, Jan 25, 2011 at 12:36:12AM CET:
> I'm currently in the process of cleaning up our source tree and
> reorganize many files. One of the last tasks here is to combine our
> documentation-relevant files into one subdirectory doc/. This worked out
> fine so far, i.e. info files are automatically generated at the correct
> place, but the previously automatically generated targets for PDF and
> DVI output completely vanished from the resulting Makefile and as such
> PDF and DVI generation are completely broken.

Please send a minimal example Makefile.am and commands to reproduce the
breakage, plus their error messages, to bug-automake (opening a new PR).

FWIW, Libtool uses subdir texinfo files for a while now.  There have
been some glitches a while ago (I only find this now:
)
but otherwise things should be working.

Thanks for the report,
Ralf

Re: Probem with docdir sudirectory

[Ralf Wildenhues]

* Sergio Belkin wrote on Thu, Jan 20, 2011 at 07:35:48PM CET:
> 2011/1/20 Ralf Wildenhues :
> > * Sergio Belkin wrote on Thu, Jan 20, 2011 at 03:39:04PM CET:
> >> I have a file into subdirectory of docdir that is not installed
> >>
> >> make distcheck complains saying:
> >>
> >> No rule to make target `doc/tests/exampleConf.txt', necesario para 
> >> `all-am'.
> >
> > Does the file exist in either the source or the build tree?
> 
> Yes it does.

Erm, but not in the distribution.  You put it in
docdir_tests_DATA.  You should use dist_docdir_tests_DATA
if you want all those files distributed.

Sorry for not seeing that the first time over, I got confused
over you writing "installed" in the sentence above.

Cheers,
Ralf

Re: Probem with docdir sudirectory

[Ralf Wildenhues]

Hello Sergio,

* Sergio Belkin wrote on Thu, Jan 20, 2011 at 03:39:04PM CET:
> I have a file into subdirectory of docdir that is not installed
> 
> make distcheck complains saying:
> 
> No rule to make target `doc/tests/exampleConf.txt', necesario para `all-am'.

Does the file exist in either the source or the build tree?

> docdir = $(datadir)/doc/${PACKAGE}
> dist_doc_DATA = doc/README doc/README.FEDORA doc/README.CENTOS
> doc/multithreading.intro doc/intro doc/INSTALL
> docdir_testsdir = $(docdir)/tests
> docdir_tests_DATA = doc/tests/testUpThreadWorkSplittable.cc \
[...]
> doc/tests/testUpConf.cc doc/tests/exampleConf.txt \


> installcheck-local:
>     @echo "We will run a manual building against installed $(PACKAGE)"

You mean 'build' not 'building' here.

>     @sleep 2
>     g++  -Wl,--as-needed -o  testUpLog $(srcdir)/tests/testUpLog.cc 
> -I$(includedir)  -L$(libdir) -lUpTools

Please don't hard-code compiler names and flags like that.  That's what
AC_PROG_CXX is for, then use $(CXX) at least.  For a portable variant of
-Wl,--as-needed, use the lib-ignore module from gnulib.

>     @echo
>     @echo  -e "You can run more tests, issuing:\n g++ -Wl,--as-needed -o 
> binary_filename tests/source_filename -l UpTools"

echo -e is not portable, you can use printf instead.
A space between -l and the library name is AFAIK not portable.

Hope that helps.

Cheers,
Ralf

more bug economics (was: bug#7648: ylwrap appears not to support bison's lalr1.cc skeleton)

[Ralf Wildenhues]

[ moving from bug-automake to automake ]

* Stefano Lattarini wrote on Tue, Jan 18, 2011 at 02:29:24PM CET:
> had already been reported in PR automake/491 (dating back to 2006!):
>  
> 
> Another good reason to fix the bug before the next release, I'd say.

For what little it's worth, here's another bug economics rule:

Regressions, at least recent ones, are (almost) infinitely more
important than ordinary bugs: you can fix all bugs you like, if
your package has significant regressions then distributions won't
move to it (and users will have to live with the bugs in the old
code).  Old regressions are less critical.  With ordinary bugs,
age is less clear an indicator of priority; there, the number of
potentially affected users matters more.

> I still plan to (try to) do so myself, even if not right away (but
> hopefully, soonish enough).

Of course, bug fixes are still nice, so be my guest!

Cheers,
Ralf

bug reports, and lack of feedback (was: make -j1 fails)

[Ralf Wildenhues]

* Dave Hart wrote on Tue, Jan 18, 2011 at 09:49:02AM CET:
> On Fri, Jan 14, 2011 at 6:27 PM, Ralf Wildenhues wrote:
> > Can you make your project available for us to try and reproduce the bug
> > (I have access to a couple of FreeBSD systems)?  If not, then I'm afraid
> > I'll not be able to pursue this further before seeing a reduced version.
> 
> While you're waiting for that,

FWIW, I was offered non-public access to the code.  Maybe it is helpful
to repeat in public my stance toward this: I treat non-public code, or
public code that is not well-known, rather cautiously, even more so if I
don't know the author(s) well either.  This is not to place mistrust in
any particular person, just application of the internet principle "be
careful in what downloaded stuff you execute", even in a jail.

This means it takes (quite) some initial time to get past a cursory
review of the code for nastiness.

> perhaps you could pursue the problem I
> did take the time to provide a reduced test case for in November:
> 
> http://lists.gnu.org/archive/html/automake/2010-11/msg00135.html

> Note that this issue is no longer a problem for NTP -- autogen's
> libopts now provides LIBOPTS_CHECK_NOBUILD, which sidesteps the need
> to conditionalize AC_CONFIG_FILES([libopts/Makefile]), and works
> correctly on Automake 1.10, which doesn't support AM_COND_IF
> conditionalization of AC_CONFIG_FILES.

Good to know.

> I am annoyed no one has taken the time to follow up after I took the
> time to produce a reduced test case illustrating the automake
> misbehavior, and each time I see a request for a reduced repro, I
> wonder what I might have done wrong in anticipating the request and
> providing the reduced test case in the initial report.

I looked at it for maybe half an hour back then, and didn't see an easy
way to fix it.  Sorry.  I should maybe have followed up to let you know.
You didn't do anything wrong, otherwise I would eventually have asked.
But anyway we should've thanked you for the report, so please allow me
to thank you now for the nice and well-written bug report!

Generally, there are more bug reports than there are people looking at
them, analyzing and fixing them.  As is the case in so many free
software projects.  If you are dissatisfied with that, and you have
resources, you are very welcome to help out.  Other than that, I guess I
should encourage using our new-ish debbugs bug tracker (just write to
bug-automake to open a new PR) to be a little more sure issues don't get
lost.

I typically try to make sure rather quickly that a report is complete,
so that when someone eventually gets to it, they have a chance to do
something productive with it even if the original reporter has gone off
to some other pasture in the meantime.

Since you now have a workaround for your bug, I hope you understand that
the priority of it is rather low.  Sorry again, but that's how bug
economics work, necessarily.

Cheers,
Ralf

Re: Avoid VPATH-related distcheck failures with non-GNU make

[Ralf Wildenhues]

* Stefano Lattarini wrote on Sun, Jan 16, 2011 at 04:39:50PM CET:
> On Sunday 16 January 2011, Ralf Wildenhues wrote:
> > * Stefano Lattarini wrote on Sat, Jan 15, 2011 at 04:50:31PM CET:
> > > Hmm... To me, this sounds an awful lot like sweeping the dirt under
> > > the rug.
> > 
> > OK, so here's a more open alternative: let's try to fix the failures.
> > Are you going to work on this, or at least provide significant help?
> >
> I might try, but obviously I cannot guarantee anything.
> 
> What about the following middle ground solution for the moment: extend
> the distcheck target to honor a switch (which should probably be an
> environment variable, say 'DISTCHECK_USE_VPATH') telling to perform
> simply an in-tree build (while continuing to perform VPATH builds by
> default).

Well, if we change distdir.am then we are already adding interfaces
and API and all that.  That's not what we should do "for the moment",
but only if we conclude that that's the only choice we have.

Looking in more detail at the test failures, it's not actually all that
many that happen in distcheck, maybe a dozen distinct tests or so.
So, apologies for my previous exaggeration.  Let's just see if we can
fix them, first.

Thanks,
Ralf

Re: Avoid VPATH-related distcheck failures with non-GNU make

[Ralf Wildenhues]

Hello Stefano,

* Stefano Lattarini wrote on Sat, Jan 15, 2011 at 04:50:31PM CET:
> On Saturday 15 January 2011, Ralf Wildenhues wrote:
> > ensure_distcheck_ ()
> > {
> >   if $MAKE --version -v | grep GNU; then
> > $MAKE distcheck
> >   else
> > :
> >   fi
> > }

> Hmm... To me, this sounds an awful lot like sweeping the dirt under
> the rug.

OK, so here's a more open alternative: let's try to fix the failures.
Are you going to work on this, or at least provide significant help?
The current amount of failures from this hunting for regressions in
Automake proper harder than it should be, because there are many
failures which could be testsuite regressions, or regressions for
features we don't advertise, or long-standing issues for which it thus
is not imperative that we fix them quickly.

I reject the notion of disallowing VPATH builds with GNU make only.
The reason is simple: I make good use of this feature every time I test
the Automake package with some non-GNU make (I build several platforms
off of one tree shared via NFS).  It would make my testing significantly
harder if GNU make needed to be added to the mix.

Just diagnosing non-GNU make at configure time in a VPATH setup is
slightly problematic because many people still do
  ../configure
  gmake

(note the missing MAKE=gmake on the configure command line).  Or, as I
also happen to do, go back to a build tree several months old and do
'git pull && cd build && make' in it, forgetting that MAKE=gmake was set
at configure time (portably warning about this at make time is
surprisingly hard to do right, when you can't assume GNU make).

> What about a more radical but IMHO more honest approach, to be implemented
> in four steps:
> 
>   [1] Clearly document in the manual that VPATH builds are expected to
>   work with GNU make only.

See above.  For lots of packages, portable make works just fine even in
VPATH mode.  I'm fine with mentioning that GNU make or non-VPATH should
be tried if there are problems with another make in a VPATH build.

>   [2] Update AM_INIT_AUTOMAKE to make it complain(*) if it detects that
>   $MAKE is not GNU make but a VPATH build is being attempted.  The
>   user should be allowed to override this check, obviously.

See above.

> (*) With a warning or an error?  Not sure yet, but it should be warning
> in the next Automake version, not to suddenly disrupt backward
> compatibility.

Generally, I want to keep even git master of Automake always in a
close-to-releasable state.  Even if all current failures are only
issues in the testsuite, or long-standing bugs, this is too much to
just ignore it.

Cheers,
Ralf

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Ralf Wildenhues]

* Bob Friesenhahn wrote on Fri, Jan 14, 2011 at 07:55:17PM CET:
> Where do these terms 'alpha' and 'beta' build system originate from?

I've read them in
.

No idea whether they are standardized somehow or somewhere.

Cheers,
Ralf

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Ralf Wildenhues]

Hi Bob,

* Bob Friesenhahn wrote on Fri, Jan 14, 2011 at 04:04:35PM CET:
> In addition to the above, Automake should be supplying enough explicit rule
> information (pertaining to the actual needs of the current project) that the
> underlying build tool should not have to go off in the weeds and try a
> million other incantations as well.

I don't think Automake needs to bother with the difference of alpha or
beta build systems at all until we get to the point where somebody
proposes that Automake also output rules suitable for a suitable beta
build system.  Or, and that is what I would prefer, somebody implements
a beta build system that parses, as much as possible, Posix or GNU
makefiles.  Because then Automake wouldn't have to change anything.  :-)

Cheers,
Ralf

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Ralf Wildenhues]

* Guido Draheim wrote on Thu, Jan 13, 2011 at 08:55:20PM CET:
> By the way, I would find it more interesting to see some
> automake-replacement that allows GNU-make extensions and
> compiles them down to standard make as far as possible...

Generally, I wouldn't, as I don't like the idea of reimplementing GNU
make inside Automake.  A couple of ideas might be worth stealing though.

> On the other hand some automake-restrictions are really not needed
> anymore. (variables with "." and "/" would be nice - along with
> http://www.gnu.org/software/automake/manual/make/Computed-Names.html
> it would easily allow a scheme to define generated compile-flags).

Well, with a bit of luck, recursive variable expansion will be part of
the next Posix standard.  Unfortunately, there are limitations in
existing make implementations that affect your (good) ideas, and also I
don't think / is portable in variable names.  See here for more info:


(Reminds me of which, I still need to do some portability testing for
that ...)

Cheers,
Ralf

Re: make -j1 fails

[Ralf Wildenhues]

* Pippijn van Steenhoven wrote on Fri, Jan 14, 2011 at 09:38:36AM CET:
> On Thu, Jan 13, 2011 at 07:22:20PM +0100, Ralf Wildenhues wrote:
> > If the failure persists, please post short configure.ac and
> > Makefile.am which expose the problem for you. You can start with
> > what I show below, and adjust that if it doesn't expose it.
> 
> The code didn't trigger the bug and I couldn't easily reproduce it by
> modifying it. It involves considerable effort to modify my existing
> project and upload it to the FreeBSD build machine and I don't have time
> to do that, now.

Understood.  This sounds like a FreeBSD make bug, but I'm not sure.
Can you make your project available for us to try and reproduce the bug
(I have access to a couple of FreeBSD systems)?  If not, then I'm afraid
I'll not be able to pursue this further before seeing a reduced version.

Thanks,
Ralf

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Ralf Wildenhues]

* NightStrike wrote on Fri, Jan 14, 2011 at 05:12:46AM CET:
> On 1/13/11, Ralf Wildenhues wrote:
> > make is a bit flawed for real large projects because it always walks
> > the whole dependency graph, unlike beta build systems who use a notify
> > daemon and a database to only walk subgraphs known to be outdated.
> 
> How big is real large?  GCC uses make, for instance, and it's the
> biggest public project that I personally know about.

Running 'make' in an up to date GCC build tree and hot file cache takes
on the order of 10s for me.  That may be tolerable, but only because any
changes causing recompiles lead to much longer build times, and because
the GCC build system provides lots of special-case make targets to only
rebuild parts of the tree (e.g., stage3-bubble, all-target-libgfortran)
which finish more quickly.  The GCC build system is fairly special
though, in that it is pretty complex and (ab?)uses recursive makefiles
also to allow ignoring big chunks of the dependency tree in some cases.

(That, by the way, is what nobody ever tells you when they point you to
the "Recursive Make Considered Harmful" paper: that walking the full
dependency tree is more expensive than walking a factorized one.  Sure,
you need to keep the toplevel deps up to date manually then.)

> At what magnitude does make break down, do you think?  And how/where
> does it become flawed?

A rough estimate is 100k files, even after quadratic scaling issues in
make are fixed.  The tup project (which is a prototype beta build
system) claims some lower numbers even[1], but their graphs indicate
that they've also measured some nonlinear effects in make, which should
all be fixable.

I'd guess that webkit would benefit slightly already though.

> In retrospect, even when dealing with GCC, I never do a partial
> rebuild.

For full rebuilds, this particular issue is irrelevant.  There, the
whole dependency tree needs to be walked no matter what build system.

Cheers,
Ralf

[1] http://gittup.org/tup/make_vs_tup.html

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Ralf Wildenhues]

* Bob Friesenhahn wrote on Thu, Jan 13, 2011 at 08:32:01PM CET:
> On Thu, 13 Jan 2011, Ralf Wildenhues wrote:
> >You can implement hash-based dependencies inside GNU make if you like.
> >Or another make.  Maybe someone has even done so already.  It has little
> >to do with Automake.
> 
> I have never heard of such a thing implemented in make.  It would
> surely bloat the makefile by more megabytes or be excruciatingly
> slow.

That's not what I meant.  You can hack the source code of GNU make
with the goal to (optionally) replace timestamp-based dependencies
with file-hash-based depdendencies.  No changes at all in any user
makefiles.  It's not a large change even.  IIRC somebody has done
this already, it just hasn't been merged upstream.

BTW, Nathanael's propositions in this thread go in the same direction:
http://gcc.gnu.org/ml/gcc-patches/2010-11/msg01504.html

Cheers,
Ralf

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Ralf Wildenhues]

* Guido Draheim wrote on Thu, Jan 13, 2011 at 08:25:05PM CET:
> But GNU Make on its own has a very large feature-set that is strong enough to
> be a build-system on its own. You don't need much of autoconf/automake 
> anymore!!

Exactly.  quagmire is one (other) implementation of that.  I don't
understand why people don't just use that if they want to.

(The fact that the project appears stale is irrelevant.  This is free
software, you have the freedom to revive even the ugliest of zombies
... ;-)

Cheers,
Ralf

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Ralf Wildenhues]

* Bob Friesenhahn wrote on Thu, Jan 13, 2011 at 07:50:08PM CET:
> Regardless, 'make's use of timestamps based on simple "newer than"
> analsys is not very robust in our real world.

That may be true, but this issue is completely orthogonal to whether
Automake caters to non-GNU make implementations or not.

You can implement hash-based dependencies inside GNU make if you like.
Or another make.  Maybe someone has even done so already.  It has little
to do with Automake.


make is a bit flawed for real large projects because it always walks
the whole dependency graph, unlike beta build systems who use a notify
daemon and a database to only walk subgraphs known to be outdated.
I maintain that even that sort of functionality can be mostly hidden
below a Posix make-like interface.  Not relevant for the Automake layer
either, though.

Cheers,
Ralf

bug#7670: About make implementations that don't chain implicit rules

[Ralf Wildenhues]

I'm merging bug 7824 with this one.

If we can find a way to safely emit additional dependencies to help
these make implementation, then I am in favor of doing it.  (We can
wrap them in 'if !GNU_MAKE' if need be.)

Thanks,
Ralf