Re: bug#9768: Makefile broken after removing included *.am file
* Stefano Lattarini wrote on Mon, Oct 17, 2011 at 10:20:05AM CEST: On Sunday 16 October 2011, Ralf Wildenhues wrote: What happens if I write include fragment-with-typo-in-name.am but I want that fragment included? I fear I cannot parse this question... Do you mean that you want to include the file `bar.am' but erronously write include baz.am in your Makefile.am instead? In this case, automake will obviously complain and bail out, as it did before my patch (in fact, said patch doesn't touch the automake script at all, so this invariance of its behaviour should be obvious). Alright. The first time you write the include line and invoke automake, it will bail out if there is a typo. But if the fragment file is later (re)moved, and you type 'make' again, you will not get an error, as automake will not be rerun automatically. Only when automake is invoked manually, or triggered by some other out of date file, will the error be noticed. Taken a step further, if some fragment file for some reason does not get included in a release tarball, then the developer might not see that during testing, but users will miss a file needed to tinker with the package. Am I missing something? Thanks, Ralf
Re: bug#9768: Makefile broken after removing included *.am file
Hello, * Stefano Lattarini wrote on Sun, Oct 16, 2011 at 08:27:12PM CEST: On Sunday 15 November 2009, Peter Johansson wrote: make: *** No rule to make target `aminclude.am', needed by `Makefile.in'. Stop. This is very similar to the deleted header file problem for *.m4 files that was fixed in version 1.11 by adding a stub rule. I suppose this problem could be solved in the same fashion. Thanks for the report and the suggestion; I've been bitten few times by this bug myself, so I agree it's time to fix it. See the attached minimalistic patch, which I'll push to maint in a few days (as usual, reviews welcome). There is also a follow-up patch (attached as well) that ensures the stub rules do not end up covering real errors. What happens if I write include fragment-with-typo-in-name.am but I want that fragment included? IIUC then automake will still error out when it gets rerun, but because of the way dependencies are handled now, it won't get rerun automatically, right? Thanks, Ralf
Re: [RFC] Releasing automake 1.11.2
Hello Stefano, * Stefano Lattarini wrote on Sun, Oct 16, 2011 at 05:44:14PM CEST: I think it's about time to release automake 1.11.2 -- the `maint' branch contains various bug fixes w.r.t. the 1.11.1 release (some of them quite important), and offers some new small features and various warnings/deprecations that should prepare the users for the backward-incompatible changes planned for automake 1.12 (so, the more 1.11.2 precedes 1.12, the more these warnings will have a chance to be effective). Agreed. Ralf, Jim, could we agree on a timeline/plan of sort for the release of automake 1.11.2? I've never done nor witnessed as an insider an automake release, so I'd rather leave someone else take the lead on this; but I'll try to contribute and learn as much as I can of course ;-) I think you should just try it out. If you get stuck, just stop and ask. I'll try to glance at the list at least every couple of days. Do you have access to test systems yet? If so, could you start a round of tests? I have a script to do that (I think I sent you before, otherwise look in my $HOME). You will need upload access; I think Jim can help with that. Thanks; and sorry for my continued lack of time. Cheers, Ralf
Re: [PATCH] tests: simplify wrapper for aclocal
Hi Stefano, * Stefano Lattarini wrote on Fri, Sep 02, 2011 at 09:58:27PM CEST: * tests/aclocal.in: Remove use of $ACLOCAL_TESTSUITE_FLAGS and extra `-I' flags; they are not really required, since the file `m4/amversion.m4' is generated in the srcdir anyway. * tests/acloca10.test: Remove use of $ACLOCAL_TESTSUITE_FLAGS. * tests/acloca18.test: Likewise. * tests/defs.in: Don't nullify $ACLOCAL_TESTSUITE_FLAGS, and do not export it. Did you do a full test suite run in a VPATH tree with it? I vaguely seem to remember that at least some test used/produced more files in this directory. Thanks, Ralf
Re: [PATCH] tests: simplify wrapper for aclocal
* Stefano Lattarini wrote on Sun, Sep 04, 2011 at 05:47:41PM CEST: On Sunday 04 September 2011, Ralf Wildenhues wrote: * Stefano Lattarini wrote on Fri, Sep 02, 2011 at 09:58:27PM CEST: * tests/aclocal.in: Remove use of $ACLOCAL_TESTSUITE_FLAGS and extra `-I' flags; they are not really required, since the file `m4/amversion.m4' is generated in the srcdir anyway. * tests/acloca10.test: Remove use of $ACLOCAL_TESTSUITE_FLAGS. * tests/acloca18.test: Likewise. * tests/defs.in: Don't nullify $ACLOCAL_TESTSUITE_FLAGS, and do not export it. Did you do a full test suite run in a VPATH tree with it? No, originally I only re-run the two changed tests in a VPATH build (they passed). Now I've re-run the full testsuite (still in a VPATH build of course), and it still passes. I vaguely seem to remember that at least some test used/produced more files in this directory. Which directory are you referring to? $top_builddir/m4 aka $test_builddir/m4. Seems however that that's not the case. Good. Thanks, Ralf
Re: [FYI] {master} gitignore: more use of anchors
* Stefano Lattarini wrote on Tue, Aug 09, 2011 at 05:04:16PM CEST: On Tuesday 09 August 2011, Eric Blake wrote: On 08/09/2011 08:44 AM, Stefano Lattarini wrote: * .gitignore: Anchor files that are intended to be ignored only if found in the same directory of the `.gitignore' file, not also in its subdirectories. * doc/.gitignore, doc/amhello/.gitignore, lib/Automake/.gitignore, lib/Automake/tests/.gitignore, tests/.gitignore: Likewise. Also, where needed, add new entries that were once implied by the non-anchored entries in the upper-level `.gitignore' files. While it is up to Ralf, I personally prefer a single .gitignore file for the entire tree. FWIW, I slightly prefer that too. I don't really have a big preference either way for Automake. For big projects, a single .gitignore file might not scale that well. So, OK if you have a patch for that. Thanks, Ralf
Re: [PATCH 8/5] tap: support colorization of testsuite progress output
On Tuesday 19 July 2011, Ralf Wildenhues wrote:
Below is what I've squashed in. OK?
Sure, thanks!
--- a/lib/tap-driver
+++ b/lib/tap-driver
@@ -230,17 +230,24 @@ sub colored ($$)
sub decorate_result ($)
{
- return $_[0] unless $cfg{color-tests};
- # Best way to simulate a 'switch' construct here.
- for (@_)
+ my $result = shift;
+ return $result unless $cfg{color-tests};
+ my %color_for_result =
+(
+ ERROR = 'mgn',
+ PASS = 'grn',
+ XPASS = 'red',
+ FAIL = 'red',
+ XFAIL = 'lgn',
+ SKIP = 'blu',
+);
+ if (my $color = $color_for_result{$result})
+{
+ return colored ($color, $result);
+}
+ else
{
- $_ eq ERROR and return colored ('mgn', $_);
- $_ eq PASS and return colored ('grn', $_);
- $_ eq XPASS and return colored ('red', $_);
- $_ eq FAIL and return colored ('red', $_);
- $_ eq XFAIL and return colored ('lgn', $_);
- $_ eq SKIP and return colored ('blu', $_);
- return $_; # Don't colorize unknown stuff.
+ return $result; # Don't colorize unknown stuff.
}
}
Re: [PATCH 1/5] {test-protocols} parallel-tests: make parsing of test results safer
* Stefano Lattarini wrote on Tue, Jul 19, 2011 at 11:54:09AM CEST: Please, this is really important: we need to research the other test protocols, what they do to be robust here. Don't NIH here, because the experience we have is not enough to not mess up this. Consider this research as part of the work needed for the SoC assignment, it is a very important part (as it is not easily corrected once released). Honestly, I really don't have a clue about where to start looking for such prior art, and my rather naive Google searches haven't returned anything useful. Do you have any suggestion? Even a loose one might be an helpful starting point. A very incomplete list: Test protocols: TAP subunit Test suite environments: any that use the above dejagnu qmtest Test::Harness http://en.wikipedia.org/wiki/Portal:Software_Testing lists several Autotest Learning from mistakes (in general): http://www.airs.com/blog/archives/499 Hope that helps. Cheers, Ralf
Re: [PATCH] docs: Add references between the 2 sections on compiling Java.
* Stefano Lattarini wrote on Sat, Jul 16, 2011 at 03:18:28PM CEST:
From: Benoit Sigoure tsuna...@gmail.com
Date: Fri, 15 Jul 2011 16:49:45 -0700
Subject: [PATCH] docs: add references between the 2 sections on java support
* doc/automake.texi (Java Support, Java): Add cross-references.
Nice one, thanks.
Cheers,
Ralf
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -6665,8 +6665,10 @@ is as follows:
@cindex Java support
@cindex Support for Java
-Automake includes support for compiled Java, using @command{gcj}, the Java
-front end to the GNU Compiler Collection.
+Automake includes support for natively compiled Java, using @command{gcj},
+the Java front end to the GNU Compiler Collection (preliminary support
+for compiling Java to bytecode using the @command{javac} compiler is
+also present; @pxref{Java}).
Any package including Java code to be compiled must define the output
variable @code{GCJ} in @file{configure.ac}; the variable @code{GCJFLAGS}
@@ -7534,8 +7536,9 @@ libtool, The Libtool Manual}) with the
@code{LTLIBRARIES} primary.
@cindex @code{JAVA} primary, defined
@cindex Primary variable, @code{JAVA}
-Automake provides some minimal support for Java compilation with the
-@code{JAVA} primary.
+Automake provides some minimal support for Java bytecode compilation with
+the @code{JAVA} primary (in addition to the support for compiling Java to
+native machine code; @pxref{Java Support}).
Any @file{.java} files listed in a @code{_JAVA} variable will be
compiled with @code{JAVAC} at build time. By default, @file{.java}
Re: [GSoC] Some patches for testsuite harness improvements and TAP support introduction
* Stefano Lattarini wrote on Fri, Jul 15, 2011 at 12:35:02AM CEST: The patches I'm going to post in this thread have already been applied to the 'GSoC/experimental/test-results-work' temporary branch, but now I think they are mature enough to be moved to the official branch 'test-protocols'; fixlets and improvements can be done with follow-up patches (I even think that this should be preferred to doing further amendaments and tweaking of these patches). Is this FYI or to be reviewed? Thanks, Ralf
Re: [PATCH 1/5] {test-protocols} parallel-tests: make parsing of test results safer
* Stefano Lattarini wrote on Fri, Jul 15, 2011 at 12:36:17AM CEST: The new code for parsing the testsuite-generated `.log' files, as introduced in commit `v1.11-872-gc96b881', considers each `:test-result:' field anywhere in a `.log' file as a declaration of a test result, and accounts for it as such in the testsuite summary. Unfortunately this could easily cause spurious test failures being reported in the testsuite summary. This happened in practice with the Automake's own testsuite; for example: $ make check TESTS='check12-p.test'; echo exit: $? ... PASS: check12-p.test = 4 of 5 tests failed See tests/test-suite.log Please report to bug-autom...@gnu.org = make[2]: *** [test-suite.log] Error 1 make: *** [check-am] Error 2 exit: 2 This change introduces a new special `:test-result:' END, that, when seen, prevents the rest of the log file from being parsed. For more information, refer to the thread: http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00199.html * lib/am/check.am ($(TEST_SUITE_LOG)): Stop the parsing of a log file as soon as the special :test-result:END directive is seen. Related changes and enhancements. * lib/test-driver: Protect the rest of the log after the result lined with a :test-result:END directive. * tests/parallel-tests-no-spurious-summary.test: New test. * tests/test-driver-end-test-results.test: Likewise. * tests/Makefile.am (TESTS): Update. I'm still not sold on this. It is not as robust as a test protocol could be; also I haven't seen this approach being used in any other test suite environments. Whether some line is considered having a result or not depends on (possibly far-away) context, on whether aggregation with other results has happened. When passed through email, misquoting can change not only the interpretation of the misquoted text (which could be expected) but possibly also of later, correctly quoted (or not quoted) text. Please, this is really important: we need to research the other test protocols, what they do to be robust here. Don't NIH here, because the experience we have is not enough to not mess up this. Consider this research as part of the work needed for the SoC assignment, it is a very important part (as it is not easily corrected once released). Thanks, Ralf
Re: [PATCH 2/5] {test-protocols} parallel-tests: new recognized test result 'ERROR'
* Stefano Lattarini wrote on Fri, Jul 15, 2011 at 12:37:07AM CEST:
* lib/am/check.am ($(TEST_SUITE_LOG)): Recognize a new test result
`ERROR'. Use it when encountering unreadable test logs (previously
a simple `FAIL' was used in this situations).
* lib/test-driver: Set the global test result to `ERROR' when the
test exit status is 99. When doing colorized output, color `ERROR'
results in magenta.
* doc/automake.texi (Log files generation and test results
recording): Update by also listing `ERROR' among the list of valid
`:test-results:' arguments.
* NEWS: Update.
* tests/trivial-test-driver: Update.
* tests/parallel-tests.test: Likewise.
* tests/parallel-tests-harderror.test: Likewise.
* tests/parallel-tests-no-spurious-summary.test: Likewise.
* tests/test-driver-global-log.test: Likewise.
* tests/test-driver-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck2.test: Likewise.
* tests/test-driver-custom-multitest.test: Likewise.
* tests/test-driver-custom-no-html.test: Likewise.
* tests/test-driver-end-test-results.test: Likewise.
* tests/color.test: Likewise. Also, make stricter, and also test
from VPATH.
* tests/color2.test: Likewise, and improve syncing with color.test.
* tests/parallel-tests-exit-statuses.test: New test.
* tests/parallel-tests-console-output.test: Likewise.
* tests/Makefile.am (TESTS): Update.
--- a/NEWS
+++ b/NEWS
@@ -13,6 +13,10 @@ New in 1.11a:
* Changes to Automake-generated testsuite harnesses:
+ - The parallel-tests harness has a new test result 'ERROR', that can be
+used to signal e.g., unexpected or internal errors, or failures to set
+up test case scenarios.
The NEWS entry could be a bit more precise in that the ERROR state was
actually used before already with exit status 99, just that it is named
ERROR now. (Please check whether the semantics of 99 already were in a
stable release, otherwise this NEWS entry could of course advertise it
as new.)
A description of ERROR semantics needs to be part of automake.texi as
well.
Also, are you going to followup with Autoconf to rename Autotest's hard
failure state ERROR as well? We should agree on common naming and
semantics, so the two systems are not harder to learn than necessary.
I'm OK with this change once these issues are resolved.
As a minor detail however, please remove the '' greps, I've already
mentioned why I consider '=' in the output not a good idea.
Thanks,
Ralf
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -9248,10 +9248,10 @@ leading whitespace will not be ignored.
@c Keep this in sync with lib/am/check-am:$(TEST_SUITE_LOG).
The only recognized test results are currently @code{PASS}, @code{XFAIL},
-@code{SKIP}, @code{FAIL} and @code{XPASS}. These results, when declared
-with @code{:test-result:}, can be optionally followed by text holding
-the name and/or a brief description of the corresponding test; the
-@option{parallel-tests} harness will ignore such extra text when
+@code{SKIP}, @code{FAIL}, @code{XPASS} and @code{ERROR}. These results,
+when declared with @code{:test-result:}, can be optionally followed by
+text holding the name and/or a brief description of the corresponding
+test; the @option{parallel-tests} harness will ignore such extra text when
generating @file{test-suite.log} and preparing the testsuite summary.
Also, @code{:test-result:} can be used with a special ``pseudo-result''
@code{END}, that will instruct the testsuite harness to stop scanning
Re: [PATCH 4/5] {test-protocols} tests defs: new auxiliary function 'count_test_results'
* Stefano Lattarini wrote on Fri, Jul 15, 2011 at 12:38:57AM CEST: * tests/defs (count_test_results): New function. * tests/check11.test: Use it. * tests/test-driver-custom-multitest.test: Likewise. * tests/test-driver-custom-multitest-recheck.test: Likewise. * tests/test-driver-custom-multitest-recheck2.test: Likewise. * tests/parallel-tests-log-override-recheck.test: Likewise. * tests/parallel-tests-log-override-recheck.test: Likewise. * tests/parallel-tests-no-spurious-summary.test: Likewise, and slightly improve debugging output. * tests/parallel-tests.test: Make use of `count_test_results'. Also, make grepping of make check output slightly stricter * tests/parallel-tests9.test: Likewise. * tests/parallel-tests-log-override-2.test: Likewise, and throw in a small optimization. OK once dependent changes are. Thanks, Ralf
Re: [PATCH 7/5] tap: some preparatory refactoring (2)
* Stefano Lattarini wrote on Mon, Jul 18, 2011 at 10:28:33AM CEST: * lib/tap-driver (console_output): Renamed ... (report): ... to this, and extended to appropriately register the test results when needed. (testsuite_error, handle_tap_comment, handle_tap_test, handle_tap_plan): Adjusted accordingly. OK once dependent changes are. Thanks, Ralf
Re: [PATCH 5/5] {test-protocols} tap: add experimental TAP-aware driver
* Stefano Lattarini wrote on Mon, Jul 18, 2011 at 10:17:01AM CEST:
On Friday 15 July 2011, Stefano Lattarini wrote:
* doc/automake.texi (Using the TAP test protocol): New section.
(Overview of Custom Test Drivers Support): Minor updates.
* lib/tap-driver: New script, TAP-aware test driver for Automake;
implemented in perl and based on TAP::Parser.
* lib/Makefile.am (dist_script_DATA): Add it.
[...]
And consider this squashed in:
The squash-in is OK (but I'm not done with the original patch yet).
Thanks,
Ralf
--- a/tests/tap-color.test
+++ b/tests/tap-color.test
@@ -52,7 +52,8 @@ AUTOMAKE_OPTIONS = color-tests
AM_TEST_LOG_DRIVER_FLAGS = --comments
TEST_LOG_COMPILER = cat
TEST_LOG_DRIVER = $(PERL) $(srcdir)/tap-driver
-TESTS = all.test skip.test bailout.test plan-errors.test
+TESTS = all.test skip.test bail.test badplan.test noplan.test \
+few.test many.test order.test
END
cat all.test 'END'
@@ -66,15 +67,41 @@ ok 5 - zardoz # TODO
END
cat skip.test 'END'
-1..0 # Whole script not run
+1..0 # SKIP whole script
END
-cat errors.test 'END'
+cat bail.test 'END'
1..1
ok 1
Bail out!
END
+cat badplan.test 'END'
+# foo
+1..1
+ok 1
+END
+
+cat noplan.test 'END'
+ok 1
+END
+
+cat few.test 'END'
+1..2
+ok 1
+END
+
+cat many.test 'END'
+1..1
+ok 1
+ok 2
+END
+
+cat order.test 'END'
+1..1
+ok 5
+END
+
$ACLOCAL
$AUTOCONF
$AUTOMAKE
@@ -92,9 +119,15 @@ test_color ()
cat stdout | grep ^${blu}SKIP${std}: all\.test 3 - baz # SKIP sk$
cat stdout | grep ^${red}FAIL${std}: all\.test 4 - quux$
cat stdout | grep ^${red}XPASS${std}: all\.test 5 - zardoz # TODO$
- cat stdout | grep ^${blu}SKIP${std}: skip\.test # Whole script not run$
- cat stdout | grep ^${grn}PASS${std}: bailout\.test 1$
- cat stdout | grep ^${mgn}ERROR${std}: all\.test - Bail out!$
+ cat stdout | grep ^${blu}SKIP${std}: skip\.test - whole script$
+ cat stdout | grep ^${grn}PASS${std}: bail\.test 1$
+ cat stdout | grep ^${mgn}ERROR${std}: bail\.test - Bail out!$
+ cat stdout | grep ^${mgn}ERROR${std}: badplan\.test - test plan in middle
of output$
+ cat stdout | grep ^${mgn}ERROR${std}: noplan\.test - missing test plan$
+ cat stdout | grep ^${mgn}ERROR${std}: few.test - too few tests run
(expected 2, got 1)$
+ cat stdout | grep ^${mgn}ERROR${std}: many.test - too many tests run
(expected 1, got 2)$
+ cat stdout | grep ^${mgn}ERROR${std}: many.test 2 # UNPLANNED$
+ cat stdout | grep ^${mgn}ERROR${std}: order.test 5 # OUT-OF-ORDER
(expecting 1)$
# Diagnostic messages shouldn't be colorized.
cat stdout | grep ^# all\.test: Hi! I shouldn't be colorized!$
Re: [PATCH 8/5] tap: support colorization of testsuite progress output
* Stefano Lattarini wrote on Mon, Jul 18, 2011 at 10:30:56AM CEST:
* lib/tap-driver (%COLORS): New variable (definition extracted
from `lib/am/check.am:$(am__tty_colors)', with some obvious
adjustments.
(report): Adjust to colorize console output when required,
using ...
(decorate_result): ... this new function.
(colored): New function, used by the one above.
* tests/tap-summary.test: Also run the checks when `color-tests'
is in use.
* tests/Makefile.am (XFAIL_TESTS): Remove `tap-color.test'.
OK once dependent changes are in, and with nits below addressed.
Thanks,
Ralf
--- a/lib/tap-driver
+++ b/lib/tap-driver
+# Keep this in sync with `lib/am/check.am:$(am__tty_colors)'.
+my %COLOR = (
+ red = '[0;31m',
I'm sure perl has a way to encode ESC without a literal ESC.
+ grn = '[0;32m',
+ lgn = '[1;32m',
+ blu = '[1;34m',
+ mgn = '[0;35m',
+ brg = '[1m',
+ std = '[m',
+);
@@ -211,17 +222,39 @@ sub stringify_test_result ($)
+sub decorate_result ($)
+{
+ return $_[0] unless $cfg{color-tests};
+ # Best way to simulate a 'switch' construct here.
Please don't, that only obfuscates the code. automake.in uses long
if ... else lists. If you don't like that, use a map.
+ for (@_)
+{
+ $_ eq ERROR and return colored ('mgn', $_);
+ $_ eq PASS and return colored ('grn', $_);
+ $_ eq XPASS and return colored ('red', $_);
+ $_ eq FAIL and return colored ('red', $_);
+ $_ eq XFAIL and return colored ('lgn', $_);
+ $_ eq SKIP and return colored ('blu', $_);
+ return $_; # Don't colorize unknown stuff.
+}
+}
+
sub report ($;$)
{
my ($msg, $result, $explanation) = (undef, @_);
if ($result =~ /^(?:X?(?:PASS|FAIL)|SKIP|ERROR)/)
{
- $msg = $result: $test_script_name;
+ $msg = : $test_script_name;
add_test_result $result;
}
elsif ($result eq #)
{
- $msg = # $test_script_name:;
+ $msg = $test_script_name:;
}
else
{
@@ -229,10 +262,11 @@ sub report ($;$)
}
$msg .= $explanation if defined $explanation;
$msg .= \n;
- print OLDOUT $msg;
+ # Output on console might be colorized.
+ print OLDOUT decorate_result ($result) . $msg;
# Log the result in the log file too, to help debugging (this is
# especially true when said result is a TAP error or Bail out!).
- print $msg;
+ print $result . $msg;
}
sub testuite_error ($)
[...]
Re: [PATCH] {testsuite-work} tests: fix typos, grammaros and other blunders in comments
* Stefano Lattarini wrote on Sat, Jul 16, 2011 at 02:30:40PM CEST: A patch doing various (mostly cosmetic) fixes to comments in tests. Before pushing, I'll allow 72 hours for comments and suggestsions. OK thanks. Subject: [PATCH] tests: fix typos, grammaros and other blunders in comments * tests/all.test: In comments, fix typos, grammaros and/or botched capitalization, and/or improve wording. * tests/alpha.test: Likewise. * tests/amopts-variable-expansion.test: Likewise. [...] The GCS now allows you to omit specifying all affected file names in the log entry, and just use something like All affected files changed. For patches such as this one, it makes sense, as there's neither copyright-relevant changes nor is it too important for later analysis whether some file was affected or not. Cheers, Ralf
Re: [PATCH] {GSoC} parallel-tests: simplify testsuite summary
On Thu, Jul 07, 2011 at 07:02:58PM +0200, Stefano Lattarini wrote: On Wednesday 06 July 2011, Ralf Wildenhues wrote: But tools like autobuild have another requirement: they would like to be able to detect, as reliably as possible, some statistics based on whatever output is thrown at them. IOW, they usually also would like to detect the style of testsuite driver that this comes from. But this is impossible to do reliably, as the new automake harness allows the use of multiple test drivers in the same Makefile.am, and even allow the user to define his own test drivers. Or am I misunderstanding what you're saying? No, you're not. The art is defining something that is as reliable as possible. This is easiest done if it can be done on a line by line basis. Your proposed format doesn't make that easy (but we could add a clue in one of the first/early summary lines to make that easier, if we also make it easy to detect the end of the summary). Ah, now I see what you mean. Maybe even add the version and type of the testsuite summary, as in: Well, I'd hope to be able to do without a versioning of the testsuite. That would denote a kind of failure to define a stable, easy to parse, interface. Also, if you want something extensible, then don't treat extensibility like an afterthought: you could just omit any line with 0 tests, no? Or better again add an option to do so (I personally prefer a count to be reported even if it is 0, but I understand that some other people might have a different preference). Point is, if you add lines for new data in the future, the user has to be resilient against them anyway. In which case you can also just omit the unimportant data now. (Note some of the 0 data may be important, but say ERROR is probably not.) please show a (non-code) definition of the output that would be sufficient for someone to write an (say, autobuild) parser that grabs testsuite results You mean a testsuite summary, right? If you do, here is a rough but good enough definition: A line composed by 76 `=' characters, followed by a line beginning with the string Testsuite summary, followed again by a line composed by 76 `=' characters -- this marks the beginning of the testsuite results' summary. The first line composed by 76 `=' characters that comes after those -- this marks the end of the testsuite results summary. Each of these results should be reported inside the testsuite summary, on a line if its own, matching the following regular expression: ^#\s*KIND-OF-RESULT:\s*COUNT(\s.*)?$ Any line in the testsuite summary that doesn't obey this format doesn't denote a test result count, and should thus be ignored. OK, let's digest this. Assume my summary goes through email quoting, a w32 user, gets inline-quoted in a reply. Then /^\([|] \)*/ should be ignored, just as \r near EOL. '=' characters are bad because q-p will encode them as =3D or something. Newline mangling can easily introduce empty lines every other line. Then, the summary is probably not identical to another known format, nor sufficiently easy to distinguish (which we already discussed). Ideally, an unobtrusive way to fix that would be nice (where printing automake would already count as obtrusive, as it shouldn't really matter to the casual user who wrote this testsuite driver). But clarity is only one criterion. Let's define all criteria is has to meet. Suggestions? I really only had clarity in mind :-) Yeah, that's why I suggested stepping back and looking at it again. ;-) Maybe this time we should clearly document the testsuite summary format? Exactly. Could you please open a bug report about that on bug-automake? This way, even if we don't manage to write the documentation about this by the end of GSoC, we will still remember to do so for 1.12. Will do. And now that I've been forced to spell that out, I see that the `--color' option name is poorly chosen; I'd say we change it to `--maybe-color', which is more faithful to the intended seantics. OK? I've done that in the attached squash-in. What if the user wants color really hard? You mean want them *even in the test-suite.log*? Hmm. --color is good enough, if it forces color on stdout, IMVHO. Oh, is this user-influenceable at all btw? For the testsuite summary that goes on stdout, yes, the user has complete control over its colorizing (as he has for the colorizing of the testsuite progress output); for the summary that goes in `test-suite.log', no, the user can't colorize it, period. This seems to me the most sensible policy. OK. Yeah, except now the summary looks _a lot_ like output from a single test. Quite confusing for a simple parser (again, look at the autobuild scripts for an example). But IMVHO, the leading `#' characters avoids confusion for the parser, and the enclosing `=...' lines avoid confusion
Re: [PATCH] {GSoC} tests defs: new auxiliary function 'count_test_results'
Hi Stefano, and sorry for yet another delay, * Stefano Lattarini wrote on Tue, Jul 05, 2011 at 10:00:25PM CEST: Posted below is a patch that I'll soon apply to the temporary branch 'GSoC/experimental/test-results-work'; it is aimed at reducing code duplication a bit in the current testsuite, at the same time offering a simpler and clearer idiom for checks about test counting. Note that the real usefulness of the patch will become manifest only after I've posted the new testcases on TAP support (I hope to get to it by this evening). That looks fine, thanks! Ralf Subject: [PATCH] tests defs: new auxiliary function 'count_test_results' * tests/defs (count_test_results): New function. * tests/test-driver-custom-multitest.test: Use it. * tests/test-driver-custom-multitest-recheck.test: Likewise. * tests/test-driver-custom-multitest-recheck2.test: Likewise.
Re: [PATCH] {GSoC} parallel-tests: simplify testsuite summary
Hi Stefano,
* Stefano Lattarini wrote on Sun, Jul 03, 2011 at 03:31:22PM CEST:
Prefer a more deterministic, tabular format for the testsuite
summary, always listing the numbers of passed, failed, xfailed,
xpassed, skipped and errored tests, even when these numbers are
zero. This simplify the logic of testsuite summary creation,
makes it more easily machine-parseable, and will probably allow
for easier addition of new kinds of test results in the future.
Applied to the temporary branch 'GSoC/experimental/test-results-work'.
This looks a bit verbose to me, also IIUC it introduces yet another
format that is different from our previous summary and different from
the summaries of any other testsuite environments. Any chance we can
lessen the NIH?
I've rejected a reformatting of the results before (see the list
archives for details). I'm not totally opposed, if there is a real
advantage /for the user/, and if things are changing a lot anyway,
but we should definitely change into something that we think can be
stable for several years. This is end-user API, a lot of stuff that's
not even using Automake (but only running a testsuite of a package that
uses Automake) can be relying on this. IOW, things like GSRC, the
autobuild parser, or whatnot else (that we don't have any control over)
that greps our output, and now will have to maintain yet another syntax
forever (since of course the old syntax won't die in the next several
years).
Subject: [PATCH] parallel-tests: simplify testsuite summary
Prefer a more deterministic, tabular format for the testsuite
summary, always listing the numbers of passed, failed, xfailed,
xpassed, skipped and errored tests, even when these numbers are
zero. This simplify the logic of testsuite summary creation,
makes it more easily machine-parseable, and will probably allow
for easier addition of new kinds of test results.
* lib/am/check.am (am__tty_colors_dummy): New make variable, to
reduce code duplication. Extracted from previous versions of
$(am__tty_colors), and extended by defining two new variables
`$mgn' and `$brg'.
[%?COLOR%, %!?COLOR%] (am__tty_colors): Use that new variable.
(am__text_box): Delete, is not needed anymore.
($(TEST_SUITE_LOG)): Rewrite associated rules to implement the
new testsuite summary format.
* NEWS: Update.
* tests/check10.test: Don't run with the parallel-tests harness
too, that makes no sense anymore.
* tests/color.test: Update and adjust.
* tests/color2.test: Likewise.
* tests/parallel-tests.test: Likewise.
* tests/parallel-tests6.test: Likewise.
* tests/parallel-tests9.test: Likewise.
* tests/parallel-tests-unreadable-log.test: Likewise.
* tests/parallel-tests-empty-testlogs.test: Likewise.
* tests/parallel-tests-log-override-recheck.test: Likewise.
* tests/parallel-tests-no-spurious-summary.test: Likewise.
* tests/test-driver-custom-multitest.test: Likewise.
* tests/test-driver-end-test-results.test: Likewise.
* tests/parallel-tests-no-color-in-log.test: New test.
* tests/testsuite-summary-color.test: Likewise.
* tests/testsuite-summary-count.test: Likewise.
* tests/testsuite-summary-count-many.test: Likewise.
* tests/testsuite-summary-reference-log.test: Likewise.
* tests/testsuite-summary-checks.sh: New auxiliary script, used
by the new tests above.
* tests/extract-testsuite-summary: Likewise.
* tests/trivial-test-driver: Optimize for speed when there are
lots of of tests.
* tests/Makefile.am (EXTRA_DIST): Distribute them.
(testsuite-summary-color.log, testsuite-summary-count.log): Depend
on them.
(testsuite-summary-count-many.log): Depend on the auxiliary scripts
'trivial-test-driver' and 'extract-testsuite-summary'.
(TESTS): Update.
+## When writing the test summary to the console, we want to color a line
+## reporting the count of a kind of result *only* if at least one test
s/a kind of/some/
+## experienced such a result. This function is handy in this regard.
+ result_count () \
+ { \
+ if test x$$1 = x--color; then \
+ colorize=yes; \
+ elif test x$$1 = x--no-color; then \
What if output doesn't go to a terminal? Does --no-color get set then,
or colorize?
+ colorize=no; \
+ else \
+ echo $@: invalid 'result_count' usage 2; exit 4; \
+ fi; \
+ shift; \
+ desc=$$1 count=$$2; \
+ if test $$colorize = yes test $$count -gt 0; then \
+ color_start=$$3 color_end=$$std; \
+ else \
+ color_start= color_end=; \
+ fi; \
+ echo $${color_start}# $$desc $$count$${color_end}; \
+ }; \
+## A shell function that creates the testsuite summary. We need it
+## because we have to create *two* summaries, one for test-suite.log,
+## and a possibly-colorized one for console output.
+ create_testsuite_report () \
+ { \
+ result_count $$1 tests: $$all $$brg; \
+ result_count $$1 pass: $$pass $$grn;
Re: bug#8969: improve synchronization between examples in the manual and test cases
* Stefano Lattarini wrote on Fri, Jul 01, 2011 at 12:59:53AM CEST: Subject: [PATCH] docs, tests: synchronize examples from docs to tests * tests/README (Writing test cases): Give suggestions on how to keep test cases and examples in the documentation synchronized. * doc/automake.texi: Improve or fix existing testcase-referencing comments, and add many new ones. * HACKING (Administrivia): Suggest to test complex examples and idioms from the manual. * tests/specflg8.test: Improve synchronization with the example in the manual. * tests/output11.test:Likewise. * tests/txinfo21.test:Likewise. * tests/interp.test: Likewise. Since we are at it, and enable the `errexit' shell flag, do related changes, and add trailing `:'command. * tests/amhello-cflags.test: New test. * tests/amhello-cross-compile.test: Likewise. * tests/amhello-binpkg.test: Likewise. * tests/tests-environment-backcompat: Likewise. * tests/parallel-tests-log-compiler-example.test: Likewise. * tests/Makefile.am (TESTS): Update. --- /dev/null +++ b/tests/amhello-binpkg.test @@ -0,0 +1,44 @@ +# Document an example from the manual about the `amhello' package: +# using DESDIR to build simple, no-frills binary packages. (DESTDIR was already fixed IIUC) +required=i586-mingw32msvc-gcc +. ./defs || Exit 1 + +set -e + +cp $testsrcdir/../doc/amhello-1.0.tar.gz . \ + || fatal_ cannot get amhello tarball + +tar zxf amhello-1.0.tar.gz The z flag is not portable to all tars, the portable spelling is gzip -dc amhello-1.0.tar.gz | tar xf - (several instances). I'm ok with fixing the manual also, although it's usually clear for people still having to use those vendor tars (and inconvenient for the rest). +cd amhello-1.0 + +./configure --prefix /usr +make +make DESTDIR=`pwd`/inst install +cd inst +find . -type f -print ../files.lst +tar cvf amhello-1.0-i686.tar.gz `cat ../files.lst` t +LC_ALL=C sort t tar.got + +diff - tar.got 'END' +./usr/bin/hello +./usr/share/doc/amhello/README +END [...] Nice patch btw! Thanks, Ralf
Re: bug#8969: improve synchronization between examples in the manual and test cases
* Stefano Lattarini wrote on Wed, Jul 06, 2011 at 10:32:56AM CEST: Oops, sorry. Fixed by the attached patch. OK for maint? I'll wait the customary 72 hours before pushing. Thanks. OK, but please remove all the comments about tar unportability in the tests. It's so obvious. ;-) (and there are probably a dozen other places in the Automake source tree that you'd have to put the comment at as well, for any amount of consistency. I think having one comment, the one that already exists in lib/am/distdir.am, is fully sufficient.) I'm ok with fixing the manual also, although it's usually clear for people still having to use those vendor tars (and inconvenient for the rest). Yes, I'd say we leave the examples in manual untouched. Agreed? OK. Subject: [PATCH] tests: portability fixes in tests on amhello examples * tests/amhello-binpkg.test: Don't use tar xzf too.tag.gz to extract a gzip-compressed tarball, that's unportable to some tar implementations; use the gzip -dc fo.tar.gz | tar xf - idiom instead. * tests/amhello-cflags.test: Likewise. * tests/amhello-cross-compile.test: Likewise. Thanks, Ralf
Re: [PATCH] {GSoC} parallel-tests: simplify testsuite summary
* Stefano Lattarini wrote on Wed, Jul 06, 2011 at 02:49:15PM CEST:
On Wednesday 06 July 2011, Ralf Wildenhues wrote:
* Stefano Lattarini wrote on Sun, Jul 03, 2011 at 03:31:22PM CEST:
Prefer a more deterministic, tabular format for the testsuite
summary, always listing the numbers of passed, failed, xfailed,
xpassed, skipped and errored tests, even when these numbers are
zero. This simplify the logic of testsuite summary creation,
makes it more easily machine-parseable, and will probably allow
for easier addition of new kinds of test results in the future.
Applied to the temporary branch 'GSoC/experimental/test-results-work'.
This looks a bit verbose to me, also IIUC it introduces yet another
format that is different from our previous summary and different from
the summaries of any other testsuite environments. Any chance we can
lessen the NIH?
Well, I'm surely open to revisit the issue later if we find an existing
format that offers all the information my new format does, is as easily
parseable, and does not suffer of the NIH syndrome :-).
Wait. Do you know what NIH is? Not Invented Here means: you create
something anew, although there exists already other code that does the
same or a similar thing, or a similar protocol or format.
In University, not copying is considered good style (especially with
scientific results). In programming, copying is good, and reusing is
even better. But whether the code you copy (or which you reuse) itself
suffered from NIH syndrome is often not so relevant. ;-
But the older
format has to go IMHO, for two orthogonal and good reasons:
- the logic necessary to create a summary that obeys that format is
complex, unclear, and not at all extensible; and
Good point.
- it's pretty hard to verify the correctness of the resulting summary
by means of grepping checks.
OK.
But tools like autobuild have another requirement: they would like to be
able to detect, as reliably as possible, some statistics based on
whatever output is thrown at them. IOW, they usually also would like to
detect the style of testsuite driver that this comes from. This is
easiest done if it can be done on a line by line basis. Your proposed
format doesn't make that easy (but we could add a clue in one of the
first/early summary lines to make that easier, if we also make it easy
to detect the end of the summary).
Also, if you want something extensible, then don't treat extensibility
like an afterthought: you could just omit any line with 0 tests, no?
Before implementing more, please show a (non-code) definition of the
output that would be sufficient for someone to write an (say, autobuild)
parser that grabs testsuite results from a stream of otherwise garbage
that generally works, without looking at your code. Then, let's see
that, and proceed from there. Thanks.
I've rejected a reformatting of the results before (see the list
archives for details). I'm not totally opposed, if there is a real
advantage /for the user/,
Personally I can say that, as a user, I find the new format definitely
clearer and more pleasant (at least when 'color-tests' is in use); and
this independently from the considerations above (that were admittedtly
done with the developer/tester hat on).
But clarity is only one criterion. Let's define all criteria is has to
meet.
This is end-user API, a lot of stuff that's
not even using Automake (but only running a testsuite of a package that
uses Automake) can be relying on this. IOW, things like GSRC, the
autobuild parser, or whatnot else (that we don't have any control over)
that greps our output, and now will have to maintain yet another syntax
forever (since of course the old syntax won't die in the next several
years).
Maybe this time we should clearly document the testsuite summary format?
Exactly.
Or even offer a small script to extract it from the make check output,
so that those project won't bash us too hard? (Note that I have written
a rough version of such a script in my patch, for use in our own test
cases; but it can't handle multiple summaries in the same output though).
I should read the whole post before starting to reply, obviously.
;-)
+ result_count () \
+ { \
+ if test x$$1 = x--color; then \
+ colorize=yes; \
+ elif test x$$1 = x--no-color; then \
What if output doesn't go to a terminal? Does --no-color get set then,
or colorize?
In that case, `$colorize' is set to yes, but the variables that should
hold the color escape sequences ($red, $grn, etc.) are set to the empty
string (by the code in $(am__tty_clors)), unless the user explicitly
required otherwise by setting `AM_COLOR_TESTS' to always. This is the
same colorizing logic that is supported for the testsuite progress
output.
And now that I've been forced to spell that out, I see that the `--color'
option name is poorly chosen; I'd say we change
Re: [PATCH] {master} coverage: new test on parallel-tests TESTS runtime overriding
* Stefano Lattarini wrote on Thu, Jun 30, 2011 at 04:40:51PM CEST:
I'd like to add this new test case to master. OK?
Sure, note typo below.
This could actually find bugs in older makes, but hey, we'd want to know
about them.
Thanks,
Ralf
Subject: [PATCH] coverage: new test on parallel-tests TESTS runtime overriding
* tests/parallel-tests-cmdline-override.test: New test, check that
we can use indirections when overriding TESTS and TEST_LOGS from
the command line.
* tests/Makefile.am (TESTS): Update.
--- /dev/null
+++ b/tests/parallel-tests-cmdline-override.test
+# Check that we can use indirections when overriding TESTS and
+# TEST_LOGS from the command line.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat configure.in 'END'
+AC_OUTPUT
+END
+
+cat Makefile.am 'END'
+TEST_EXTENSIONS = .test .t
+TEST_LOG_COMPILER = cat
+T_LOG_COMPILER = cat
+TESTS = bad.test
+var1 = b.test $(var2)
+var2 = c.test
+var3 = d.d
+var4 = e
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+rm -f config.log # Not to create false psotives below.
positives
and s/Not to/Do not/ I guess.
+
+LC_ALL=C sort exp-log 'END'
+a.log
+b.log
+c.log
+d.log
+e.log
+test-suite.log
+END
+
+LC_ALL=C sort exp-out 'END'
+PASS: a.t
+PASS: b.test
+PASS: c.test
+PASS: d.t
+PASS: e.test
+END
+
+do_check ()
+{
+ env $@ $MAKE -e check stdout || { cat stdout; Exit 1; }
+ cat stdout
+ grep '^PASS:' stdout | LC_ALL=C sort got-out
+ cat got-out
+ ls . | grep '\.log$' | LC_ALL=C sort got-log
+ cat got-log
+ st=0
+ diff exp-out got-out || st=1
+ diff exp-log got-log || st=1
+ return $st
+}
+
+tests='a.t $(var1) $(var3:.d=.t) $(var4:=.test)'
+test_logs='a.log $(var1:.test=.log) $(var3:.d=.log) $(var4:=.log)'
+
+touch a.t b.test c.test d.t e.test
+
+do_check TESTS=$tests
+do_check TEST_LOGS=$test_logs
Re: [FYI] {master} docs: fix unportable example of AM_TESTS_ENVIRONMENT usage
Hi Stefano, * Stefano Lattarini wrote on Thu, Jun 23, 2011 at 10:56:31PM CEST: On Thursday 23 June 2011, Ralf Wildenhues wrote: -AM_TESTS_ENVIRONMENT = exec 92; warn_fileno=9; export warn_fileno; This example served two purposes: use of AM_TESTS_ENVIRONMENT, and showing how to produce output while the test is running. Your patch removes the second feature. Can we have it back, ideally in a portable fashion? Alas, no. The only way to do that portably is to either: 1. add a trailing `; 2' to AM_TESTS_ENVIRONMENT, *and* force the user not to define TESTS_ENVIRONMENT; or 2. use TESTS_ENVIRONMENT, not AM_TESTS_ENVIRONMENT, in the Makefile.am (still with a trailing `; 2' obviously). Actually, it used to be nonportable to put the redirection before the command. I'm fairly sure that's only with historic shells and not an issue in practice any more (Sven Mascheck's pages will have the details for the curious). In each case, the user's namespace is invaded, and we go against our own advice (in [1] we violate the always terminate AM_TESTS_ENVIRONMENT with a semicolon rule, in [2] we violate don't define TESTS_ENVIRONMENT in the Makefile.am, it's user-reserved rule). Also, the example above belongs IMHO more in a FAQ rather than in a reference manual. Are you volunteering to update the FAQ? ;-) Thanks, Ralf
Re: [MERGE REQUEST] merge patches for AM_TESTSUITE_ENVIRONMENT into maint
* Stefano Lattarini wrote on Thu, Jun 30, 2011 at 05:12:38PM CEST: I think that the patches introducing support for AM_TESTSUITE_ENVIRONMENT should be merged into maint, as they'd make a nice addition for 1.11.2. I've verified that almost all these patches are based off of maint, and the few that aren't (e.g., `v1.11-867-g1e005df') I don't have v1.11-867-g1e005df nor v1.11-351-g3e334a2 in my tree. Can you make a branch available that you'd like to merge? Or alternatively, links to the patch mails in the archive? are just testsuite additions (and could probably be cherry-picked anyway, but I'm not sure whther that would be worthwhile). My proposal is to push the last relevant commit (should be `v1.11-351-g3e334a2') to a new public branch 'am-tests-environment', tie up the last loose ends (if any), and finally merge that branch into maint. Objections? Thanks, Ralf
Re: [PATCH] parallel-tests: new recognized test result 'ERROR'
* Stefano Lattarini wrote on Thu, Jun 30, 2011 at 04:58:21PM CEST:
I've applied the attached patch to the 'GSoC/experimental/test-results-work'
branch Note that this is not an FYI, since that branch is temporary, and can
thus be amended and modified. Reviews are welcome!
Just a couple of quick thoughts:
- is ERROR the same term other testing frameworks use for similar
semantics? Let's strive to be consistent.
- will users confuse FAIL and ERROR and try to use the latter where the
former is appropriate (or vice versa)?
Let's try to avoid inventing different terms where and when we can.
Changing semantics requires a NEWS entry. It's actually an
incompatibility, but a very minor one, so it's not a big problem.
Thanks,
Ralf
Subject: [PATCH] parallel-tests: new recognized test result 'ERROR'
* lib/am/check.am ($(TEST_SUITE_LOG)): Recognize a new test result
`ERROR'. Use it when encountering unreadable test logs (previously
a simple `FAIL' was used in this situations).
* lib/test-driver: Set the global test result to `ERROR' when the
test exit status is 99. When doing colorized output, color `ERROR'
results in magenta.
* doc/automake.texi (Log files generation and test results
recording): Update by also listing `ERROR' among the list of valid
`:test-results:' arguments.
* tests/trivial-test-driver: Update.
* tests/parallel-tests.test: Likewise.
* tests/parallel-tests-harderror.test: Likewise.
* tests/parallel-tests-no-spurious-summary.test: Likewise.
* tests/test-driver-global-log.test: Likewise.
* tests/test-driver-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck2.test: Likewise.
* tests/test-driver-custom-multitest.test: Likewise.
* tests/test-driver-custom-no-html.test: Likewise.
* tests/test-driver-end-test-results.test: Likewise.
* tests/color.test: Likewise, and make stricter.
* tests/color2.test: Likewise, and improve syncing with color.test.
* tests/parallel-tests-exit-statuses.test: New test.
* tests/parallel-tests-console-output.test: New test.
* tests/Makefile.am (TESTS): Update.
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -9248,10 +9248,10 @@ leading whitespace will not be ignored.
@c Keep this in sync with lib/am/check-am:$(TEST_SUITE_LOG).
The only recognized test results are currently @code{PASS}, @code{XFAIL},
-@code{SKIP}, @code{FAIL} and @code{XPASS}. These results, when declared
-with @code{:test-result:}, can be optionally followed by text holding
-the name and/or a brief description of the corresponding test; the
-@option{parallel-tests} harness will ignore such extra text when
+@code{SKIP}, @code{FAIL}, @code{XPASS} and @code{ERROR}. These results,
+when declared with @code{:test-result:}, can be optionally followed by
+text holding the name and/or a brief description of the corresponding
+test; the @option{parallel-tests} harness will ignore such extra text when
ERROR semantics are not actually described.
generating @file{test-suite.log} and preparing the testsuite summary.
Also, @code{:test-result:} can be used with a special ``pseudo-result''
@code{END}, that will instruct the testsuite harness to stop scanning
--- a/lib/am/check.am
+++ b/lib/am/check.am
@@ -66,7 +66,7 @@ include inst-vars.am
## appended.
##
## In addition to the magic exit 77 means SKIP feature (which was
-## imported from automake), there is a magic exit 99 means FAIL feature
+## imported from automake), there is a magic exit 99 means ERROR feature
## which is useful if you need to issue a hard error no matter whether the
## test is XFAIL or not. You can disable this feature by setting the
## variable DISABLE_HARD_ERRORS to a nonempty value.
@@ -153,7 +153,7 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
## Readable test logs.
list2=`for f in $$list; do test ! -r $$f || echo $$f; done`; \
## Each unreadable test log counts as a failed test.
- results1=`for f in $$list; do test -r $$f || echo FAIL; done`; \
+ results1=`for f in $$list; do test -r $$f || echo ERROR; done`; \
## Now we're going to extract the outcome of all the testcases from the
## test logs.
results2=''; \
@@ -187,6 +187,10 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
skip=`echo $$results | grep -c '^SKIP'`; \
xfail=`echo $$results | grep -c '^XFAIL'`;\
xpass=`echo $$results | grep -c '^XPASS'`;\
+ error=`echo $$results | grep -c '^ERROR'`;\
+## FIXME: for the moment, we count errors as failures, otherwise the code
+## that display the testsuite summary will become too much complicated.
grammar nits: s/display/s/; s/much //
+ fail=`expr $$fail + $$error`; \
failures=`expr $$fail + $$xpass`; \
all=`expr $$all - $$skip`; \
if test
Re: [MERGE REQUEST] merge patches for AM_TESTSUITE_ENVIRONMENT into maint
* Ralf Wildenhues wrote on Thu, Jun 30, 2011 at 10:52:16PM CEST: * Stefano Lattarini wrote on Thu, Jun 30, 2011 at 05:12:38PM CEST: I think that the patches introducing support for AM_TESTSUITE_ENVIRONMENT should be merged into maint, as they'd make a nice addition for 1.11.2. I've verified that almost all these patches are based off of maint, and the few that aren't (e.g., `v1.11-867-g1e005df') I don't have v1.11-867-g1e005df nor v1.11-351-g3e334a2 in my tree. D'oh. EWRONGREPO. I still don't have the latter commit in my tree, and the former has many many commits on top of maint. So no, you cannot merge that into maint. That would require more than just a cursory review, and I don't see the gain right now. are just testsuite additions (and could probably be cherry-picked anyway, but I'm not sure whther that would be worthwhile). My proposal is to push the last relevant commit (should be `v1.11-351-g3e334a2') to a new public branch 'am-tests-environment', tie up the last loose ends (if any), and finally merge that branch into maint. Objections? Sorry, Ralf
Re: [FYI] {master} docs: fix unportable example of AM_TESTS_ENVIRONMENT usage
* Stefano Lattarini wrote on Thu, Jun 30, 2011 at 10:45:02PM CEST: On Thursday 30 June 2011, Ralf Wildenhues wrote: I'm fairly sure that's only with historic shells and not an issue in practice any more (Sven Mascheck's pages will have the details for the curious). Anyway, I do think that Eric's original proposal of extending the parallel-tests harness to allow arbitrary and portable fd redirections is the way to go eventually; Link? here are my reasons: - it would be pretty easy to implement and document; - the testsuite harness is being revamped anyway; - that change would also benefit all the future custom and built-in test drivers; - the current hack with TESTS_ENVIRONMENTS = ...; 92, while continuing to work, will prevent the use of AM_TESTS_ENVIRONMENT :-( In each case, the user's namespace is invaded, and we go against our own advice (in [1] we violate the always terminate AM_TESTS_ENVIRONMENT with a semicolon rule, in [2] we violate don't define TESTS_ENVIRONMENT in the Makefile.am, it's user-reserved rule). Also, the example above belongs IMHO more in a FAQ rather than in a reference manual. Are you volunteering to update the FAQ? ;-) Oh no ;-) Not this summer at least. That's a pretty strong statement, and when accompanied by a documentation regression (if I may exaggerate a bit), I'm not too happy when I read this. Please do not consider documentation work as second class citicen. Actually, writing (at least a sketch of) the documentation before implementing a feature is a very good way to get more organized and better design, IMVHO. Thanks, Ralf
Re: [PATCH] docs: avoid a footnote, some related rewordings and improvements (was: Re: [Automake-commit] [SCM] GNU Automake branch, maint, updated. v1.11-393-gc1040a7)
* Stefano Lattarini wrote on Tue, Jun 28, 2011 at 08:24:13AM CEST: On Monday 27 June 2011, Ralf Wildenhues wrote: Sure. Well, do they use some format already? Well, there are only two of them, and both follows this format: @c The test case for the setup described here is @c test/subdircond2.test @c Try to keep it in sync. (which is wrongish BTW, as the testsuite subdirectory is `tests/', not `test/') I'd go with one of these simple formats: @c Keep in sync with subdircond2.test I like this one (with a trailing dot ;-) ;-) @c The following is covered by subdircond2.test which are short and easy to grep; and I'd document this idiom in tests/README. I can do that by this evening if nobody beats me. Thanks, Ralf
Re: [FYI] New public temporary branch, about documentation for custom test drivers
Hi Stefano,
* Stefano Lattarini wrote on Sun, Jun 26, 2011 at 02:20:48PM CEST:
I've pushed the documentation work I've done so far to the temporary
public branch GSoC/experimental/docs-custom-test-drivers, in case
anyone wants to take a look at it (I had hoped to finish it this morning,
but I didn't, and I'll be AFK until this evening, so there's no point in
delaying even more). Notice that that branch is bound to be amended and
rebased, and finally deleted (it won't be merged in the 'test-protocols'
branch, rather its changes will be re-applied on the top of that).
A few comments from a cursory look:
+2011-06-25 Stefano Lattarini stefano.lattar...@gmail.com
+
+ docs: document custom test drivers and protocols
+ * doc/automake.texi (Simple Tests): Note that the TESTS_ENVIRONMENT
+ use suggested here is not portable to 'parallel-tests'.
+ (Simple Tests using parallel-tests): Document new restrictions on
+ the uses of TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT.
+ (Custom Test Drivers): New section and node.
+ (Overview of Custom Test Drivers Support): New subsection.
+ (Declaring Custom Test Drivers in @file{Makefile.am}): Likewise.
+ (APIs for Custom Test Drivers): Likewise.
+ (Options): Update description of color-tests.
+ * lib/am/check (recheck, recheck-html): Minor adjustments to better
+ conform to the documentation (this should cause no semantic changes
+ w.r.t. the former behaviour); minor improvements and extensions to
+ existing comments.
+ * tests/test-driver-create-log-dir.test: New test.
+ * tests/test-driver-strip-vpath.test: Likewise.
+ * tests/test-driver-global-log.test: Likewise.
+ * tests/test-driver-recheck.test: Likewise.
+ * tests/Makefile.am (TESTS): Update.
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -8691,10 +8708,12 @@ Simple Tests
TESTS = foo.pl bar.pl baz.pl
@end example
-Note that the @option{parallel-tests} driver provides a more elegant
-way to achieve the same effect, freeing the @code{TESTS_ENVIRONMENT}
-variable for the user to override (@pxref{Simple Tests using
-parallel-tests}).
+It's important to note that this last use of @code{TESTS_ENVIRONMENT}
+is invalid when the @option{parallel-tests} option is active; however,
+the @option{parallel-tests} driver provides a more elegant way to
+achieve the same effect, with the further benefit of freeing the
+@code{TESTS_ENVIRONMENT} variable for the user
+(@pxref{Simple Tests using parallel-tests}).
@cindex Tests, expected failure
@@ -8739,7 +8758,9 @@ Simple Tests using parallel-tests
the @code{check_*} variables are honored, and the environment variable
@env{srcdir} is set during test execution. Also, @code{TESTS_ENVIRONMENT}
is still honored, but is complemented by a new developer-reserved variable
-@code{AM_TESTS_ENVIRONMENT} (described below).
+@code{AM_TESTS_ENVIRONMENT} (described below), and its allowed uses are
+somewhat restricted (other features of the @option{parallel-tests} driver
+compensate for this, though).
This test driver is still experimental and may undergo changes in order
to satisfy additional portability requirements.
@@ -8812,12 +8833,13 @@ Simple Tests using parallel-tests
but should be reserved for the user.
@vindex AM_TESTS_ENVIRONMENT
-The @code{AM_TESTS_ENVIRONMENT} variable can be used to run initialization
-code and set environment variables for the tests' runs. The user can
-still employ the @code{TESTS_ENVIRONMENT} variable to override settings
-from @code{AM_TESTS_ENVIRONMENT}. Note that, for implementation reasons,
-if the @code{AM_TESTS_ENVIRONMENT} variable is set, its contents
-@emph{must} be terminated by a semicolon.
+The @code{AM_TESTS_ENVIRONMENT} and @code{TESTS_ENVIRONMENT} variables can
+be used to run initialization code and set environment variables for the
+test scripts. The former variable is developer-reserved, and can be
+defined in the @file{Makefile.am}, while the latter is reserved for the
How about s/reserved // here, to reflect prior usage?
+user, which can employ it to extend or override the settings in the
+former. For implementation reasons, if the @code{AM_TESTS_ENVIRONMENT}
s/\. / /
+variable is set, its contents @emph{must} be terminated by a semicolon.
Why is that necessarily so?
@@ -8834,6 +8856,23 @@ Simple Tests using parallel-tests
AM_TESTS_ENVIRONMENT = exec 92; warn_fileno=9; export warn_fileno;
@end example
+It's important to note that, differently from what we've seen for the
+serial testsuite driver (@pxref{Simple Tests using parallel-tests}), the
+@code{AM_TESTS_ENVIRONMENT} and @code{TESTS_ENVIRONMENT} variables
+@emph{cannot} be use to define a custom test driver; the
+@code{LOG_COMPILER} and @code{LOG_FLAGS} (or their extension-specific
+counterparts) should be used instead:
+
+@example
+## This is WRONG!
+AM_TESTS_ENVIRONMENT =
Re: [PATCH] docs: avoid a footnote, some related rewordings and improvements (was: Re: [Automake-commit] [SCM] GNU Automake branch, maint, updated. v1.11-393-gc1040a7)
* Stefano Lattarini wrote on Thu, Jun 23, 2011 at 11:33:41PM CEST:
On Thursday 23 June 2011, Ralf Wildenhues wrote:
Footnotes are hard to read and distract the flow, esp. in an info file.
Sorry, I never use info(1) directly (never bothered to learn it actually);
Actually, I had been reluctant to use it for a long time, basically
until I learned --index and C-s (interactive search), which is a lot
slower in PDF and less trivial to do with HTML.
and footnotes are not so bad in HTML and PDF.
True.
Please avoid them whenever possible (consider that they cost you 10
bucks, or that you may only introduce 3 per year or so). Here, you can
easily reword things.
OK. What about the attached patch?
Cool, thanks!
Cheers,
Ralf
Subject: [PATCH] docs: avoid a footnote, some related rewordings and
improvements
* doc/automake.texi (Dist): Reword the part about automatically
distributed files to avoid a footnote. Since we are at it, extend
a bit, and add an example and a reference to a relevant test case.
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -8292,16 +8292,20 @@ is run. The default setting is @option{--best}.
@cmindex include
For the most part, the files to distribute are automatically found by
Automake: all source files are automatically included in a distribution,
-as are all @file{Makefile.am}s and @file{Makefile.in}s. Automake also
+as are all @file{Makefile.am} and @file{Makefile.in} files. Automake also
has a built-in list of commonly used files that are automatically
included if they are found in the current directory (either physically,
-or as the target of a @file{Makefile.am} rule). This list is printed by
-@samp{automake --help}@footnote{Note that some of these files are actually
-distributed only when other certain conditions hold}. Also, files that
-are read by @command{configure}
+or as the target of a @file{Makefile.am} rule); this list is printed by
+@samp{automake --help}. Note that some files in this list are actually
+distributed only if other certain conditions hold (for example,
+@c The following example should be covered by the test case
+@c 'autodist-config-headers.test'.
@c The following example is covered by autodist-config-headers.test.
?
Maybe even some kind of formatted comment, to allow some checking in the
future?
+the @file{config.h.top} and @file{config.h.bot} files are automatically
+distributed only if, e.g., @samp{AC_CONFIG_HEADERS([config.h])} is used
+in @file{configure.ac}). Also, files that are read by @command{configure}
(i.e.@: the source files corresponding to the files specified in various
Autoconf macros such as @code{AC_CONFIG_FILES} and siblings) are
-automatically distributed. Files included in @file{Makefile.am}s (using
+automatically distributed. Files included in a @file{Makefile.am} (using
@code{include}) or in @file{configure.ac} (using @code{m4_include}), and
helper scripts installed with @samp{automake --add-missing} are also
distributed.
Re: [FYI] {parallel-tests-maint} docs: parallel-tests is not experimental anymore
* Stefano Lattarini wrote on Fri, Jun 24, 2011 at 09:32:53AM CEST: Subject: [PATCH] Revert docs: parallel-tests is not experimental anymore This reverts commit a9eef973b5ea47cc3495f1a8307d4f7b85aea46f. It turned out that the current work to introduce TAP and SubUnit support in Automake-generated testsuite harnesses will probably require the introduction of slight incompatibilities in the 'parallel-tests' behaviour, starting from release 1.12 onward. So it's advisable to continue to characterize the 'parallel-tests' support as experimental in maintenance release 1.11.2. OK. Although you don't really need all the explanation for the patch. Thanks, Ralf
Re: [PATCH] docs: avoid a footnote, some related rewordings and improvements (was: Re: [Automake-commit] [SCM] GNU Automake branch, maint, updated. v1.11-393-gc1040a7)
* Stefano Lattarini wrote on Mon, Jun 27, 2011 at 03:35:23PM CEST: On Monday 27 June 2011, Ralf Wildenhues wrote: +@c The following example should be covered by the test case +@c 'autodist-config-headers.test'. @c The following example is covered by autodist-config-headers.test. Maybe even some kind of formatted comment, to allow some checking in the future? A nice idea. Suggestions? autoconf.texi uses stuff like: @c If you change this example, adjust tests/semantics.at:AC_CHECK_SIZEOF struct. Oh, and I'd keep this for a follow-up, bacause there are already similar testcase-referring comments in the manual, which will have to be adjusted. Sure. Well, do they use some format already? Thanks, Ralf
Re: [FYI] New public temporary branch, about documentation for custom test drivers
* Stefano Lattarini wrote on Mon, Jun 27, 2011 at 01:11:51PM CEST:
On Monday 27 June 2011, Ralf Wildenhues wrote:
* Stefano Lattarini wrote on Sun, Jun 26, 2011 at 02:20:48PM CEST:
+The @code{AM_TESTS_ENVIRONMENT} and @code{TESTS_ENVIRONMENT} variables
can
+be used to run initialization code and set environment variables for the
+test scripts. The former variable is developer-reserved, and can be
+defined in the @file{Makefile.am}, while the latter is reserved for the
How about s/reserved // here, to reflect prior usage?
Well, I do think we should now discourage such prior usage...
Yeah, we should. It's very wide-spread however. And when you think in
terms of a developer eager to provide code portable to several Automake
versions, then changing that is very far down the road. gnulib is in
this position, Libtool is, and several projects that don't provide
macros for other projects or Makefile.am fragments also are eager to
work with a range of Automake releases (often starting from 1.9.6 on).
+variable is set, its contents @emph{must} be terminated by a semicolon.
Why is that necessarily so?
Otherwise the user couldn't safely use TESTS_ENVIRONMENT for overriding
AM_TESTS_ENVIRONMENT; for example, with foo=0 foo=1 COMMAND ..., some
shells (e.g., bash and dash) will export foo to `0' in the environment
of COMMAND, while other shells (e.g., Solaris /bin/sh) will export foo
to `1'. And BTW, it was you that pointed this out to me originally ;-)
See, I simply didn't think of TESTS_ENVIRONMENT. So, the documentation
should make it clear that variables actually need to be exported (not
just set); it can't hurt to say why the semicolon is useful: so that
the user can set TESTS_ENVIRONMENT to override variables further.
+to analyze their execution and outcome, to create the @file{.log} files
+associated to these test runs, and to display the test results on the
+console.
It is the sole responsibility of the author of the test driver
+to ensure that it implements all the above steps meaningfully and
+correctly; Automake isn't and can't be of any help here.
I'm wondering whether it's better to omit this sentence. It sounds like
a teacher talking to me; in programming, I am always expected to do
things correctly.
Yes, but often your tools offer diagnostic that helps to prevent some, or
many, mistakes. In case the tools can't or won't do that (like here),
extra warnings and be careful exhortations in the documentation are
IMHO warranted.
OK. How about s/sole // then? We might actually want to (or be able
to) provide help at some point in the future.
+@itemize
+@item
+concurrency through the use of @command{make}'s option @option{-j};
+@item
+per-test @file{.log} files, and generation of a summary @file{.log}
+file from them;
+@item
+@code{recheck} target and lazy reruns of tests;
+@item
+inter-test dependencies;
+@item
+support @code{check_*} variables (@code{check_PROGRAMS},
+@code{check_LIBRARIES}, ...);
+@item
+use of @code{VERBOSE} environment variable to get verbose output on
+testsuite failures;
+@item
+definition and honoring of @code{TESTS_ENVIRONMENT} and
+@code{AM_TESTS_ENVIRONMENT} variables;
+@item
+definition of generic and extension-specific @code{LOG_COMPILER} and
+@code{LOG_FLAGS} variables.
+@end itemize
+
@noindent?
You mean before the following lines? (I guess you do)
Yes.
+Custom testsuite drivers are declared by defining the make variables
+@code{LOG_DRIVER} or @code{@var{ext}_LOG_DRIVER} (where @var{ext} must
+be declared in @code{TEST_EXTENSIONS}). They must be defined to
+programs or scripts that will be used to drive the execution, logging,
+and outcome report of the tests with corresponding extensions (or of
+those with no registered extension in the case of @code{LOG_DRIVER}).
+Clearly, multiple distinct test drivers can be declared in the same
+@file{Makefile.am}. Note moreover that the @code{LOG_DRIVER} variables
+are @emph{not} a substitute for the @code{LOG_COMPILER} variables: the
+two sets of variables can, and often do, usefully and legitimately
+coexist.
+
+@c TODO: We should really be able to point to a clarifying example here!
Yep. As a reader, this leaves me confused. Why should there be two
separate script instances?
Actually, why should there (other than maybe for backward
compatibility)? I mean, not just in the documentation, but in the
Automake code as well? Can't we simplify this? Thanks.
Oh no! They are really meant to fullfill two orthogonal and complementary
needs. Let's see if an example can help to clarify the situation.
Assume you are testing a static code checker for C; in this case, you
might to write most of your test cases as C files with expected diagnostic
embedded in special comments, as in e.g
Re: [FYI] {parallel-tests-maint} docs: parallel-tests is not experimental anymore
* Stefano Lattarini wrote on Mon, Jun 27, 2011 at 03:48:58PM CEST: On Monday 27 June 2011, Ralf Wildenhues wrote: * Stefano Lattarini wrote on Fri, Jun 24, 2011 at 09:32:53AM CEST: Subject: [PATCH] Revert docs: parallel-tests is not experimental anymore This reverts commit a9eef973b5ea47cc3495f1a8307d4f7b85aea46f. It turned out that the current work to introduce TAP and SubUnit support in Automake-generated testsuite harnesses will probably require the introduction of slight incompatibilities in the 'parallel-tests' behaviour, starting from release 1.12 onward. So it's advisable to continue to characterize the 'parallel-tests' support as experimental in maintenance release 1.11.2. OK. Although you don't really need all the explanation for the patch. Pushed. I now think that the parallel-tests-maint should be merged to maint. Objections? OK. Thanks, Ralf
Re: [PATCH] {test-protocols} parallel-tests: fix bug in Automake's own testsuite
* Stefano Lattarini wrote on Thu, Jun 23, 2011 at 09:52:55AM CEST:
On Wednesday 22 June 2011, Ralf Wildenhues wrote:
Also, have you thought about a quoting mechanism so that test authors do
not have to think about breaking the test driver mechanism accidentally?
That would seem more robust to me.
I don't understand. A quoting mechanism for own our testsuite is already
offered by the new `dump_log_files' function (and the new maintainer
checks try to unsure that this function is used when required). OTOH,
the authors of third-party test drivers *must* allow the direct writing
of `:test-result:' directives in the driver output -- this is the only
way to declare test results that are understood and accounted for by the
new Automake test harness.
So, we are doing inband signaling.
But Automake's own testsuite is not a TAP or a subunit one. Thus it
does not need to do inband-signaling. Right?
The reason I'm harping on this is that, if all existing users of
Automake test drivers are updated to use the same kind of inband
signaling, then they all are liable to similar bad signals. That
is a potential source for regression.
What do TAP and subunit do to protect against this? Quote normal
log output? Like email, prepend ' ' to lines? Please explain.
Also, most of your uses of useless are actually superfluous: either
you go on to explain why something is not actually useless after all
(in which case there seems no point in calling it useless in the first
place), or it is useless, in which case why have it?
What about extra quotes instead? I'll use that if you don't object.
Well, just explain the reason /for/ something. For example:
+# FIXME: the useless quoting below is to please maintainer-check.
+env VERBOSE'='yes $MAKE -e check stdout { cat stdout; Exit 1; }
You can just write:
# Quote for maintainer-check (FIXME).
(I'd even leave out the FIXME, but I can see why you'd like that.)
It has all information your comment gave.
No time for more detailed review in the short span allowed for feedback.
Why BTW?
Because without this patch I'm experiencing spurious failures in the
testsuite. OK, I can visually scan the output of make check and
see that there is no unexpected foo.test: FAIL line in there, and
thus conclude that the failure is really only spurious. But this is
cumbersome to say the least, and surely error-prone.
You can always git rebase later, and it's not like this is
security relevant
This is true.
or needs to be rolled out this week.
I disagree: Ggven what I said above, I'd rather apply the patch today
(I'll wait until this evening or tomorrow morning though, to give you
enough time to answer).
None of your reasons make sense to me. Of course, you should fix this
in your tree in order to not see spurious errors any more. But there is
absolutely no reason to rush with a push, or not to go back and update
the patch after I review it in three days. Please give a reason against
*that*.
Thanks,
Ralf
Re: [FYI] {maint} docs: minor cosmetic fixes
* Stefano Lattarini wrote on Thu, Jun 23, 2011 at 06:40:25PM CEST: * doc/automake.texi: Break few overly long lines, throughout the file. (Simple Tests): Move @vindex for XFAIL_TESTS to the correct position, i.e., before and not after the paragraph where it is introduced. (Options @item ansi2knr): Don't user @xref where @pxref should use be used instead. This fixes a texinfo warning. (Other things Automake recognizes @item AM_C_PROTOTYPES): Don't @ref where @pxref should be used instead. s/@ref/use / I prefer active voice btw, e.g., Use @pxref instead of @ref. Thanks, Ralf
Re: [FYI] {master} docs: fix unportable example of AM_TESTS_ENVIRONMENT usage
* Stefano Lattarini wrote on Thu, Jun 23, 2011 at 07:09:16PM CEST:
* doc/automake.texi (Simple Tests using parallel-tests): The
old example on AM_TESTS_ENVIRONMENT relied on unportable shell
features, and in particular didn't work with various Korn
Shells (see also commit `v1.11-925-g29ca903'). Give another
example, simpler this time, but still inspired to real-world
usage (the GNU coreutils testsuite).
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -8742,21 +8742,23 @@ code and set environment variables for the tests'
runs. The user can
still employ the @code{TESTS_ENVIRONMENT} variable to override settings
from @code{AM_TESTS_ENVIRONMENT}. Note that, for implementation reasons,
if the @code{AM_TESTS_ENVIRONMENT} variable is set, its contents
-@emph{must} be terminated by a semicolon.
-
-@example
-# The tests below are expected to use the file descriptor passed in
-# the environment variable 'warn_fileno' to print warnings (e.g.,
-# about skipped and failed tests). If this variable were to be set
-# to `2' (i.e. default stderr), the warnings would be redirected by
-# the automake parallel-tests driver into the .log files. But the
-# AM_TESTS_ENVIRONMENT definition below will cause the reasons for
-# skip/failure to be printed to the console instead. The user
-# can still override this by setting TESTS_ENVIRONMENT to e.g.
-# `warn_fileno=2' at make runtime, which will cause the warnings
-# to be sent to the .log files again.
-TESTS = test1.sh test2.sh ...
-AM_TESTS_ENVIRONMENT = exec 92; warn_fileno=9; export warn_fileno;
This example served two purposes: use of AM_TESTS_ENVIRONMENT, and
showing how to produce output while the test is running. Your patch
removes the second feature. Can we have it back, ideally in a portable
fashion?
+@emph{must} be terminated by a semicolon. Here is an example of a
+slightly elaborate definition:
+
+@example
+# Some environment initializations are kept in a separate shell file
+# `tests-env.sh', which can be helpful to run tests from the command line.
+# We don't want to depend on the `srcdir' value exported by the Automake
+# testsuite harness; we define it statically in the `tests-env' file.
This sentence does not make sense to me, in this particular context.
As a user, it would confuse me, at least without further explanation.
+# On Solaris, prefer more POSIX-compliant versions of the standard tools
+# by default.
+AM_TESTS_ENVIRONMENT = \
+ unset srcdir; \
+ . $(srcdir)/tests-env.sh; \
+ if test -d /usr/xpg4/bin; then \
+PATH=/usr/xpg4/bin:$$PATH; export PATH; \
+ fi;
+@c $$ restore font-lock
@end example
@trindex mostlyclean
Thanks,
Ralf
Re: [Automake-commit] [SCM] GNU Automake branch, maint, updated. v1.11-393-gc1040a7
[ from automake-commit ]
since I didn't review the source patch yet:
* Stefano Lattarini wrote on Thu, Jun 23, 2011 at 11:03:52AM CEST:
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -8293,7 +8293,9 @@ as are all @file{Makefile.am}s and @file{Makefile.in}s.
Automake also
has a built-in list of commonly used files that are automatically
included if they are found in the current directory (either physically,
or as the target of a @file{Makefile.am} rule). This list is printed by
-@samp{automake --help}. Also, files that are read by @command{configure}
+@samp{automake --help}@footnote{Note that some of these files are actually
Footnotes are hard to read and distract the flow, esp. in an info file.
Please avoid them whenever possible (consider that they cost you 10
bucks, or that you may only introduce 3 per year or so). Here, you can
easily reword things.
+distributed only when other certain conditions hold}. Also, files that
+are read by @command{configure}
(i.e.@: the source files corresponding to the files specified in various
Autoconf macros such as @code{AC_CONFIG_FILES} and siblings) are
automatically distributed. Files included in @file{Makefile.am}s (using
Thanks,
Ralf
Re: [PATCH] {test-protocols} parallel-tests: fix bug in Automake's own testsuite
* Stefano Lattarini wrote on Wed, Jun 22, 2011 at 09:52:13PM CEST:
The first bugfix in the new branch. Luckily, the bug (detailedly
described in the ChangeLog entry) is only in the Automake's own
testsuite, not in the implementation.
You could halven the length of the log entry without loss of
information.
Also, have you thought about a quoting mechanism so that test authors do
not have to think about breaking the test driver mechanism accidentally?
That would seem more robust to me.
Also, most of your uses of useless are actually superfluous: either
you go on to explain why something is not actually useless after all
(in which case there seems no point in calling it useless in the first
place), or it is useless, in which case why have it?
No time for more detailed review in the short span allowed for feedback.
Why BTW? You can always git rebase later, and it's not like this is
security relevant or needs to be rolled out this week.
Thanks,
Ralf
Subject: [PATCH] parallel-tests: fix bug in Automake's own testsuite
The new code for parsing the testsuite-generated `.log' files,
as introduced in commit `v1.11-872-gc96b881', considers each
`:test-result:' field anywhere in a `.log' file as a declaration
of a test result, and accounts for it as such in the testsuite
summary. Unfortunately, some test scripts in the Automake's
own testsuite, which were checking the just-described new parsing
semantics, ended up putting spurious `:test-result:' fields in
their log files. These spurious fields did not refer to actual
results of some Automake test, but were instead emitted by dummy
testcases being run from within the Automake test scripts that
were verifying the new code for parsing the testsuite-generated
`.log' files.
This situation was causing a botched summary and a wrong exit
status for the Automake testsuite; for example:
$ make check TESTS='check12-p.test'; echo exit: $?
...
PASS: check12-p.test
=
4 of 5 tests failed
See tests/test-suite.log
Please report to bug-autom...@gnu.org
=
make[2]: *** [test-suite.log] Error 1
make: *** [check-am] Error 2
exit: 2
This changes should fix the problem.
* tests/defs (dump_log_files): New subroutine, display `.log'
files created by the Automake test driver(s), mangling them to
prevent the Automake's own testsuite harness from picking up
spurious test results from them.
* tests/check12.test: Use it when displaying the content of `.log'
files created here by $MAKE check and similar, to avoid confusing
Automake's own testsuite harness with spurious test results. Other
related changes and simplifications.
* tests/check-exported-srcdir.test: Likewise.
* tests/parallel-tests-am_tests_environment.test: Likewise.
* tests/parallel-tests-empty-testlogs.test: Likewise.
* tests/parallel-tests-interrupt.test: Likewise.
* tests/parallel-tests-reset-term.test: Likewise.
* tests/tests-environment-and-log-compiler.test: Likewise.
* tests/tests-environment-fd-redirect.test: Likewise.
* tests/test-driver-custom.test: Likewise.
* tests/test-driver-custom-no-html.test: Likewise.
* tests/test-driver-custom-xfail-tests.test: Likewise.
* tests/test-driver-custom-multitest.test: Likewise.
* tests/test-driver-custom-multitest-recheck.test: Likewise.
* Makefile.am (sc_tests_cat_log_file): New maintainer check.
(sc_tests_make_check_verbose): Likewise.
(syntax_check_rules): Add them.
* tests/parallel-tests.test: Adjust to avoid triggering the new
`sc_tests_make_check_verbose' maintainer check.
--- a/tests/parallel-tests.test
+++ b/tests/parallel-tests.test
@@ -138,8 +138,13 @@ grep baz.test stdout Exit 1
grep '2.*tests.*failed' stdout
# Test VERBOSE.
-env VERBOSE=yes $MAKE -e check stdout { cat stdout; Exit 1; }
-cat stdout
+# FIXME: the useless quoting below is to please maintainer-check.
+env VERBOSE'='yes $MAKE -e check stdout { cat stdout; Exit 1; }
+# With VERBOSE set to yes, the standard output from make is expected
+# to contain the content of test-suite.log, so we need the following
+# in order not to confuse the Automake testsuite summary with spurious
+# results (as would be the case with a simpler cat stdout).
+dump_log_files stdout
grep 'this is.*bar.test' stdout
grep 'this is.*baz.test' stdout
Re: [FYI] {maint} maintcheck: avoid few spurious failures
* Stefano Lattarini wrote on Mon, Jun 20, 2011 at 11:59:57PM CEST: On Monday 20 June 2011, Ralf Wildenhues wrote: * Stefano Lattarini wrote on Mon, Jun 20, 2011 at 05:05:45PM CEST: sc_tests_plain_automake: - @if grep -v '^#' $(srcdir)/tests/*.test | grep -E ':[ ]*automake([^:]|$$)'; then \ + @if grep -v '^#' $(srcdir)/tests/*.test | grep -E ':[ ]*automake\([^:]|$$)'; then \ The RE that was there before was there specifically to emulate the nonportable '\' construct. Now, I'm not sure I should fight for using Posix compatible regular expressions in maintainer-check rules (seems I lost that battle earlier already), Well, notice that I've just followed the existing practice in using GNU grep extensions in the maintcheck rules; Yes; notice that I had noticed that. but if you require GNU grep, please be consistent and remove the now-unneeded stuff afterwards and the -E. OK, I will push the attached patch if that's OK with you. Ugh; 1/2 seems to be going into the wrong direction of changing test code to work around suboptimal maintainer checks, so sorry for leading you on a wrong path there again. 2/2 seems fine, thanks. And since I was at it, I noticed that the `sc_tests_plain_*' checks were incomplete, as they didn't look for autoreconf, autoheader and autom4te too. What fixing this with the second attached patch? Cheers, Ralf
Re: [PATCH v4 3/3] parallel-tests: allow each test to have multiple results
* Stefano Lattarini wrote on Mon, Jun 20, 2011 at 10:58:05PM CEST: On Monday 20 June 2011, Ralf Wildenhues wrote: For example the parallel BSD makes tend to reuse shells for running the recipe commands; But only for the commands in the same recipe, right? I think not. I'm not so sure they like if their shells go away with exec. In this case, not a problem I think, since the invocation of the testsuite driver is the last command in the recipe. Well, they should cope with it anyway, and use a new shell if the old one is gone. Cheers, Ralf
Re: [PATCH 1/2] tests: make test runner a script, not a shell function
* Stefano Lattarini wrote on Mon, Jun 20, 2011 at 11:12:23PM CEST: Maybe we should also say that using TESTS_ENVIRONMENT to define a custom test runner is now not only strongly deprecated (as it already was I hope), No it wasn't. test runner is not a term I would recognize, btw. but also unsupported and not working anymore? Also, should I look for TESTS_ENVIRONMENT usages in google code search? I was really hoping to spae myself the pain... ;-) Not sure. If anything, you can use regexes to avoid stuff you're not interested in. and if not, is it possible to have a compatible one (without too much maintenance effort duplication)? No need to go the effort right away. Well, we could add a new option old-parallel-tests or something like that, that causes the old code in 'check.am' (with few tweaks in order to support the new parsing of `.log' files) to be used instead of the 'test-driver' script. By refactoring some code in handle_tests(), we could ensure not to add any real complexity to the automake script (w.r.t. to my patches at least); but the duplication between 'check.am' and 'test-driver' will unavoidable IMHO. Well, if we need to use a name, then the name should be for the new one. That way you can have full backward compatibility. I hope that we do not have to maintain four or more such drivers. Me too. And I also think we should start deprecating the old serial driver ASAP (i.e., in 1.12, as I think 1.11.2 is sadly tooearly); but this is for another thread, and another time. Erm, there are quite a few packages out there that can only use the serial driver. They require tests run in a specific order, or some tests run twice, or output not redirected. I fixed Libtool a while ago, but the changed code is still quite buggy. I don't think we can expect everyone to migrate. Cheers, Ralf
Re: [FYI] {maint} maintcheck: avoid few spurious failures
* Stefano Lattarini wrote on Tue, Jun 21, 2011 at 09:53:19AM CEST: On Tuesday 21 June 2011, Ralf Wildenhues wrote: * Stefano Lattarini wrote on Mon, Jun 20, 2011 at 11:59:57PM CEST: On Monday 20 June 2011, Ralf Wildenhues wrote: * Stefano Lattarini wrote on Mon, Jun 20, 2011 at 05:05:45PM CEST: sc_tests_plain_automake: - @if grep -v '^#' $(srcdir)/tests/*.test | grep -E ':[ ]*automake([^:]|$$)'; then \ + @if grep -v '^#' $(srcdir)/tests/*.test | grep -E ':[ ]*automake\([^:]|$$)'; then \ The RE that was there before was there specifically to emulate the nonportable '\' construct. Now, I'm not sure I should fight for using Posix compatible regular expressions in maintainer-check rules (seems I lost that battle earlier already), Well, notice that I've just followed the existing practice in using GNU grep extensions in the maintcheck rules; Yes; notice that I had noticed that. OK :-) Sorry for being a language lawyer there again. ;-) but if you require GNU grep, please be consistent and remove the now-unneeded stuff afterwards and the -E. OK, I will push the attached patch if that's OK with you. Ugh; 1/2 seems to be going into the wrong direction of changing test code to work around suboptimal maintainer checks, so sorry for leading you on a wrong path there again. 2/2 seems fine, thanks. So should I drop the first patch and keep only the second one? Yes, please. Thanks! Ralf
Re: [PATCHES v5] Allow custom testsuite drivers in Automake
* Stefano Lattarini wrote on Tue, Jun 21, 2011 at 09:22:42PM CEST: Hopefully the last iteration of the patches. A follow-up for extending and fixing the documentation will go in a new thread. Can you push your branch for this to a branch in the savannah git? I'd like to take another look at the patches in detail, but why not fix all remaining issues in later patches. Please try not to merge from other branches into this one too much (at the moment), if you don't have to. Thanks. It'd be good to start getting to the meat, i.e., some kind of TAP/subunit support. (I suppose you're working on this already.) Thanks, Ralf
Re: [PATCH] {maint} remake: behave better with non-GNU make in subdirectories
* Stefano Lattarini wrote on Sun, May 29, 2011 at 04:26:36PM CEST: --- a/lib/am/configure.am +++ b/lib/am/configure.am @@ -22,7 +22,7 @@ ## %MAKEFILE% is updated before considering the am--refresh target. The comment up here ^^^ needs to be updated in this particular patch. if %?TOPDIR_P% .PHONY: am--refresh -am--refresh: +am--refresh: %MAKEFILE% @: endif %?TOPDIR_P% * Stefano Lattarini wrote on Tue, May 31, 2011 at 11:28:50AM CEST: On Tuesday 31 May 2011, Peter Rosin wrote: My attempt follows: remake: behave better with non-GNU make in subdirectories. With a decent make program, it is possible to rebuild out-of-date autotools-generated files with a simple make Makefile. Remove the limitation that make Makefile has to be issued from the top-level directory with non-GNU make implementations. Thanks; this helped me to come up with this other entry, which while being unfortunately more complex, is also more precise: remake: behave better with non-GNU make in subdirectories. Remove the limitation that, with non-GNU make implementations, make Makefile has to be issued from the top-level directory in order to rebuild autotools-generated files due to an updated configure.ac (or to an updated prerequisite thereof). This is what I'll use if there are no further objections. I have an objection: you guys manage to discuss a log entry for half a dozen mails, yet never address the most interesting question the log entries throw up: what is that 'silly' limitation that non-GNU makes have here? Also, you violate the put the explanation in the code, not in the log entry principle, actually falsifying an existing comment in the code. Cheers, Ralf
Re: [PATCH] {maint} remake: behave better with non-GNU make in subdirectories
What's more: have you tried this patch on a nontrivial source tree (where regenerating takes more than a second or so) with a few non-GNU makes and GNU make? I kinda fear that it can cause an endless regen loop. It might actually be smarter to use some newer BSD make features to mark Makefile as prerequisite to rebuild, and ignore other makes. But well, this thread doesn't yet contain analysis about the bug in question ... Thanks, Ralf
Re: [PATCH] {maint} remake: behave better with non-GNU make in subdirectories
* Stefano Lattarini wrote on Tue, Jun 21, 2011 at 10:43:06PM CEST: On Tuesday 21 June 2011, Ralf Wildenhues wrote: * Stefano Lattarini wrote on Sun, May 29, 2011 at 04:26:36PM CEST: --- a/lib/am/configure.am +++ b/lib/am/configure.am @@ -22,7 +22,7 @@ ## %MAKEFILE% is updated before considering the am--refresh target. The comment up here ^^^ needs to be updated in this particular patch. OK will fix in a follow-up patch (probably tomorrow). The above comment holds only for GNU make BTW (and correctly states so), so I think I'll just remove it. No. I have an objection: you guys manage to discuss a log entry for half a dozen mails, yet never address the most interesting question the log entries throw up: what is that 'silly' limitation that non-GNU makes have here? It's not a silly limitation of those make implementations at all -- it is was silly limitation in automake! Automake-generated code was relying on GNU make semantics even when POSIX semantics would have worked just as well. This is not true, IIRC. And IMHO ChangeLog entry describes this silly limitation in detail (albeit probably with suboptimal wording): ``... the limitation [was] that, with non-GNU make implementations, make Makefile has to be issued from the top-level directory in order to rebuild autotools-generated files *due to an updated configure.ac* ...'' This just describes a symptom, not the underlying issue. I am pretty sure if the issue wasn't deeper, and the fix you issued not problematic for other use cases, then Alexandre would have implemented that years ago. I don't approve of this change without a good analysis of why this does not introduce regressions. (No need to do it right away.) Cheers, Ralf
Re: [PATCH v4 3/3] parallel-tests: allow each test to have multiple results
* Stefano Lattarini wrote on Fri, Jun 17, 2011 at 09:33:13AM CEST:
On Friday 17 June 2011, Ralf Wildenhues wrote:
Actually, why not s/testcase/test/g globally in all your text.
Because I'm trying to make a distinction between test scripts and
test cases. With the new interface, a single test script can contain
multiple testcases, and thus have multiple test results. It is a
simple concept, but not immediate to get for someone who was used
the older Automake tests drivers (which obeyed the principle one
test script, one test result, so that you could call a test script
also a testcase, or simply a test, without risk of confusion.
As I said, use the name other testsuite environments use.
:test-result: seems better.
(EXTRA_DIST): Distribute `ostp-driver'.
(test-driver-custom-multitest.log): Depend on `ostp-driver'.
(test-driver-custom-multitest-recheck.log): Likewise.
(test-driver-custom-multitest-recheck2.log): Likewise.
(test-driver-custom-html.log): Likewise.
* doc/automake.texi (API for Custom Test Drivers): Update (still
in Texinfo comments only).
I don't like TODO and comments stuff in finished patches documentation
if we can help it (and here, we surely can).
While I mostly agree, I'd like to point out that I had already rewritten
and revesited the documentation several times, and after some point you
risk more harm than good in tweaking and re-shaping the same text again;
also, you're somewhat fed up with it, so that the risk of becoming sloppy
or of producing confused text increases. I'll fix the TODOs in this patch
if you insist, but honestly I'd rather to that in a follow-up patch if
possible.
Why not just split the whole documentation change into a followup patch
then?
@b{TODO!} @i{Options and flags that the driver must handle. Generation
-of ``.log'' files. Console output the driver is expected to produce.
-Support for colored output, XFAIL_TESTS, and DISABLE_HARD_ERRORS}
+of ``.log'' files, and format they must obey. Console output the driver
@file{.log}
You should almost never need to use ``...'' in texinfo, at least not for
any entity that has a meaning in code.
Note that this hunk is just temporary, and bounded to be removed by follow-up
patches that will (or would have) completed the TODOs.
See above.
+@cexecutes two test cases, one successful and one failing, and skip
+@canother test case, the driver should end up writing the following
+@cin the test log:
+@c :am-testcase-result: PASS [passed testcase name or details]
+@c :am-testcase-result: FAIL [failed testcase name or details]
+@c :am-testcase-result: SKIP [skipped testcase name or details]
is the [...] literal or metasyntactic?
Metasyntactic.
Please use proper notation here.
What would that be? Sorry if it's a stupid question, but my grasp of
Texinfo is still pretty limited.
@example would seem appropriate.
+@cThe above lines (each of which *must* be followed by a blank line
+@cin order for the HTML output generation to work) are used only
+@cwhen generating the `test-suite.log' from the individual test
See above.
About what exactly?
@file{test-suite.log}
+ results2=`sed -n s/^$$rst_magic[ ]*//p $$list2`; \
tab space
That's deliberate, we don't want to be too strict in parsing user's
output (or in this case, third-party drivers output).
Sorry. I just meant the tab should come before the space, not after.
My editor highlights the space otherwise (which, granted, is a
limitation in the highlighting rule, but also helps prevent other
editors, human or programmatic, to mangle the code in an unwanted way).
+$@ 21 | tee $tmp_out | (
+ i=0 st=0
+ : $tmp_res
+ while read line; do
+case $line in
+ PASS:*|FAIL:*|XPASS:*|XFAIL:*|SKIP:*)
+i=`expr $i + 1`
+result=`LC_ALL=C expr $line : '\([A-Z]*\):.*'`
This will be quite fork expensive, if done in real-world code.
But this is in a script used only for testing. I don't think
it's worth optimizing it.
No, it's not, but your real scripts won't look all that different.
Besides, why not do it right the first time?
Might just want to use one single sed script replacing this
while while loop.
But then again, measuring is king.
And BTW, premature optimization is the root of all evil ;-)
True. If we'd do it right the first time then there would be no need
to fix efficiency regressions afterwards.
I don't actually care much about micro optimizations like the above at
this point, but I do care when the whole set of code changes will
introduce a factor of 2 slowdown in the test suite overhead. It looks
like it may eventually, judging from your measurements done, and that's
what I am trying to prevent. On w32, that would cause real pain.
I agree that we should improve my code for speed later (after all, we
expect
Re: [PATCH 1/2] tests: make test runner a script, not a shell function
* Stefano Lattarini wrote on Mon, Jun 20, 2011 at 10:22:28AM CEST: [Adding bug-grep, dropping bug-coreutils and automake-patches] re-adding the latter. I've noticed that grep uses a definition of TESTS_ENVIRONMENT very similar to that of coreutils (the one we've just fixed), so it will break with oncoming automake too. That may be a sign that you may not want to actually break this code with your proposed changes to Automake. Put another way: it's a good idea to estimate the level of breakage you're going to burden upon others (a couple of projects, dozens, hundreds), the amount of work needed on their side to fix it, and the amount of work (or possibility) it would take to change your code so they are not broken in the first place. Also, of course, NEWS entries (and probably automake.texi entries) for such changes are a good idea. One thing I've regularly done with new code that is not 100% backward compatible is have a new Automake option for them. That is exactly why there is a 'parallel-tests': it is not fully compatible with the simple test driver, and requires work on behalf of developers using Automake. Unfortunately, I don't have an FSF assignment for grep, so someone else should fix that issue :-( You can do minor fixes without assignments. (Of course, I don't maintain grep, so it's up to them.) Cheers, Ralf
Re: [PATCH] check: don't use multi-line coloring for the report
* Stefano Lattarini wrote on Mon, Jun 20, 2011 at 10:33:53AM CEST:
OK, I've amended the patch on Bert's behalf, as he can't do that himself
at the moment. Attached is what I've pushed (to maint).
Thanks for handling this, and to Bert for the report and patch!
Minor nit:
--- a/lib/am/check.am
+++ b/lib/am/check.am
@@ -75,15 +75,21 @@ am__rst_title = sed 's/.*/
/;h;s/./=/g;p;x;p;g;p;s/.*//'
am__rst_section = sed 'p;s/./=/g;p;g'
# Put stdin (possibly several lines separated by . ) in a box.
-am__text_box = $(AWK) '{ \
- n = split($$0, lines, \\. ); max = 0; \
- for (i = 1; i = n; ++i) \
-if (max length(lines[i])) \
- max = length(lines[i]);\
- for (i = 0; i max; ++i) line = line =; \
- print line;\
- for (i = 1; i = n; ++i) if (lines[i]) print lines[i];\
- print line;\
+# Prefix each line by 'col' and terminate each with 'std', for coloring.
+# Multi line coloring is problematic with less -R, so we really need
+# to color each line individually.
Inconsistent comment level, when compared with below. I'd use '#' here,
as this is not really interesting to anyone reading Makefile.in files.
I acknowledge that they are undercommented for many users, but then again,
getting them up to a comment level easily understandable for everyone
would just blow their size up a real lot, so a better recommendation is
to look at the .am files. And make sure they are commented well, but not
too repetitious.
@@ -398,14 +403,17 @@ check-TESTS: $(TESTS)
fi; \
dashes=`echo $$dashes | sed s/./=/g`; \
if test $$failed -eq 0; then \
- echo $$grn$$dashes; \
+ col=$$grn; \
else \
- echo $$red$$dashes; \
+ col=$$red; \
fi; \
- echo $$banner; \
- test -z $$skipped || echo $$skipped; \
- test -z $$report || echo $$report; \
- echo $$dashes$$std; \
+## Multi line coloring is problematic with less -R, so we really need
+## to color each line individually.
+ echo $${col}$$dashes$${std}; \
+ echo $${col}$$banner$${std}; \
+ test -z $$skipped || echo $${col}$$skipped$${std}; \
+ test -z $$report || echo $${col}$$report$${std}; \
+ echo $${col}$$dashes$${std}; \
test $$failed -eq 0; \
else :; fi
Thanks,
Ralf
Re: [PATCH v4 3/3] parallel-tests: allow each test to have multiple results
Hi Stefano, * Stefano Lattarini wrote on Mon, Jun 20, 2011 at 10:26:06PM CEST: On Monday 20 June 2011, Ralf Wildenhues wrote: Why not just split the whole documentation change into a followup patch then? Because that would only postpone, not avoid, the continous tweaking and amending the documentation; what I'd like instead is to improve it organically and incrementally, in separate patches; i.e., commit a sketchy but correct (even if incomplete) documentation first, and then improve it with follow-ups (maybe handling one concept or one part at the time). Well, then please split the incomplete doc part of your patches into separate patches, so I can say no to them and yes to the other ones easily. ;-) This will be quite fork expensive, if done in real-world code. But this is in a script used only for testing. I don't think it's worth optimizing it. No, it's not, but your real scripts won't look all that different. Besides, why not do it right the first time? I still don't see the point this honestly, but I've thrown in a couple optimizations for bash, zsh and XSI shells (see attached squash-in). Is that enough? Just leave that out. You are right that doing such micro optimizing will not at all be a good strategy, if you do it unorganized and without planning. Leave it for later, but still keep half an eye on the stuff not getting like 4 times slower overhead. I don't actually care much about micro optimizations like the above at this point, but I do care when the whole set of code changes will introduce a factor of 2 slowdown in the test suite overhead. It looks like it may eventually, judging from your measurements done, and that's what I am trying to prevent. On w32, that would cause real pain. But maybe it would be worth trying to instead optimize stuff like $(am__check_pre) and $(am__vpath_adj_setup), where we could trim extra forks in the case of XSI shells or bash. Doesn't sound like it would bring your project forward at this point. I'm sorry I brought this topic up before. I shouldn't have. I.e., optimize an existing and tested implementation instead of holding back a promising design due to *possible* future performance problems. Well, I don't like this attitude. If something will have a performance problem, then maybe it was not all that promising after all. I'm not claiming your approach has, however. All I'm suggesting is that you do keep an eye on it. Also, execing the test driver in check2.am instead of spawning it could avoid an expensive fork. But we should then test at configure time that $SHELL can gracefully handle such execing w.r.t. the use of $(TESTS_ENVIRONMENT); i.e., that an usage like: 92 foo=bar exec sh -c 'echo $foo 9' does the expected thing (hint: it does with dash, bash, zsh, NetBSD /bin/sh and Debian ksh; it doesn't with Solaris /bin/ksh and /bin/sh). Such changes should only be done after demonstrating that they actually cause measureable speedups. And has no semantic changes (which I am not so sure of). For example the parallel BSD makes tend to reuse shells for running the recipe commands; I'm not so sure they like if their shells go away with exec. Cheers, Ralf
Re: [PATCH] check: don't use multi-line coloring for the report
* Ralf Wildenhues wrote on Mon, Jun 20, 2011 at 10:29:02PM CEST: * Stefano Lattarini wrote on Mon, Jun 20, 2011 at 10:33:53AM CEST: +# Prefix each line by 'col' and terminate each with 'std', for coloring. +# Multi line coloring is problematic with less -R, so we really need +# to color each line individually. Inconsistent comment level, when compared with below. I'd use '#' here, D'oh. I meant: I'd use '##' here. Sorry about that. (Also, this is so minor it can be dealt with along other changes later.) as this is not really interesting to anyone reading Makefile.in files. I acknowledge that they are undercommented for many users, but then again, getting them up to a comment level easily understandable for everyone would just blow their size up a real lot, so a better recommendation is to look at the .am files. And make sure they are commented well, but not too repetitious.
Re: [FYI] {maint} maintcheck: avoid few spurious failures
* Stefano Lattarini wrote on Mon, Jun 20, 2011 at 05:05:45PM CEST: * Makefile.am (sc_tests_plain_aclocal, sc_tests_plain_perl, sc_tests_plain_autoconf, sc_tests_plain_automake, sc_tests_plain_autoupate): Be stricter in matching an erroneous literal command, i.e., `aclocal', `automake', `perl', etc. sc_tests_plain_automake: - @if grep -v '^#' $(srcdir)/tests/*.test | grep -E ':[ ]*automake([^:]|$$)'; then \ + @if grep -v '^#' $(srcdir)/tests/*.test | grep -E ':[ ]*automake\([^:]|$$)'; then \ The RE that was there before was there specifically to emulate the nonportable '\' construct. Now, I'm not sure I should fight for using Posix compatible regular expressions in maintainer-check rules (seems I lost that battle earlier already), but if you require GNU grep, please be consistent and remove the now-unneeded stuff afterwards and the -E. sc_tests_plain_aclocal: - @if grep -v '^#' $(srcdir)/tests/*.test | grep ':[ ]*aclocal'; then \ + @if grep -v '^#' $(srcdir)/tests/*.test | grep ':[ ]*aclocal\'; then \ echo 'Do not run aclocal in the above tests. Use $$ACLOCAL instead.' 12; \ exit 1; \ fi Thanks, Ralf
Re: [PATCH v4 1/3] parallel-tests: add auxiliary script 'pt-driver', refactor
* Stefano Lattarini wrote on Thu, Jun 16, 2011 at 10:00:31AM CEST:
This refactoring should cause no API of functionality change,
and is meant only to simplify the future implementation of TAP
and SubUnit testsuite drivers. More precisely, our roadmap is
to move most of the testsuite driving features out of the
Automake-generated Makefiles, and into external scripts with
well-defined interfaces. This will allow the user to define
its own personalized testsuite drivers, and will also offer us
a framework upon which to implement our new TAP and SubUnit
drivers, all in a very unobtrusive way and retaining an high
degree of code reuse and backward-compatibility.
I generally like the direction this is taking. The point of best
separation between which code goes into Makefile.in and which into
the driver scripts can be fine-tuned when we have more than one such
script.
Actually, yes, before deciding on this for real I really do want to see
a nontrivial other driver script. There is no point in hardcoding
too much in several driver scripts if it all needs to be the same
anyway.
Please measure the time overhead your changes introduce into the current
code, for a trivial testsuite (say, 50 tests running 'true'), and a
nontrivial one like Automake's and one with faster tests. Thanks.
I've already mentioned that I don't like the name of the script. ;-)
(You could basically replace 'pt' everywhere else with 'test', too,
as the non-parallel test setup stuff does not allow for a driver either,
but that is not something we announce in all our names either.)
* lib/pt-driver: New auxiliary script.
* lib/Makefile.am (dist_SCRIPT_DATA): Add it.
* automake.in (handle_tests): Require the new auxiliary script
`pt-driver', and define new makefile variable `$(am__pt_driver)',
So this could just be $(am__test_driver).
used to call it. Perform new substitution on `DRIVER' when
processing the `check2.am' file.
* lib/check.am (am__tty_colors): Define new shell variable
`$am__color_tests'.
(am__rst_section): Removed, its role taken over by the new
`pt-driver' script.
(am__test_driver_flags): New variable, contains the command
line options passed to `pt-driver'.
(am__check_pre): Do not deal with temporary files and exit
traps anymore, as the `pt-driver' script takes care of that now.
Define shell variable `$am__enable_hard_errors', used by
`$(am__test_driver_flags)'. Reorder so that we don't need to
save and restore the value of the `TERM' environment variable
anymore.
Other related adjustments.
(am__check_post): Remove, as its role has been completely taken
over by the `pt-driver' script.
* am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
?!GENERIC?%OBJ%): Call the test script through the Automake
substituted `%DRIVER%', and honor the command-line options
in `$(am__test_driver_flags)'. Do not call the obsoleted
`$(am__check_post)' anymore.
* tests/check.test: Adjust.
* tests/check2.test : Likewise.
* tests/check3.test : Likewise.
* tests/check4.test : Likewise.
* tests/check10.test: Likewise.
* tests/color.test: Likewise.
* tests/color2.test: Likewise.
* tests/comment9.test: Likewise.
* tests/dejagnu.test: Likewise.
* tests/exeext4.test: Likewise.
* tests/maken3.test: Likewise.
* tests/maken4.test: Likewise.
* tests/parallel-tests-interrupt.test: Likewise.
* tests/posixsubst-tests.test: Likewise.
* tests/repeated-options.test: Likewise.
* tests/check-no-pt-driver.test: New test.
* tests/parallel-tests-pt-driver.test: Likewise.
* tests/Makefile.am (TESTS): Update.
* NEWS: Update.
index 054c62d..f3116c8 100644
--- a/lib/am/check2.am
+++ b/lib/am/check2.am
@@ -17,7 +17,9 @@
## From a test file to a log file.
?GENERIC?%EXT%.log:
?!GENERIC?%OBJ%: %SOURCE%
- @p='%SOURCE%'; $(am__check_pre) %COMPILE% $$tst $(am__check_post)
+ @p='%SOURCE%'; $(am__check_pre) \
+ %DRIVER% $(am__test_driver_flags) -- \
+ %COMPILE% $$tst
Erm, was there not a need to keep post flags for redirection? I must
confess I haven't gone through all gnulib threads from the last weeks
yet, but I vaguely remember related issues coming up there.
--- /dev/null
+++ b/lib/pt-driver
@@ -0,0 +1,129 @@
+#! /bin/sh
+# pt-driver - basic driver script for the `parallel-tests' mode.
+# Make unconditional expansion of undefined variables an error. This
+# helps a lot in preventing typo-related bugs.
+set -u
Not sure how older shells handle this, but for development it should be
ok. We haven't used it elsewhere.
+print_usage ()
+{
+ cat END
+Usage:
+ pt-driver [--help|--version] --test-name=NAME --log-file=PATH
+ [--expect-failure={yes|no}] [--color-tests={yes|no}]
+ [--enable-hard-errors={yes|no}] [--] TEST-SCRIPT
+The \`--test-name' and \`--log-file' options are mandatory.
+END
echo \
Usage ...
?
(Old shells will create (and sometimes forget to remove) a temporary
file for this here-document, even if print_usage is never
Re: [PATCH v4 3/3] parallel-tests: allow each test to have multiple results
* Stefano Lattarini wrote on Thu, Jun 16, 2011 at 10:03:59AM CEST:
With this change, we improve the code creating the `test-suite.log'
global log and the console testsuite summary to make it able to
grasp multiple results per test script. This is required in order
to introduce the planned support for test protocols, like TAP and
SubUnit, which can run multiple testcases per test script.
The implementation makes use of a custom reStructuredText field
`:am-testcase-result:'.
Why an Automake-specific name here? This is user facing, it should
be ':testcase-result:'. But 'testcase' is not a word, at least not
a good one. Please check what other test suite codes use, but I'd
just use ':test-result:' or ':result:' (if that is not ambiguous).
Other third-party code might want to make use of this too.
Actually, why not s/testcase/test/g globally in all your text.
I haven't gone over this in detail yet, so just a few random remarks
below.
And by the way, thanks for all your work on this so far!
Cheers,
Ralf
* lib/check.am ($(TEST_SUITE_LOG)): When processing .log files,
recognize a testcase result report only if it is declared with
the custom `:am-testcase-result:' reStructuredText field placed
at the beginning of a line. Extend and add explanatory comments.
(recheck, recheck-html): Add explanatory comments.
* lib/pt-driver: Write an appropriate `:am-testcase-result:'
reStructuredText field in the generated log file. Use a
reStructuredText transition to better separate the test outcome
report from the test registered output. Improve comments.
* tests/test-driver-custom-xfail-tests.test: Adapt.
* tests/parallel-tests-empty-testlogs.test: New test.
* tests/parallel-tests-recheck-override.test: Likewise.
* tests/parallel-tests2.test: Extend and keep more in-sync with ...
* tests/test-driver-custom-html.test: ... this new related test.
* tests/test-driver-custom-no-html.test: New test.
* tests/test-driver-custom-multitest.test: Likewise.
* tests/test-driver-custom-multitest-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck2.test: Likewise.
* tests/ostp-driver: New file, used by the last four tests above.
tests/simple-driver, or maybe trivial-test-driver (as simple driver is
already used in the manual for something different) or something. ostp
is just confusing, and you've argued against 8+3 conventions all the
time. ;-
* tests/Makefile.am (TESTS): Update.
(EXTRA_DIST): Distribute `ostp-driver'.
(test-driver-custom-multitest.log): Depend on `ostp-driver'.
(test-driver-custom-multitest-recheck.log): Likewise.
(test-driver-custom-multitest-recheck2.log): Likewise.
(test-driver-custom-html.log): Likewise.
* doc/automake.texi (API for Custom Test Drivers): Update (still
in Texinfo comments only).
I don't like TODO and comments stuff in finished patches documentation
if we can help it (and here, we surely can).
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -9052,8 +9052,9 @@ if the exact interpretation of the associated semantics
can change
between a test driver and another, and even be a no-op in some drivers).
@b{TODO!} @i{Options and flags that the driver must handle. Generation
-of ``.log'' files. Console output the driver is expected to produce.
-Support for colored output, XFAIL_TESTS, and DISABLE_HARD_ERRORS}
+of ``.log'' files, and format they must obey. Console output the driver
@file{.log}
You should almost never need to use ``...'' in texinfo, at least not for
any entity that has a meaning in code.
+is expected to produce. Support for colored output, XFAIL_TESTS, and
+DISABLE_HARD_ERRORS.}
@c
@c The driver script should follow a simple protocol in order to really
@@ -9075,6 +9076,29 @@ Support for colored output, XFAIL_TESTS, and
DISABLE_HARD_ERRORS}
@cdriver can use temporary files if it needs to, only it should clean
@cthem up properly).
@c
+@c * The result of each testcase run by a test script/program *must*
+@cbe registered in the test log using a custom reStructuredText
+@cfield ``am-testcase-result''. For example, if the test script
See above.
+@cexecutes two test cases, one successful and one failing, and skip
+@canother test case, the driver should end up writing the following
+@cin the test log:
+@c :am-testcase-result: PASS [passed testcase name or details]
+@c :am-testcase-result: FAIL [failed testcase name or details]
+@c :am-testcase-result: SKIP [skipped testcase name or details]
is the [...] literal or metasyntactic? Please use proper notation here.
+@cThe above lines (each of which *must* be followed by a blank line
+@cin order for the HTML output generation to work) are used only
+@cwhen generating the `test-suite.log' from the individual test
See above.
+@clogs, and can be placed in any order and position within the logs
+@citself.
+@c
+@c * The result of each testcase run by a test
Re: [PATCH] check: don't use multi-line coloring for the report
Hello, * Bert Wesarg wrote on Thu, Jun 16, 2011 at 08:19:23PM CEST: the parallel part is a little trickier. Because the line printing is done by awk. I would like to know, whether it is portable to use the printf function of awk. It is POSIX, but you may know that this doesn't count much. True. In case of doubt, try Solaris awk, that's fairly old. I couldn't find any prior usage in automake and autoconf either. Nor does the autoconf manual list printf as a functions in the traditional awk. Then I would prefer to do without. I don't see why you would need it for your change at all, however. All the coloring is done from echo statements even in the parallel case, and you can put the resets in there as well, methinks. Thanks, Ralf
Re: [REQUEST] Public temporary rewindable branches for GSoC features
Hi Stefano, * Stefano Lattarini wrote on Tue, Jun 14, 2011 at 05:55:02PM CEST: Would be ok with you if in the future I create some temporary branches *in the automake official repository* that I can rebase, edit, and delete at will? I don't mind, as long as they are marked/documented as such (and your naming strategy would seem to indicate that clearly enough), but I have a couple of (non rhetorical) questions: I think that keeping future unfinished or experimental work open and public could help speeding up the project advancement, Why do you think this would be the case? and IMHO it would be more expedient than having to keep track of 3 o 4 versions of the same patch series only through the mailing list archives. Why does work for *you* get less when you publish branches (as opposed to not)? Is this aimed to increase visibility of your changes? Or help the reviewer? As reviewer I'm usually more concerned with why something was done, and maybe how the patch evolved. But things like rationale can typically only be found in the mails describing a change, rather than the change itself. Thanks, Ralf
Re: Patch reviews
Hello guys, On Tue, Jun 07, 2011 at 09:25:33AM +0200, Peter Rosin wrote: Now that Ralf is doing something elsetm for a while, Thanks for the roses. After moving twice within 5 weeks (and generally having far too little time), things should gradually improve as soon as I have a landline internet connection at home again ... I'm more inclined to steal a few minutes here and there to suggest something. That may be true for others as well, so I think it's bad to assume that Ralf is the only possible reviewer. Indeed, a single point of failure is a very bad situation in general. So, any help is appreciated. Thanks. Cheers, Ralf
Re: [FYI] {maint} tests/README: update obsoleted advice
* Stefano Lattarini wrote on Fri, May 20, 2011 at 10:21:09PM CEST: * tests/README (Section Writing test cases subsection Do): Do not suggest to use the `*-p.test' pattern for the names of hand-written tests which use the `parallel-tests' Automake option. Not only is this not respected by the existing tests, but it is more likely to cause conflicts with auto-generated tests. So, suggest to *avoid* using the `*-p.test' pattern in names of hand-written tests instead. (Section Writing test cases subsection Do not): When suggesting not to override Makefile variables using command line arguments, do not use the badly outdated variables `U' and 'ANSI2KNR' in the example; instead, use the more common and typical `DESTDIR'. Actually, this change has a slight technical error: when some variable is never initialized in the Makefile, -e is not necessary in order to override it. DESTDIR is such a variable: we ensure that we do not ever initialize it. And as such, it is quite portable to use make DESTDIR=/foo/bar install and in fact, quite widely used. Can we use some other variable as example? How about prefix? Cheers, Ralf --- a/tests/README +++ b/tests/README @@ -107,8 +107,8 @@ Do For tests that use the `parallel-tests' Automake option, set the shell variable `parallel_tests' to yes before including ./defs. Also, - use for them a name that ends in `-p.test' and does not clash with any - generated tests in the suite. + do not use for them a name that ends in `-p.test', since that would + risk to clash with automatically-generated tests. ./defs sets a skeleton configure.in. If possible, append to this file. In some cases you'll have to overwrite it, but this should @@ -177,12 +177,12 @@ Do not reason, but at least it makes sure the original error is still here.) - Do not override Makefile variables using make arguments, as in -$MAKE ANSI2KNR=./ansi2knr U=_ all - this is not portable for recursive targets (targets that - call a sub-make may not pass `ANSI2KNR=./ansi2knr U=_' along). - Use the following instead. -ANSI2KNR=./ansi2knr U=_ $MAKE -e all + Do not override Makefile variables using make arguments, as in e.g.: +$MAKE DESTDIR=/foo/bar install + This is not portable for recursive targets (targets that call a + sub-make may not pass `DESTDIR=/foo/bar' along). Use the following + instead: +DESTDIR=/foo/bar $MAKE -e install Do not send a test case without signing a copyright disclaimer. See http://sources.redhat.com/automake/contribute.html or
Re: {testsuite-work} [PATCH] tests: avoid spurious failures in cross-compile mode
Hi Stefano, * Stefano Lattarini wrote on Wed, May 18, 2011 at 06:19:55PM CEST: I will wait until tomorrow for a review, before pushing. I really don't have the time to do a close review right now, but can you use 'native' instead of 'non-cross' and go ahead with that? That would fit the lingo better. Thanks, Ralf Subject: [PATCH] tests: avoid spurious failures in cross-compile mode * tests/depcomp2.test: Ensure verbose printing of captured stderr from configure. * tests/ansi3.test ($required): Add 'non-cross', as the ansi2knr functionality is not meant to work with a cross-compiler. [...]
Re: [FYI] {parallel-tests-maint} docs: parallel-tests is not experimental anymore
Hi Stefano, * Stefano Lattarini wrote on Wed, May 11, 2011 at 04:07:46PM CEST: The parallel-tests driver has now been used quite extensively by a fair number of real-world applications (e.g., GNU coreutils, GNU libtool, GNU grep, and various packages using Gnulib), and thus exposed to adequate on-field testing. So there's no point in declaring it experimental anymore (which would risk to make potential users shy away from it). Well, the point of declaring it experimental is being able to do at least slightly incompatible changes and mostly getting away with it. The changes might not just be needed for portability reasons in the *current* code, but also for newer features in changed code. I'm willing to bet that your SoC project will turn up one or two such situations. Cheers, Ralf * doc/automake.texi (Simple Tests using parallel-tests): Do not declare the parallel-tests driver as experimental anymore.
Re: [FYI] {parallel-tests-maint} docs: parallel-tests is not experimental anymore
* Stefano Lattarini wrote on Sun, May 15, 2011 at 04:10:29PM CEST: On Sunday 15 May 2011, Ralf Wildenhues wrote: Well, the point of declaring it experimental is being able to do at least slightly incompatible changes and mostly getting away with it. The changes might not just be needed for portability reasons in the *current* code, but also for newer features in changed code. I'm willing to bet that your SoC project will turn up one or two such situations. Should I revert this patch then? Or should we add a more watered-down warning in place of the previous scary one? What was scary about the previous one? I didn't find it scary. Thanks, Ralf
Re: [FYI] {maint} testsuite: be more cross-compile friendly
* Stefano Lattarini wrote on Sun, May 15, 2011 at 03:48:43PM CEST:
I've applied the attached patch to 'maint', rather than to 'testsuite-work',
only to make it easier to (potentially) backport tests for future bugfixing.
I plan to check in fixes tests that don't currently work in cross-compile
mode only in the 'testsuite-work' branch, to avoid excessive churn on
'maint' and 'master'.
Subject: [PATCH] testsuite: be more cross-compile friendly
* tests/defs.in (cross_compiling): New subroutine.
(am__tool_prefix): New internal variable.
(gcc, g++, gcj): Force the use of the correct tool prefix
when cross compiling.
(gfortran, g77, non-cross): New requirements.
--- a/tests/defs.in
+++ b/tests/defs.in
@@ -148,6 +148,23 @@ fail_ () { warn_ $me: failed test: $@; Exit 1; }
skip_ () { warn_ $me: skipped test: $@; Exit 77; }
framework_failure_ () { warn_ $me: set-up failure: $@; Exit 99; }
+# cross_compiling
+# ---
+# Tell whether we are cross-compiling. This is especially useful to skip
+# tests (or portions of them) that requires a native compiler.
+cross_compiling ()
+{
+ test x$host_alias != x
+}
FWIW, this condition isn't the same as the one configure uses in order
to determine whether cross compilation is enabled or not. Is that an
oversight or on purpose? In the latter case, it would be prudent to
mention this in the comment.
Thanks,
Ralf
Re: [FYI] {maint} tests: fix spurious failure of txinfo21.test on FreeBSD
[ apologies if you receive this multiple times ] Hi Stefano, * Stefano Lattarini wrote on Sat, May 07, 2011 at 03:02:02PM CEST: * tests/txinfo21.test: Use the `is_newest' subroutine instead of the `ls -t' hack to to determine whether a file has been updated. This is required because at least FreeBSD `ls' do not sort files with the same timestamp in alphabetical order when using the `-t' option. Please make sure autoconf.texi and the FreeBSD PR database know about this ls glitch. Thank you, Ralf
Re: [PATCH] java: allow both dist_JAVA and nodist_JAVA in the same Makefile.am
* Stefano Lattarini wrote on Wed, Apr 20, 2011 at 09:10:49AM CEST: On Wednesday 20 April 2011, Ralf Wildenhues wrote: I expect to be able for some review on Friday, maybe on Thursday after that, and then hopefully on March 3. I guess you mean May 3 here. Yeah; sorry. What about the following middleground solution. I trust you to do the right thing, however it looks like. To me, the important bits are: the number of regressions should be as monotonically non-increasing as possible, and I would like to be able to review stuff at some point (i.e., glance at all changes, do a proper review where appropriate). I smoke-tested FreeBSD last night, about 15 test failures. If it helps you, I can spawn a full test run (which branch would you like?) tonight and upload logs by Friday, that should give an idea about recent developments. Thanks, Ralf
Re: [PATCH] java: allow both dist_JAVA and nodist_JAVA in the same Makefile.am
Hello Stefano, * Stefano Lattarini wrote on Mon, Apr 18, 2011 at 09:35:52PM CEST: I'll push in 72 hours if there is no objection. I expect to be able for some review on Friday, maybe on Thursday after that, and then hopefully on March 3. There might be occasions in between, but if you want to give me a realistic chance for review, 72 hours are not going to suffice during the next two weeks as I might be without net access for longer than that. I don't want to keep you from making changes, but with stuff that should be reviewed, you might want to increase the timeout somewhat. Thanks, Ralf
Re: [FYI] {maint,master} test defs: allow overriding of `$me'
* Stefano Lattarini wrote on Mon, Apr 18, 2011 at 10:27:04AM CEST: On Monday 18 April 2011, Ralf Wildenhues wrote: * Stefano Lattarini wrote on Sun, Apr 17, 2011 at 09:36:42PM CEST: http://lists.gnu.org/archive/html/automake-patches/2011-02/msg00044.html That explains why, from within the testsuite, you'd like to be able to override it for some tests. It doesn't explain why an override from the user calling 'make check' should be possible. (IOW, I understand the this would be convenient aspect, but it seems it should be possible to construct the testsuite in a way to still not allow user overrides.) Honestly, I'd not worry about this ATM (but I share your concerns about the lack of namespace cleanliness). Presently, a determined user can anyway wreak havoc by exporting variables such as 'required', 'MISSING' and 'parallel_tests' (and there are even more in the master branch: 'original_AUTOMAKE', 'am__using_gmake', 'instspc_action'). instspc_action is not dangerous, because its usage is safe, unlike original_AUTOMAKE. But 'me' is both a generic name, and dangerous (me=../../../oops). 'required' is less dangerous that way, but still a bit. A first step would IMHO be making these variables at least namespace-safe. Should I write a patch? Oh no please don't; the others are at least not so generic names, and I'd really loath the ripple that renaming $required will have. I would like to put out a stable point release in the near future and I'd prefer not so much churn before that. (Yeah, it doesn't really mesh well with me having close to no time at all right now, but it's about time we did that ...) Thanks, Ralf
Re: [PATCH] test defs: don't allow `$me' to be overridden from the environment
* Stefano Lattarini wrote on Mon, Apr 18, 2011 at 12:33:25AM CEST:
Subject: [PATCH] test defs: don't allow `$me' to be overridden from the
environment
* tests/defs.in ($me): Use the namespace-safe `$am_test_name' (if
it's nonempty) as the default for the initialization of `$me', so
that a (not unlikely) environment variable `me' won't wreak havoc
in the testsuite.
This is better, but still lacking an explanation for the question
I asked earlier. OK with that, and below.
--- a/tests/defs.in
+++ b/tests/defs.in
@@ -65,10 +65,13 @@ test -f $srcdir/defs.in || {
}
# The name of the current test (without the `.test' suffix).
-# Test scripts can override it if they need to (but this should
-# be done carefully, and *before* including ./defs).
-if test -z $me; then
+# Test scripts can override it if they need to, through the use of
+# the namespace-safe variable `$am_test_name' (but this should be
+# done carefully, and *before* including ./defs).
The comment change is redundant with the code change.
+if test -z $am_test_name; then
me=`echo $0 | sed -e 's,.*[\\/],,;s/\.test$//'`
+else
+ me=$am_test_name
fi
How about reordering so it reads nicely? I.e.,
if test -n $am_test_name; then
me=$am_test_name
else
...
Thanks,
Ralf
Re: [PATCH] test defs: don't allow `$me' to be overridden from the environment
* Stefano Lattarini wrote on Mon, Apr 18, 2011 at 09:26:30PM CEST: On Monday 18 April 2011, Ralf Wildenhues wrote: * Stefano Lattarini wrote on Mon, Apr 18, 2011 at 12:33:25AM CEST: Subject: [PATCH] test defs: don't allow `$me' to be overridden from the environment * tests/defs.in ($me): Use the namespace-safe `$am_test_name' (if it's nonempty) as the default for the initialization of `$me', so that a (not unlikely) environment variable `me' won't wreak havoc in the testsuite. This is better, but still lacking an explanation for the question I asked earlier. Hmmm... I thought I had addressed all your questions properly, but clearly I'm wrong. Which question have I missed? The one here: | am_test_name is better, but doesn't explain either why it would be | needed in the first place. | | Second patch of: | http://lists.gnu.org/archive/html/automake-patches/2011-02/msg00044.html | And possible similar patches in the future. That still doesn't answer why a user-available override would be needed. It answers why you want to make $me overridable *from within* the testsuite, but not why for the user running 'make check'. For example, tests/Makefile.am could have unset me ||:; as part of AM_TESTS_ENVIRONMENT. That wouldn't defeat users running tests themselves, but it would defeat breakage induced by differences in the user environment. Thanks, Ralf
Re: [PATCH] {maint} tests: new subroutines for test skipping/failing
Hello Stefano, * Stefano Lattarini wrote on Sun, Apr 17, 2011 at 03:27:17PM CEST: Reference: http://lists.gnu.org/archive/html/automake-patches/2011-03/msg00052.html On Wednesday 30 March 2011, Stefano Lattarini wrote: $(AM_TESTS_SETUP) $(TESTS_SETUP) $(TESTS_ENVIRONMENT) I think the AM_TESTS_SETUP naming is not a good one. It may be technically more correct than AM_TESTS_ENVIRONMENT, but I think the latter is the better one simply because it is easier to remember, and even if you've never heard of it and only know the semantics of TESTS_ENVIRONMENT, you can have a straightforward way to figure out how AM_TESTS_ENVIRONMENT would work. Can we agree to rename it? Sorry for jumping out now, and for not coming forward with a review earlier. Thanks, Ralf
Re: [FYI] {maint,master} test defs: allow overriding of `$me'
* Stefano Lattarini wrote on Sun, Apr 17, 2011 at 06:27:44PM CEST: Subject: [PATCH] test defs: allow overriding of `$me' * tests/defs.in ($me): Allow overriding by the including test script. Add some explicative comments. What if the user environment contains $me? Will that be overridden still by the testsuite? Otherwise you will have spurious failures (and to some extent arbitrary code execution upon a testsuite run). Thanks, Ralf
Re: [FYI] {maint,master} test defs: allow overriding of `$me'
Hi Stefano, * Stefano Lattarini wrote on Sun, Apr 17, 2011 at 09:36:42PM CEST: On Sunday 17 April 2011, Ralf Wildenhues wrote: am_test_name is better, but doesn't explain either why it would be needed in the first place. Second patch of: http://lists.gnu.org/archive/html/automake-patches/2011-02/msg00044.html And possible similar patches in the future. That explains why, from within the testsuite, you'd like to be able to override it for some tests. It doesn't explain why an override from the user calling 'make check' should be possible. (IOW, I understand the this would be convenient aspect, but it seems it should be possible to construct the testsuite in a way to still not allow user overrides.) Thanks, Ralf
Re: bug#8508: 8 test failures on Fedora 15
Hi Jim, * Jim Meyering wrote on Sat, Apr 16, 2011 at 11:59:33AM CEST: Subject: [PATCH] depcomp: correct invalid sed invocation * lib/depcomp: Insert missing -e before '/:$/d'. Otherwise, that use of sed would treat '/:$/d' as a file name. Thanks. Merged from the 'fix-depcomp' branch into all active branches. Sorry for the inconvenience. Cheers, Ralf
Re: [FYI] {yacc-work} coverage: test for automake bug#8485 (known regression)
Hello Stefano, * Stefano Lattarini wrote on Tue, Apr 12, 2011 at 03:26:33PM CEST: I've pushed the attached patch to yacc-work. The new testcase should correctly capture and expose the regression, as it passes with automake 1.11 and fails with developement (maint, master and yacc-work) versions of automake. Thanks for all your testing work. However, I think it is not ideal if we add tests for regressions on branches other than where the regressions occurred. When we fix the regression, we want the test in place right where the fix is needed, i.e., ideally on a new branch off of the offending commit just like we'd do with a fix. Am I gathering correctly that the src/foo.c: src/$(dirstamp) rule lets GNU make build the thing in the wrong directory due to the target lookup rules? Have you analyzed this failure, and if yes, could you please explain it? Thanks, Ralf Subject: [PATCH] coverage: test for automake bug#8485 (known regression) * tests/yacc-dist-nobuild-subdir.test: New test. * tests/Makefile.am (TESTS, XFAIL_TESTS): Update. index 000..b6811d7 --- /dev/null +++ b/tests/yacc-dist-nobuild-subdir.test + +# Check that VPATH builds and make distcheck works with packages +# using yacc and the automake 'subdir-objects' option. +# Exposes automake bug#8485. [...]
Re: bug#8473: depcomp bug with HP-UX cc and VPATH build
Hi Stefano, I've spotted a couple of harmful typos in the patch, and I also have a very minor nit. You'd do me a big favor if you fixed them right away (if you have time) and merge to active branches. I don't have my ssh keys available until tonight. Thanks, and sorry for the messup, Ralf
Re: [FYI] maint merged into yacc-work
* Stefano Lattarini wrote on Mon, Apr 11, 2011 at 05:33:47PM CEST: The yacc-work branch, due to its slow advancing pace, was falling more and more out of sync with maint, and thus with all the improvements and bug fixings done there in the meantime. Since yacc-work is meant to be eventually merged into either maint or master (we still have to decide which one fits better), there is no point in leaving this situation worsen further. I've thus merged maint into yacc-work, fixed the spurious resulting conflicts, and pushed. Thank you. If it helps you, feel free to do that more regularly. FWIW, I usually don't leave the Conflicts: comment in the merge commit log, because when the conflicts have been resolved, there is little reason to clutter up the log. You can anyway easily see the actual places that conflicted later, by looking at 'git show $merge_sha'. Cheers, Ralf
Re: bug#8473: depcomp bug with HP-UX cc and VPATH build
Hi Stefano, * Stefano Lattarini wrote on Mon, Apr 11, 2011 at 12:29:51PM CEST: On Monday 11 April 2011, Ralf Wildenhues wrote: I've spotted a couple of harmful typos in the patch, and I also have a very minor nit. You'd do me a big favor if you fixed them right away (if you have time) and merge to active branches. Done in the attached patch. It also turned out that depcomp10.test wasn't executable; I've fixed that too. Thank you for fixing up my brown paper bag bug. That should teach me to not reverse the ordering of coffee, getting patch review, pushing. Unfortunately, I cannot truly fully test the patch, because the 'hp' depmode doesn't work with gcc and Sun C compiler (and I don't hav access to any other compiler ATM). The test 'depmode10.test' is correctly skipped with those compilers. It passes on HP-UX 11.00 with MAKE=gmake. Thanks again, Ralf
Re: [PATCH] {maint} test defs: new requirement for the default java compiler
* Stefano Lattarini wrote on Sun, Apr 10, 2011 at 09:11:07AM CEST: IMHO having a proper requirement for the java compiler in `tests/defs' would be better than the current behaviour running the check for the availability of javac in the generated configure script in each relevant test. And it would also be faster for systems that lack a java compiler (which are then to skip the test). The usefulness of this patch will be further increased if we add more tests on java support in the future. So, OK to apply the attached patch to maint? Of course. Please be sure to convert all instances in master too when you merge (IIRC there should be more than in maint). Thanks! Ralf Subject: [PATCH] test defs: new requirement for the default java compiler * tests/defs.in (for tool in $required): New requirement 'javac'. * tests/java.test: Use it instead of ad-hoc configure check. * tests/java-check.test: Likewise.
Re: [PATCH] {maint} test defs: new requirement for the default java compiler
* Stefano Lattarini wrote on Sun, Apr 10, 2011 at 09:50:14AM CEST:
Thanks for the fast review,
Stefano
Hmm. Did you test this though? The tests are all skipping for me now now,
but I have javac installed:
SKIP: instdir-java.test (exit: 77)
==
/tmp/build/tests:/home/ralf/local/bin
instdir-java: running javac -version
javac 1.5.0_22
javac: no source files
Usage: javac options source files
where possible options include:
-g Generate all debugging info
-g:noneGenerate no debugging info
-g:{lines,vars,source} Generate only some debugging info
-nowarnGenerate no warnings
-verbose Output messages about what the compiler is doing
-deprecation Output source locations where deprecated APIs are u
-classpath path Specify where to find user class files
-cp path Specify where to find user class files
-sourcepath path Specify where to find input source files
-bootclasspath path Override location of bootstrap class files
-extdirs dirsOverride location of installed extensions
-endorseddirs dirs Override location of endorsed standards path
-d directory Specify where to place generated class files
-encoding encoding Specify character encoding used by source files
-source release Provide source compatibility with specified release
-target release Generate class files for specific VM version
-version Version information
-help Print a synopsis of standard options
-X Print a synopsis of nonstandard options
-Jflag Pass flag directly to the runtime system
Thanks,
Ralf
Re: [PATCH] {maint} test defs: new requirement for the default java compiler
* Stefano Lattarini wrote on Sun, Apr 10, 2011 at 10:23:11AM CEST: On Sunday 10 April 2011, Ralf Wildenhues wrote: Hmm. Did you test this though? Yes, but I only have javac 1.6.0 on my system, which works even with just the `-version' option: Oh, ok. Maybe adding also the `-help' option to the java invocation works? E.g., $ javac -version -help Yes, that works for me. Patches to that end preapproved. Thanks, Ralf
Re: bug#7648: [PATCH] {yacc-work} yacc: extension of headers modelled after extension of sources
* Stefano Lattarini wrote on Sun, Apr 10, 2011 at 10:26:40AM CEST:
http://lists.gnu.org/archive/html/automake-patches/2011-01/msg00306.html
Related to automake bug#7648 and PR automake/491.
* lib/am/yacc.am (am__yacc_c2h): New internal variable.
(?GENERIC?%EXT%%DERIVED-EXT%, ?!GENERIC?%OBJ%): Get the name of
the header dynamically at make runtime, so that its extension is
modelled after the extension of the source.
* automake.in (lang_yacc_target_hook): Adjust the calculation of
`$header' accordingly.
* tests/yacc-cxx.test: New test.
* tests/yacc-d-cxx.test: Likewise.
* tests/yacc-weirdnames.test: Likewise.
* tests/yacc-basic.test: Update comments.
* tests/yacc-d-basic.test: Likewise.
* tests/yaccpp.test: Updated and extended.
* tests/Makefile.am (TESTS): Update.
This patch has been applied in the meantime, but it lacked NEWS and
documentation updates. I will post them soonish in two different
patches.
While at it, you might want to fix
+// Valid as C++, but deliberatly invalid as C.
deliberately
I think you meant to write:
Valid C++, but deliberately invalid C.
+#include cstdio
+#include cstdlib
+int yylex (void) { return (getchar ()); }
Extra parentheses, not needed for return.
which is how far I got with reviewing before you pushed the patch.
Cheers,
Ralf
Re: [PATCH 1/2] yacc: update NEWS w.r.t. Yacc-generated headers extesions
* Stefano Lattarini wrote on Sun, Apr 10, 2011 at 10:46:33AM CEST:
OK for yacc-work? I will push in 72 hours if there is no objection.
OK with nits addressed. You can squash both changes together.
Subject: [PATCH 1/2] yacc: update NEWS w.r.t. Yacc-generated headers extesions
--- a/NEWS
+++ b/NEWS
@@ -5,9 +5,20 @@ New in 1.11.0a:
- The `lzma' compression scheme and associated automake option `dist-lzma'
is obsoleted by `xz' and `dist-xz' due to upstream changes.
+* Changes to Yacc support:
+
- C source and header files derived from non-distributed Yacc sources are
now removed by make clean, not only by make maintainer-clean.
+ - Slightly backward-incompatible change, relevant only for use of Yacc
+with C++: the extensions of the header files produced by the Yacc
+rules are now modelled after extension of the sources corresponding
+sources. For example, yacc files named foo.y++ and bar.yy will
+produce header files named respectively foo.h++ and bar.hh, where
+they would have previously produced header files named simply foo.h
+and bar.h. This change offers a better compatibility with the
s/ a / /
+results of `bison -o' calls.
s/the result of `bison -o' calls/`bison -o'/
* Stefano Lattarini wrote on Sun, Apr 10, 2011 at 10:46:48AM CEST:
Subject: [PATCH 2/2] yacc: update docs w.r.t. extension of yacc-generated
headers
* doc/automake.texi (Yacc and Lex): Document explicitly that
extensions of yacc-generated headers are modelled after
extension of the corresponding sources.
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -6048,10 +6048,14 @@ cause the intermediate file to be named @file{foo.c}
(as opposed to
@file{y.tab.c}, which is more traditional).
The extension of a yacc source file is used to determine the extension
-of the resulting C or C++ file. Files with the extension @file{.y}
-will be turned into @file{.c} files; likewise, @file{.yy} will become
-@file{.cc}; @file{.y++}, @file{c++}; @file{.yxx}, @file{.cxx}; and
-@file{.ypp}, @file{.cpp}.
+of the resulting C or C++ source and header file(s) (note that header
s/file(s)/files./
Start a new sentence after that, no need to put it in parentheses.
+files are generated only when the @option{-d} Yacc option is used; see
+below for more information about this flag, and how to specify it).
+Files with the extension @file{.y} will be turned into @file{.c}
+sources and @file{.h} headers; likewise, @file{.yy} will become
+@file{.cc} and @file{.hh}; @file{.y++}, @file{c++} and @file{h++};
... @file{.hh}, @file{.y++} will become @file{c++} and @file{h++},
..., and @file{.ypp} will become @file{.cpp} and @file{.hpp}.
+@file{.yxx}, @file{.cxx} and @file{.hxx}; and @file{.ypp}, @file{.cpp}
+and @file{.hpp}.
Likewise, lex source files can be used to generate C or C++; the
Here, I'd continue with Similarly, now.
extensions @file{.l}, @file{.ll}, @file{.l++}, @file{.lxx}, and
Thanks,
Ralf
Re: [PATCH] cosmetics: fix typos and wording in some yacc tests
* Stefano Lattarini wrote on Sun, Apr 10, 2011 at 11:17:10AM CEST: On Sunday 10 April 2011, Ralf Wildenhues wrote: While at it, you might want to fix I've fixed that, plus some similar blunders, and added a few other misc fixlets too. The resulting patch is attached. OK for yacc-work? Sure. Thanks, Ralf Subject: [PATCH] cosmetics: fix typos and wording in some yacc tests * tests/yacc-cxx.test (foo.cc): Clarify comment about the content of this file being valid C++ but invalid C. (parse1.yy): Likewise. Also, remove redundant parentheses in a `return' statement. * tests/yacc-d-cxx.test (write_parse): Clarify comment about the content of the generated files being valid C++ but invalid C. (write_main): Likewise. * tests/yacc-basic.test: Remove redundant parentheses in a `return' statement. * tests/yacc-d-vpath.test: Adjust spacing around curly brackets. * tests/yaccvpath.test: Likewise. * tests/yaccdry.test: Likewise. * tests/yacc8.test: Likewise. * tests/yacc4.test: Likewise.
Re: [PATCH] {yacc-work} coverage: test mixed C/C++ yacc-generated parsers in the same dir
* Stefano Lattarini wrote on Sun, Apr 10, 2011 at 12:57:26PM CEST:
A final testcase I'd like to check in before submitting the final
patch for automake bug#7648 and PR automake/491 (patch not yet
complete ATM, but I think I'm almost there). It checks that many
different Yacc parsers (both C and C++) can co-exists in the same
directory.
OK for yacc-work? I will push in 72 hours if there is no objection.
OK, a couple of minor nits.
Thanks,
Ralf
Subject: [PATCH] coverage: test mixed C/C++ yacc-generated parsers in the
same dir
* tests/yacc-mix-c-cxx.test: New test.
* tests/Makefile.am (TESTS): Update.
--- /dev/null
+++ b/tests/yacc-mix-c-cxx.test
+# Check that many different Yacc parsers (both C and C++) can co-exists
+# in the same directory.
+
+required=yacc
+. ./defs || Exit 1
+
+set -e
+
+distdir=$me-1.0
+
+cat configure.in 'END'
+AC_PROG_CC
+AC_PROG_CXX
+AC_PROG_YACC
+AC_OUTPUT
+END
+
+cat Makefile.am 'END'
+bin_PROGRAMS = c1 c2 cxx1 cxx2 cxx3
+AM_YFLAGS = -d
+
+c1_SOURCES = p.y p.h 1.c
+c2_SOURCES = p.y 2.c
+c2_YFLAGS =
+
+cxx1_SOURCES = parse.yy main1.cc parse.hh
+
+cxx2_SOURCES = parse2.y++ main2.c++
+cxx2_YFLAGS =
+
+cxx3_SOURCES = parse3.yxx main3.cxx
+BUILT_SOURCES = parse3.hxx
+END
+
+# The content of all the .c and .y files created below is valid C but
+# deliberately invalid as C++.
s/as // I think.
+# Vice versa, the content of all the .c++, .cxx, .cc, .y++, .yxx and
+# .yy files created below is valid C++ but deliberately invalid C.
+
+cat p.y 'END'
+%{
+int yylex (void) { int new = 0; return new; }
+void yyerror (char *s) { return; }
+%}
+%token ZARDOZ
+%%
+x : 'x' {};
+%%
+END
+
+cat 1.c 'END'
+#include p.h
+int main ()
+{
+int new = ZARDOZ;
+return yyparse () + new;
+}
+
+END
+
+cat 2.c 'END'
+int main ()
+{
+int yyparse ();
+int new = 0;
+return yyparse () + new;
+}
+END
+
+cat parse.yy 'END'
+%{
+#include cstdlib
+#include parse.hh
+int yylex (void) { return 0; }
+void yyerror (const char *s) { return; }
+%}
+%token FOOBAR
+%%
+x : 'x' {};
+%%
+END
+
+cat parse2.y++ 'END'
+%{
+#include cstdlib
+int yylex (void) { return 0; }
+void yyerror (const char *s) { return; }
+%}
+%%
+x : 'x' {};
+%%
+END
+
+cat main1.cc 'END'
+using namespace std;
+#include parse.hh
+int main (int argc, char **argv)
+{
+int yyparse (void);
+return yyparse () + FOOBAR;
+}
+END
+
+cat main2.c++ 'END'
+using namespace std;
+int main (int argc, char **argv)
+{
+int yyparse (void);
+return yyparse ();
+}
+END
+
+edit () { sed -e 's/FOOBAR/BAZQUUX/' -e 's/parse\.hh/parse3.hxx/'; }
+edit parse.yy parse3.yxx
+edit main1.cc main3.cxx
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+$MAKE
+ls -l # For debugging.
+
+test -f p.c
+test -f p.h
+test -f c2-p.c
+test ! -r c2-p.h
+
+test -f parse.cc
+test -f parse.hh
+test -f parse3.cxx
+test -f parse3.hxx
+
+test -f cxx2-parse2.c++
+test ! -r parse2.h++
+test ! -r cxx2-parse2.h++
+
+# Minimal checks about recovering from header removal.
+rm -f p.h parse.hh parse3.hxx
+$MAKE p.h parse.hh
+test -f p.h
+test -f parse.hh
+test ! -r parse3.hxx
+$MAKE
+test -f parse3.hxx
I think it would make sense to also do parallel build tests
(only if make is GNU make, of course) here.
Thanks,
Ralf
Re: [PATCH] {yacc-work} coverage: test mixed C/C++ yacc-generated parsers in the same dir
* Stefano Lattarini wrote on Sun, Apr 10, 2011 at 02:50:28PM CEST: On Sunday 10 April 2011, Ralf Wildenhues wrote: I think it would make sense to also do parallel build tests here Agreed (and BTW this uncovered a bug in the testcase: I had forgotten to add `p.h' and `parse.hh' to BUILT_SOURCES in the Makefile.am; it's fixed now). Hehe. Here is what we could do IMHO: first try a VPATH serial build, then an in-tree parallel build. See the attached squash-in. Tested with GNU make (3.80, 3.81, 3.82), FreeBSD and NetBSD make (Debian ports), Heirloom make, Solaris XPG4 and CCS make, and Solaris dmake. OK? OK. (only if make is GNU make, of course) Why? I've never found a make implementation rejecting the `-j' option. HP-UX make has -P but not -j. The semantics of -P are sufficiently different so that it is not worth trying to support that, however. $ make -j; echo $? Make: Unknown flag argument j. Stop. 1 Several other makes accept -j only when given a number, not without, e.g., OpenBSD make: $ make -j make: option requires an argument -- j usage: make [-BeiknPqrSst] [-D variable] [-d flags] [-f makefile] [-I directory] [-j max_jobs] [-m directory] [-V variable] [NAME=value] [target ...] And I think for portability to all GNU make versions, the number should follow immediately, i.e., without intervening space, but I'm going from memory here. It's fine to try out first if make groks -j, and use that, that gives more coverage then. But you should try it out in a way to not expose the issues you just spotted. For example, you could use make -j2 clean || Exit 77 (or Exit 0, if you don't consider that part of the test important enough). +# Try a VPATH and by default serial build first, and then an in-tree +# and by default parallel build. + +for try in 0 1; do + + if test $try -eq 0; then +mkdir build +cd build +srcdir=.. +debug_info=ls -l . $srcdir +run_make=$MAKE + elif test $try -eq 1; then +srcdir=. +debug_info=ls -l +case $MAKE in + *\ -j*) run_make=$MAKE;; + *) run_make=$MAKE -j 3;; See above for spacing. Also, note that $MAKE might already include -jN. Thanks, Ralf
Re: [PATCH] {yacc-work} coverage: test mixed C/C++ yacc-generated parsers in the same dir
* Stefano Lattarini wrote on Sun, Apr 10, 2011 at 03:34:12PM CEST: Or, since we are doing both VPATH an in-tree builds, just continue with serial make if `-j' doesn't work. See the attached squash-in (which goes on the top of the previous squash-in). OK? OK. Thanks!
Re: bug#8473: depcomp bug with HP-UX cc and VPATH build
Hi Bruno, * Bruno Haible wrote on Mon, Apr 11, 2011 at 02:44:48AM CEST: The dependencies mechanism of Automake leads to a compilation failure when used in a VPATH build (with GNU make, of course) Actually, in this case you could have avoided this particular bug by using HP make. :-p Automake still strives to be portable to non-GNU make, even if you have decided to (understandably) not worry about the numerous and hideous VPATH issues. source='../../gllib/nonblocking.c' object='../../gllib/nonblocking.o' libtool=no \ DEPDIR=.deps depmode=hp /bin/sh ../../build-aux/depcomp \ cc -Ae -O -DHAVE_CONFIG_H -I. -I../../gllib -I.. -DGNULIB_STRICT_CHECKING=1 -g -c ../../gllib/nonblocking.c cpp: , line 0: error 4066: Cannot create ../../gllib/.deps/nonblocking.TPo file for -M../../gllib/.deps/nonblocking.TPo option. (No such file or directory[errno=2]) gmake[4]: *** [../../gllib/nonblocking.o] Error 1 How nice, I just fixed a similar bug in makedepend mode last week. I'm pushing this to maint (and merging to branch-1.11 and master) to fix the above. Thanks for the report, Ralf Fix hp depmode for VPATH builds with GNU make. * lib/depcomp: Be sure to remove VPATH-prefixed object from dependency output when creating stub rule. * tests/depcomp10.test: New test. * tests/Makefile.am (TESTS): Update. * NEWS: Update. Report by Bruno Haible. diff --git a/NEWS b/NEWS index 3cd5ab0..c8219bd 100644 --- a/NEWS +++ b/NEWS @@ -57,7 +57,7 @@ Bugs fixed in 1.11.0a: - The parallel-tests driver now does not produce erroneous results with Tru64/OSF 5.1 sh upon unreadable log files any more. - - The makedepend depmode now works better with VPATH builds. + - The makedepend and hp depmodes now works better with VPATH builds. - Java sources specified with check_JAVA are not compiled anymore upon make all, but only upon make check. diff --git a/lib/depcomp b/lib/depcomp index 8097c19..e996e5d 100755 --- a/lib/depcomp +++ b/lib/depcomp @@ -1,7 +1,7 @@ #! /bin/sh # depcomp - compile a program generating dependencies as side-effects -scriptversion=2011-04-09-11; # UTC +scriptversion=2011-04-11-05; # UTC # Copyright (C) 1999, 2000, 2003, 2004, 2005, 2006, 2007, 2009, 2011, # Free Software Foundation, Inc. @@ -158,10 +158,12 @@ gcc) ' $tmpdepfile | ## Some versions of gcc put a space before the `:'. On the theory ## that the space means something, we add a space to the output as -## well. +## well. hp depmode also adds that space, but also prefixes the VPATH +## to the object. Take care to not repeat it in the output. ## Some versions of the HPUX 10.20 sed can't process this invocation ## correctly. Breaking it into two sed invocations is a workaround. -sed -e 's/^\\$//' -e '/^$/d' -e '/:$/d' | sed -e 's/$/ :/' $depfile +sed -e 's/^\\$//' -e '/^$/d' -e -e s|.*$object$|| '/:$/d' \ + | sed -e 's/$/ :/' $depfile rm -f $tmpdepfile ;; diff --git a/tests/Makefile.am b/tests/Makefile.am index bf07f2b..4e87805 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -278,6 +278,7 @@ depcomp7.test \ depcomp8a.test \ depcomp8b.test \ depcomp9.test \ +depcomp10.test \ depdist.test \ depend.test \ depend2.test \ diff --git a/tests/depcomp10.test b/tests/depcomp10.test new file mode 100644 index 000..4fdee40 --- /dev/null +++ b/tests/depcomp10.test @@ -0,0 +1,81 @@ +#! /bin/sh +# Copyright (C) 2011 Free Software Foundation, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2, or (at your option) +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see http://www.gnu.org/licenses/. + +# hp depmode should work with GNU make in VPATH mode (bug similar to +# depcomp9.test). + +# Here's the bug: hp depmode will prefix VPATH to the object file name, +# thus the second gmake will invoke depcomp with object='../../src/foo.o', +# causing errors such as: +# cpp: , line 0: error 4066: Cannot create ../../gllib/.deps/nonblocking.TPo file for -M../../gllib/.deps/nonblocking.TPo option. (No such file or dir ++ectory[errno=2]) + +required=GNUmake +. ./defs || Exit 1 + +mkdir src src/sub build + +cat configure.in 'END' +AC_PROG_CC +AM_PROG_CC_C_O +AC_CONFIG_FILES([src/Makefile]) +AC_OUTPUT +END + +cat Makefile.am 'END' +SUBDIRS = src +END + +cat src/Makefile.am 'END' +AUTOMAKE_OPTIONS = subdir-objects +bin_PROGRAMS = foo +foo_SOURCES = foo.c foo.h sub/subfoo.c +END + +cat src/foo.h EOF +extern int subfoo (void); +EOF + +cat src/foo.c EOF +#include
Re: bug#8473: depcomp bug with HP-UX cc and VPATH build
* Ralf Wildenhues wrote on Mon, Apr 11, 2011 at 07:13:00AM CEST: Fix hp depmode for VPATH builds with GNU make. * lib/depcomp: Be sure to remove VPATH-prefixed object from dependency output when creating stub rule. * tests/depcomp10.test: New test. * tests/Makefile.am (TESTS): Update. * NEWS: Update. Report by Bruno Haible. Of course I should be merging this test and depcomp9, and generalizing for all depmodes, and tuning my test scripts to exercise both make and gmake on all systems. (on my TODO list) Cheers, Ralf
Re: Fix makedepend depmode to cope with VPATH builds
Hi Stefano,
* Stefano Lattarini wrote on Sat, Apr 09, 2011 at 11:38:47AM CEST:
On Friday 08 April 2011, Ralf Wildenhues wrote:
* Stefano Lattarini wrote on Fri, Apr 08, 2011 at 01:05:53PM CEST:
On Wednesday 06 April 2011, Ralf Wildenhues wrote:
+ # makedepend may prepend the VPATH from the source file name to the
object.
+ sed_object_re=`echo $object | sed 's/[].[^$\\*|]//g'`
A couple of questions/observations about the sed command:
1. I wasn't sure whether `.', `$', `^' and `*' are to be escaped when
used in
a bracket expression; however, the POSIX standard at:
http://pubs.opengroup.org/onlinepubs/007908799/xbd/re.html
says they shouldn't be. Also, I couldn't find any reference in the
Autoconf manual pointing to a sed implementation that doesn't respect
this part of the standard.
Ah, cool. Then I recon we don't need this escaping step at all.
Why not? The sed command below:
sed s|^.*\($sed_object_re *:\)|\1| $tmpdepfile $depfile
doesn't use any bracket, does it?
I'm not sure I understand the question. Can you rephrase it?
The only character I was worried about was $, as it can occur in
object names now and then (ok, mostly java, and that doesn't use
depcomp). Any others in the list are pretty much excluded anyway,
because they will evoke shell expansion errors when used in a
makefile unquoted. Except for '.', but I'm not afraid of matching
a bit too broadly here, the final 'o' will take care that we don't
match any header files.
OK, this makes sense (even if I'd err on the safe side w.r.t. the `.'
character),
OK, can you come up with a practical example (i.e., a valid object file
name) where we would match something wrongly here? Remember the input
file only contains lines of the form
$object: header header...
$object: header \
header header...
While somebody might arguably name their header foo.o, I do think they
deserve what they get.
but I don't see how it's related to my observation above.
What am I missing?
I guess then I don't understand your observation then. Your observation
tells me that object file names of the form foo^bar.o and foo$baz.o are
not problematic. I observe that no one in their right mind names their
object foo[bar.o or foo]bar.o or foo\bar.o (see the various instspc*
tests for why they would not work in practice anyway). So I conclude
that it is not needed to do regex-escaping for the object file name.
@@ -503,7 +503,8 @@ makedepend)
touch $tmpdepfile
${MAKEDEPEND-makedepend} -o$obj_suffix -f$tmpdepfile $@
rm -f $depfile
- cat $tmpdepfile $depfile
+ # makedepend may prepend the VPATH from the source file name to the
object.
I'd add a comment here telling why there's no need to escape $object for the
sed commmand just below.
I will, after I've understood your remaining comments.
Thanks,
Ralf
Re: Fix makedepend depmode to cope with VPATH builds
* Stefano Lattarini wrote on Sat, Apr 09, 2011 at 12:29:43PM CEST:
On Saturday 09 April 2011, Ralf Wildenhues wrote:
I guess then I don't understand your observation then. Your observation
tells me that object file names of the form foo^bar.o and foo$baz.o are
not problematic.
No. It just tells that sed regular expressions like [ab.^$0-9] are not
problematic, because `.' and `$' are *not* recognized as metacharacters
in *bracket expression*, and `^' is recognized as a metacharacter in a
bracket expression only when used as *first* character there. Which
simply means that your original command:
sed_object_re=`echo $object | sed 's/[].[^$\\*|]//g'`
was fully correct, and didn't lack any required escaping (while, on the
first impression, I had *erronously* thought it did).
OK.
But independently of bracket expressions, Posix also specifies that ^
and $ are not special when they are not first or last characters in a
regex, respectively. This is what makes object file names of the form
foo^bar.o and foo$bar.o acceptable. Even ^foo.o, since our sed regex
does not start with $object.
But re-reading what I wrote in my first reply, I must admit it sounded
like I was adding something new to the discussion, while in fact I was
just saying hey, at first I wasn't sure that this change of yours was
correct, but by reading the standards/manuals more carefully, I realized
you are indeed completely right. Miscommunication on my part -- sorry
for the noise and the time I made us loose.
Naah, it's better to be picky and find out we're safe after all, than
the other way round.
Remember the input
file only contains lines of the form
$object: header header...
$object: header \
header header...
While somebody might arguably name their header foo.o, I do think they
deserve what they get.
Agreed. Especially because, to cause a spurious match, they would have to
*also* have a `:' character in the header name, i.e., have an header named
like foo.o:bar.h or foo.obj:bar.h -- in which case they probably do
deserve to suffer a bit.
Right.
@@ -503,7 +503,8 @@ makedepend)
touch $tmpdepfile
${MAKEDEPEND-makedepend} -o$obj_suffix -f$tmpdepfile $@
rm -f $depfile
- cat $tmpdepfile $depfile
+ # makedepend may prepend the VPATH from the source file name to the
object.
I'd add a comment here telling why there's no need to escape $object for
the
sed commmand just below.
I will, after I've understood your remaining comments.
Hope this reply is clearer than the previous ones.
Yup. I'm adding the following.
Thanks,
Ralf
Clarify regex code in depcomp.
* lib/depcomp: Add comment why we don't need regex-escaping here.
Suggested by Stefano Lattarini.
diff --git a/lib/depcomp b/lib/depcomp
index ce9419c..8097c19 100755
--- a/lib/depcomp
+++ b/lib/depcomp
@@ -1,7 +1,7 @@
#! /bin/sh
# depcomp - compile a program generating dependencies as side-effects
-scriptversion=2011-04-08-18; # UTC
+scriptversion=2011-04-09-11; # UTC
# Copyright (C) 1999, 2000, 2003, 2004, 2005, 2006, 2007, 2009, 2011,
# Free Software Foundation, Inc.
@@ -504,6 +504,7 @@ makedepend)
${MAKEDEPEND-makedepend} -o$obj_suffix -f$tmpdepfile $@
rm -f $depfile
# makedepend may prepend the VPATH from the source file name to the object.
+ # No need to regex-escape $object, excess matching of '.' is harmless.
sed s|^.*\($object *:\)|\1| $tmpdepfile $depfile
sed '1,2d' $tmpdepfile | tr ' ' '
' | \
Re: [PATCH] {master} tests: remove useless sleep from tests on remake rules
* Stefano Lattarini wrote on Sun, Apr 03, 2011 at 11:05:54AM CEST: On Sunday 03 April 2011, Ralf Wildenhues wrote: * Stefano Lattarini wrote on Sun, Apr 03, 2011 at 10:23:57AM CEST: The sleeps were there to make generated autotools files strictly newer than their sources; however, this is not necessary, since POSIX mandates that make considers files with the same timestamp of their dependencies to be up-to-date. info Autoconf Timestamps and Make, in git Autoconf, tells you why these are good for HP-UX make. Yes, but the worst that can happen would be that the first $MAKE call is not really a no-op, and some autotools files get rebuilt. Ah ok. Also, the involved tests currently require GNU make anyway. Oh, then my argument is moot. But I'm planning to enhance them to work with non-GNU make too, so this objection is moot. Then not any more. ;-) So, up to you I guess. Thanks, Ralf
Re: [PATCH] {master} coverage: more on java support: EXTRA_ and noinst_ prefixes
* Stefano Lattarini wrote on Sat, Apr 09, 2011 at 01:57:50PM CEST: --- a/tests/java-extra.test +++ b/tests/java-extra.test @@ -26,6 +26,7 @@ set -e cat configure.in 'END' AC_CHECK_PROG([HAS_JAVAC], [javac], [:], [exit]) ($HAS_JAVAC 77); $HAS_JAVAC 77 +AM_CONDITIONAL([COND], [test x$cond = xyes]) AC_OUTPUT END The test passes with this edit. OK. I will push shortly, barring new objections. Thanks, Ralf
Re: [PATCH] {master} coverage: add tests on remake rules in more complex situations
Hello Stefano, Bruno,
nice collaboration work there, thank you!
I have a few nits, mostly really trivial. Feel free to push after
addressing them. --author?
* Stefano Lattarini wrote on Fri, Apr 08, 2011 at 12:54:27PM CEST:
Subject: [PATCH] coverage: add tests on remake rules in more complex
situations
* tests/defs (using_gmake): New function.
(for tool in $required): Use it when $tool is 'GNUmake'.
* tests/remake-moved-m4-file.test: New test.
* tests/remake-deleted-m4-file.test: Likewise.
* tests/remake-renamed-m4-file.test: Likewise.
* tests/remake-renamed-m4-macro-and-file.test: Likewise.
* tests/remake-renamed-m4-macro.test: Likewise.
* tests/remake-add-acsubst-gnulib.test: Likewise.
* tests/remake-add-header-gnulib.test: Likewise.
* tests/remake-remove-header-gnulib.test: Likewise.
* tests/Makefile.am (TESTS): Update.
--- a/tests/defs
+++ b/tests/defs
@@ -146,6 +146,38 @@ AUTOMAKE_fails ()
AUTOMAKE_run 1 ${1+$@}
}
+# using_gmake
+# ---
+# Return success if $MAKE is GNU make, return failure otherwise.
+# Caches the result for speed reasons.
+using_gmake ()
+{
+ case $am__using_gmake in
+yes)
+ return 0
+ ;;
+no)
+ return 1
+ ;;
Would it be ok to merge the ;; to the previous line? My laptop doesn't
have very many vertical lines, and the spacing doesn't add information
here. Same below in this case statement. Thanks.
+'')
+ # Use --version AND -v, because SGI Make doesn't fail on --version.
+ # Also grep for GNU because newer versions of FreeBSD make do
+ # not complain about `--version' (they seem to silently ignore it).
+ if $MAKE --version -v | grep GNU; then
+am__using_gmake=yes
+return 0
+ else
+am__using_gmake=no
+return 1
+ fi
+ ;;
+*)
+ echo invalid value for \$am__using_gmake: '$am__using_gmake' 2
+ Exit 99
+ ;;
+ esac
+}
+
commented_sed_unindent_prog='
/^$/b# Nothing to do for empty lines.
x# Get xindent into pattern space.
@@ -217,11 +249,8 @@ do
etags --version -o /dev/null || exit 77
;;
GNUmake)
- # Use --version AND -v, because SGI Make doesn't fail on --version.
- # Also grep for GNU because newer versions of FreeBSD make do
- # not complain about `--version' (they seem to silently ignore it).
- echo $me: running $MAKE --version -v | grep GNU
- ( $MAKE --version -v | grep GNU ) || exit 77
+ echo $me: determine if $MAKE is GNU make
+ using_gmake || exit 77
;;
gcc)
# When gcc is required, export `CC=gcc' so that ./configure
--- /dev/null
+++ b/tests/remake-deleted-m4-file.test
+# Test remake rules when an m4 file gets removed and the macros it
+# defined gets inlined into the caller. Try with both an indirect
s/gets/get/
+# call and a direct one. This can be seen as testing the deleted
+# header file issue w.r.t. aclocal.m4 dependencies.
add cross-ref to acloca22.test?
+. ./defs || Exit 1
+
+cat configure.in 'END'
+FOO_MACRO
+AC_OUTPUT
+END
+
+cat Makefile.am 'END'
+ACLOCAL_AMFLAGS = -I m4
+.PHONY: test
+test:
+ test '$(the_answer)' = 42
-eq ? More instances below.
+END
+
+macro_value='the_answer=42; AC_SUBST([the_answer])'
+
+mkdir m4
+
+cat m4/foo.m4 'END'
+AC_DEFUN([FOO_MACRO], [BAR_MACRO])
+END
+
+cat m4/bar.m4 END
+AC_DEFUN([BAR_MACRO], [$macro_value])
+END
+
+$ACLOCAL -I m4
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+$MAKE test
+
+$sleep
+
+sed -e s|BAR_MACRO|$macro_value| m4/foo.m4 t
+mv -f t m4/foo.m4
+rm -f m4/bar.m4
+
+using_gmake || $MAKE Makefile
+$MAKE test
+
+$sleep
+
+sed -e s|FOO_MACRO|$macro_value| configure.in t
+mv -f t configure.in
+rm -f m4/foo.m4
+
+using_gmake || $MAKE Makefile
+$MAKE test
--- /dev/null
+++ b/tests/remake-gnulib-add-acsubst.test
+# Test remake rules when a new AC_SUBST'd variable is added, and C header
+# files are involved.
+# This test overlaps with others, and is not strictly necessary per se,
+# but it exercises a real use case (from gnulib, see:
+# http://lists.gnu.org/archive/html/bug-gnulib/2011-04/msg5.html
+# for more info), so keep it anyway.
You could s/,.*// in the last line if you like (more instances below),
as that part is not needed any more once the test is applied. ;-)
+. ./defs || Exit 1
+
+cat configure.in 'END'
+AC_PROG_CC
+MY_MACROS
+AC_OUTPUT
+END
+
+cat Makefile.am 'END'
+ACLOCAL_AMFLAGS = -I m4
+noinst_PROGRAMS = foo
+foo_SOURCES = foo.c
+BUILT_SOURCES = foo.h
+edit_h = sed -e 's|[@]foovar@|@foovar@|g'
+foo.h: foo.in.h
+ $(edit_h) $(srcdir)/foo.in.h $@-t
+ cat $@-t;: for debugging
+ mv -f $@-t $@
+EXTRA_DIST = foo.in.h
+MOSTLYCLEANFILES = foo.h foo.h-t
+END
+
+mkdir m4
+
+cat m4/foo.m4 'END'
+AC_DEFUN([MY_MACROS], [
+ FOO_MACRO
+dnl: ZAP_MACRO
+])
+END
+
Re: bug#8234: check_JAVA built during make
* Stefano Lattarini wrote on Wed, Apr 06, 2011 at 06:52:24PM CEST: OK for maint? I'll push in 72 hours if there is no objection. This is OK with a NEWS entry and the nit below addressed. Thanks! Ralf Subject: [PATCH] java: check_JAVA does not cause compilation by make all anymore Fixes automake bug#8234. * automake.in (handle_java): Make stamp of class files built from java sources in $(check_JAVA) a dependency of `check' target, not `all' target. * tests/java-check.test: New test. * tests/Makefile.am (TESTS): Update. * THANKS: Update. Report from Petteri Räty. --- /dev/null +++ b/tests/java-check.test +# Make sure that check_JAVA causes *.class files to be build only with s/build/built/ +# make check, and not also with make all. +# See automake bug#8234.
Re: Fix makedepend depmode to cope with VPATH builds
Hi Stefano,
* Stefano Lattarini wrote on Fri, Apr 08, 2011 at 01:05:53PM CEST:
Hello Ralf, sorry for the delay.
I don't think you need to apologize for any delays on your behalf,
for at least a few months. ;-)
On Wednesday 06 April 2011, Ralf Wildenhues wrote:
Ugh, a looong-standing depmode=makedepend bug that silently breaks
rebuilds in VPATH trees. It has now hit me in the wild. :-(
Just out of curiosity: with which compiler?
IBM XL on BlueGene.
--- a/lib/depcomp
+++ b/lib/depcomp
@@ -557,7 +557,9 @@ makedepend)
touch $tmpdepfile
${MAKEDEPEND-makedepend} -o$obj_suffix -f$tmpdepfile $@
rm -f $depfile
- cat $tmpdepfile $depfile
+ # makedepend may prepend the VPATH from the source file name to the
object.
+ sed_object_re=`echo $object | sed 's/[].[^$\\*|]//g'`
A couple of questions/observations about the sed command:
1. I wasn't sure whether `.', `$', `^' and `*' are to be escaped when used in
a bracket expression; however, the POSIX standard at:
http://pubs.opengroup.org/onlinepubs/007908799/xbd/re.html
says they shouldn't be. Also, I couldn't find any reference in the
Autoconf manual pointing to a sed implementation that doesn't respect
this part of the standard.
Ah, cool. Then I recon we don't need this escaping step at all.
The only character I was worried about was $, as it can occur in
object names now and then (ok, mostly java, and that doesn't use
depcomp). Any others in the list are pretty much excluded anyway,
because they will evoke shell expansion errors when used in a
makefile unquoted. Except for '.', but I'm not afraid of matching
a bit too broadly here, the final 'o' will take care that we don't
match any header files.
2. I guess the doubled `\\' in the sed command is there to account for the
extra stripping performed by the backtick `` command substitution, right?
But is that truly portable, considering that the doubled `\\' is also
into a single-quotes string? Or would it be better to play safe and use
an indirection like:
sed_quoting_cmd='s/[].[^$\\*|]/\\/g'
sed_object_re=`echo $object | sed -e $sed_quoting_cmd`
WDYT?
This is all moot given above.
+ sed s|^.*\($sed_object_re *:\)|\1| $tmpdepfile $depfile
sed '1,2d' $tmpdepfile | tr ' ' '
' | \
## Some versions of the HPUX 10.20 sed can't process this invocation
--- /dev/null
+++ b/tests/depcomp9.test
+cd build
+../configure am_cv_CC_dependencies_compiler_type=makedepend
+$MAKE || Exit 77
Why skip the test here?
Well, makedepend mode might not actually work for some reason.
It's pretty contrived, granted, but let's say the makedepend script
is so broken that _AM_DEPENDENCIES wouldn't have chosen it because
it would fail one of the tests in m4/depend.m4. Then I don't really
want to see a test failure here.
Wouldn't be better to just let the test fail if $MAKE fails the first
time?
See above.
(Which means the first $MAKE
call and the subsequent $MAKE clean are just to be removed as
redundant).
Ah, no. This sequence is very important. Sorry for not being clear
enough. The error I'm chasing does not happen upon the first make,
only the second one: the first time, the .Po files contain only the
dummy text. During the first build, makedepend fills them, but without
the depcomp fix, it would create dependencies like this:
../../src/foo.o: ../../src/foo.c ../../src/foo.h
Then, the second time, make sees that foo.o is a prerequisite of foo,
sees a '.c.o' inference rule, and the above. GNU make then infers that
'foo.o' should actually be '../../src/foo.o', so the command to compile
foo.c gets to be
source=../../src/foo.c' object='../../src/foo.o' ... depcomp $(CC) ...
instead of the correct
source=../../src/foo.c' object='foo.o' ... depcomp $(CC) ...
and that messes up the depcomp script work, causing the error messages
quoted in my patch.
I'm adding a comment now:
# Do not error out with the first make, as the forced 'makedepend'
# depmode might not actually work, but we have overridden the
# _AM_DEPENDENCIES tests. The actual error only happens the second time
# the objects are built, because 'makedepend' has silently messed up the
# .Po files the first time.
Hope that makes it clearer.
+$MAKE clean
+
+$MAKE out 21 || { cat out; Exit 1; }
+cat out
+$FGREP 'src/.deps' out Exit 1
+
This is probably paranoid but ... what if $(DEPDIR) is != `.deps'?
(e.g., it is `_deps').
Thanks, fixed now.
Below's what I'm pushing to maint.
Thanks for the review,
Ralf
Fix makedepend depmode for VPATH builds.
* lib/depcomp [makedepend]: Remove any VPATH prefix from the
object file name, so a rebuild doesn't attempt to update the
.Po files in the source tree.
* tests/depcomp9.test: New test.
* tests/Makefile.am (TESTS): Update.
* NEWS: Update.
diff --git a/NEWS b/NEWS
index 3132b16..6bd67f6 100644
--- a/NEWS
Fix makedepend depmode to cope with VPATH builds
Ugh, a looong-standing depmode=makedepend bug that silently breaks
rebuilds in VPATH trees. It has now hit me in the wild. :-(
Anyway, this is for maint, and I'd appreciate review, but will
otherwise push soonish.
(I'll probably have to defer any reviewing until the weekend ...)
Thanks,
Ralf
Fix makedepend depmode for VPATH builds.
* lib/depcomp [makedepend]: Remove any VPATH prefix from the
object file name, so a rebuild doesn't attempt to update the
.Po files in the source tree.
* tests/depcomp9.test: New test.
* tests/Makefile.am (TESTS): Update.
* NEWS: Update.
diff --git a/NEWS b/NEWS
index d368eec..1fc539d 100644
--- a/NEWS
+++ b/NEWS
@@ -120,6 +120,8 @@ Bugs fixed in 1.11a:
- The parallel-tests driver now does not produce erroneous results
with Tru64/OSF 5.1 sh upon unreadable log files any more.
+
+ - The makedepend depmode now works better with VPATH builds.
New in 1.11:
diff --git a/lib/depcomp b/lib/depcomp
index c3163be..3740d29 100755
--- a/lib/depcomp
+++ b/lib/depcomp
@@ -1,10 +1,10 @@
#! /bin/sh
# depcomp - compile a program generating dependencies as side-effects
-scriptversion=2010-10-07.20; # UTC
+scriptversion=2011-04-06.20; # UTC
-# Copyright (C) 1999, 2000, 2003, 2004, 2005, 2006, 2007, 2009, 2010
-# Free Software Foundation, Inc.
+# Copyright (C) 1999, 2000, 2003, 2004, 2005, 2006, 2007, 2009, 2010,
+# 2011 Free Software Foundation, Inc.
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -557,7 +557,9 @@ makedepend)
touch $tmpdepfile
${MAKEDEPEND-makedepend} -o$obj_suffix -f$tmpdepfile $@
rm -f $depfile
- cat $tmpdepfile $depfile
+ # makedepend may prepend the VPATH from the source file name to the object.
+ sed_object_re=`echo $object | sed 's/[].[^$\\*|]//g'`
+ sed s|^.*\($sed_object_re *:\)|\1| $tmpdepfile $depfile
sed '1,2d' $tmpdepfile | tr ' ' '
' | \
## Some versions of the HPUX 10.20 sed can't process this invocation
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 7f165ac..880206e 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -371,6 +371,7 @@ depcomp6.test \
depcomp7.test \
depcomp8a.test \
depcomp8b.test \
+depcomp9.test \
depdist.test \
depend.test \
depend2.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index d1e3a73..90f3d6b 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -632,6 +632,7 @@ depcomp6.test \
depcomp7.test \
depcomp8a.test \
depcomp8b.test \
+depcomp9.test \
depdist.test \
depend.test \
depend2.test \
diff --git a/tests/depcomp9.test b/tests/depcomp9.test
new file mode 100755
index 000..c4dd2ed
--- /dev/null
+++ b/tests/depcomp9.test
@@ -0,0 +1,84 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see http://www.gnu.org/licenses/.
+
+# makedepend should work in VPATH mode.
+
+# Here's the bug: makedepend will prefix VPATH to the object file name,
+# thus the second make will invoke depcomp with object='../../src/foo.o',
+# causing errors such as:
+# touch: cannot touch `../../src/.deps/foo.TPo': No such file or directory
+# makedepend: error: cannot open ../../src/.deps/foo.TPo
+# ../../depcomp: line 560: ../../src/.deps/foo.TPo: No such file or directory
+
+# We include subfoo only to be sure that we don't remove too much
+# from the object file name.
+
+required='makedepend'
+. ./defs || Exit 1
+
+mkdir src src/sub build
+
+cat configure.in 'END'
+AC_PROG_CC
+AM_PROG_CC_C_O
+AC_CONFIG_FILES([src/Makefile])
+AC_OUTPUT
+END
+
+cat Makefile.am 'END'
+SUBDIRS = src
+END
+
+cat src/Makefile.am 'END'
+AUTOMAKE_OPTIONS = subdir-objects
+bin_PROGRAMS = foo
+foo_SOURCES = foo.c foo.h sub/subfoo.c
+END
+
+cat src/foo.h EOF
+extern int subfoo (void);
+EOF
+
+cat src/foo.c EOF
+#include foo.h
+int main (void)
+{
+ return subfoo ();
+}
+EOF
+
+cat src/sub/subfoo.c EOF
+#include foo.h
+int subfoo (void)
+{
+ return 0;
+}
+EOF
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+cd build
+../configure am_cv_CC_dependencies_compiler_type=makedepend
+$MAKE || Exit 77
+$MAKE clean
+
+$MAKE out 21 || { cat out; Exit 1; }
+cat out
+$FGREP 'src/.deps' out Exit 1
+
+:
Re: [PATCH] {master} tests: remove useless sleep from tests on remake rules
* Stefano Lattarini wrote on Sun, Apr 03, 2011 at 10:23:57AM CEST: The sleeps were there to make generated autotools files strictly newer than their sources; however, this is not necessary, since POSIX mandates that make considers files with the same timestamp of their dependencies to be up-to-date. info Autoconf Timestamps and Make, in git Autoconf, tells you why these are good for HP-UX make. OK for master? I will push in 72 hours if there is no objection. I'd say let's omit the patch for safety. Thanks, Ralf
Fix locale issue in check-exported-srcdir.test.
I had to merge this patch into maint in order to avoid a test failure when LC_COLLATE=de_DE.UTF-8: + mkdir SrcDir BuildDir + mv BuildDir configure.in depcomp install-sh missing SrcDir SrcDir mv: Verschieben von „SrcDir“ in eigenes Unterverzeichnis („SrcDir/SrcDir“) nicht möglich + exit_status=1 Thanks, Ralf Fix locale issue in check-exported-srcdir.test. * tests/check-exported-srcdir.test: Reformulate glob to not fail in a locale that ignores or interleaves character case. diff --git a/tests/check-exported-srcdir.test b/tests/check-exported-srcdir.test index 9209fc8..430c9cc 100755 --- a/tests/check-exported-srcdir.test +++ b/tests/check-exported-srcdir.test @@ -32,9 +32,9 @@ show_info () fi } -mkdir SrcDir BuildDir - -mv [a-z]* SrcDir +mkdir SrcDir +mv [!S]* SrcDir +mkdir BuildDir cd SrcDir cat configure.in 'END'
