Re: [PATCH] docs: Add references between the 2 sections on compiling Java.
* Stefano Lattarini wrote on Sat, Jul 16, 2011 at 03:18:28PM CEST:
From: Benoit Sigoure tsuna...@gmail.com
Date: Fri, 15 Jul 2011 16:49:45 -0700
Subject: [PATCH] docs: add references between the 2 sections on java support
* doc/automake.texi (Java Support, Java): Add cross-references.
Nice one, thanks.
Cheers,
Ralf
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -6665,8 +6665,10 @@ is as follows:
@cindex Java support
@cindex Support for Java
-Automake includes support for compiled Java, using @command{gcj}, the Java
-front end to the GNU Compiler Collection.
+Automake includes support for natively compiled Java, using @command{gcj},
+the Java front end to the GNU Compiler Collection (preliminary support
+for compiling Java to bytecode using the @command{javac} compiler is
+also present; @pxref{Java}).
Any package including Java code to be compiled must define the output
variable @code{GCJ} in @file{configure.ac}; the variable @code{GCJFLAGS}
@@ -7534,8 +7536,9 @@ libtool, The Libtool Manual}) with the
@code{LTLIBRARIES} primary.
@cindex @code{JAVA} primary, defined
@cindex Primary variable, @code{JAVA}
-Automake provides some minimal support for Java compilation with the
-@code{JAVA} primary.
+Automake provides some minimal support for Java bytecode compilation with
+the @code{JAVA} primary (in addition to the support for compiling Java to
+native machine code; @pxref{Java Support}).
Any @file{.java} files listed in a @code{_JAVA} variable will be
compiled with @code{JAVAC} at build time. By default, @file{.java}
Re: [GSoC] Some patches for testsuite harness improvements and TAP support introduction
* Stefano Lattarini wrote on Fri, Jul 15, 2011 at 12:35:02AM CEST: The patches I'm going to post in this thread have already been applied to the 'GSoC/experimental/test-results-work' temporary branch, but now I think they are mature enough to be moved to the official branch 'test-protocols'; fixlets and improvements can be done with follow-up patches (I even think that this should be preferred to doing further amendaments and tweaking of these patches). Is this FYI or to be reviewed? Thanks, Ralf
Re: [GSoC] Some patches for testsuite harness improvements and TAP support introduction
On Monday 18 July 2011, Ralf Wildenhues wrote: * Stefano Lattarini wrote on Fri, Jul 15, 2011 at 12:35:02AM CEST: The patches I'm going to post in this thread have already been applied to the 'GSoC/experimental/test-results-work' temporary branch, but now I think they are mature enough to be moved to the official branch 'test-protocols'; fixlets and improvements can be done with follow-up patches (I even think that this should be preferred to doing further amendaments and tweaking of these patches). Is this FYI or to be reviewed? To be reviewed. Thanks, and sorry for the confusion, Stefano
Re: [PATCH 5/5] {test-protocols} tap: add experimental TAP-aware driver
On Friday 15 July 2011, Stefano Lattarini wrote:
* doc/automake.texi (Using the TAP test protocol): New section.
(Overview of Custom Test Drivers Support): Minor updates.
* lib/tap-driver: New script, TAP-aware test driver for Automake;
implemented in perl and based on TAP::Parser.
* lib/Makefile.am (dist_script_DATA): Add it.
* tests/tap-autonumber.test: New test.
* tests/tap-bailout.test: Likewise.
* tests/tap-basic.test: Likewise.
* tests/tap-deps.test: Likewise.
* tests/tap-diagnostic.test: Likewise.
* tests/tap-empty.test: Likewise.
* tests/tap-empty-diagnostic.test: Likewise.
* tests/tap-escape-directive.test: Likewise.
* tests/tap-exit.test: Likewise.
* tests/tap-fancy.test: Likewise.
* tests/tap-fancy2.test: Likewise.
* tests/tap-global-result.test: Likewise.
* tests/tap-html.test: Likewise.
* tests/tap-log.test: Likewise.
* tests/tap-merge-stdout-stderr.test: Likewise.
* tests/tap-more.test: Likewise.
* tests/tap-more2.test: Likewise.
* tests/tap-no-merge-stdout-stderr.test: Likewise.
* tests/tap-no-spurious-summary.test: Likewise.
* tests/tap-no-spurious.test: Likewise.
* tests/tap-not-ok-skip.test: Likewise.
* tests/tap-numeric-description.test: Likewise.
* tests/tap-out-of-order.test: Likewise.
* tests/tap-passthrough.test: Likewise.
* tests/tap-plan.test: Likewise.
* tests/tap-plan-errors.test: Likewise.
* tests/tap-plan-corner.test: Likewise.
* tests/tap-realtime.test: Likewise.
* tests/tap-recheck-logs.test: Likewise.
* tests/tap-recheck.test: Likewise.
* tests/tap-skip-whole.test: Likewise.
* tests/tap-summary.test: Likewise.
* tests/tap-todo-skip.test: Likewise.
* tests/tap-todo-skip-together.test: Likewise.
* tests/tap-todo-skip-whitespace.test: Likewise.
* tests/tap-skipall-whitespace.test: Likewise.
* tests/tap-unplanned.test: Likewise.
* tests/tap-whitespace-normalization.test: Likewise.
* tests/tap-with-and-without-number.test: Likewise.
* tests/tap-xfail-tests.test: Likewise.
* tests/tap-bad-prog.test: New xfailing test.
* tests/tap-color.test: Likewise.
* tests/tap-plan-corner2.test: Likewise.
* tests/tap-message-0.test: Likewise.
* tests/tap-signal.test: Likewise.
* tests/Makefile.am (TESTS, XFAIL_TESTS): Update.
---
And consider this squashed in:
-*-*-*-
diff --git a/tests/tap-color.test b/tests/tap-color.test
index 8ed7265..82c0b51 100755
--- a/tests/tap-color.test
+++ b/tests/tap-color.test
@@ -52,7 +52,8 @@ AUTOMAKE_OPTIONS = color-tests
AM_TEST_LOG_DRIVER_FLAGS = --comments
TEST_LOG_COMPILER = cat
TEST_LOG_DRIVER = $(PERL) $(srcdir)/tap-driver
-TESTS = all.test skip.test bailout.test plan-errors.test
+TESTS = all.test skip.test bail.test badplan.test noplan.test \
+few.test many.test order.test
END
cat all.test 'END'
@@ -66,15 +67,41 @@ ok 5 - zardoz # TODO
END
cat skip.test 'END'
-1..0 # Whole script not run
+1..0 # SKIP whole script
END
-cat errors.test 'END'
+cat bail.test 'END'
1..1
ok 1
Bail out!
END
+cat badplan.test 'END'
+# foo
+1..1
+ok 1
+END
+
+cat noplan.test 'END'
+ok 1
+END
+
+cat few.test 'END'
+1..2
+ok 1
+END
+
+cat many.test 'END'
+1..1
+ok 1
+ok 2
+END
+
+cat order.test 'END'
+1..1
+ok 5
+END
+
$ACLOCAL
$AUTOCONF
$AUTOMAKE
@@ -92,9 +119,15 @@ test_color ()
cat stdout | grep ^${blu}SKIP${std}: all\.test 3 - baz # SKIP sk$
cat stdout | grep ^${red}FAIL${std}: all\.test 4 - quux$
cat stdout | grep ^${red}XPASS${std}: all\.test 5 - zardoz # TODO$
- cat stdout | grep ^${blu}SKIP${std}: skip\.test # Whole script not run$
- cat stdout | grep ^${grn}PASS${std}: bailout\.test 1$
- cat stdout | grep ^${mgn}ERROR${std}: all\.test - Bail out!$
+ cat stdout | grep ^${blu}SKIP${std}: skip\.test - whole script$
+ cat stdout | grep ^${grn}PASS${std}: bail\.test 1$
+ cat stdout | grep ^${mgn}ERROR${std}: bail\.test - Bail out!$
+ cat stdout | grep ^${mgn}ERROR${std}: badplan\.test - test plan in middle
of output$
+ cat stdout | grep ^${mgn}ERROR${std}: noplan\.test - missing test plan$
+ cat stdout | grep ^${mgn}ERROR${std}: few.test - too few tests run
(expected 2, got 1)$
+ cat stdout | grep ^${mgn}ERROR${std}: many.test - too many tests run
(expected 1, got 2)$
+ cat stdout | grep ^${mgn}ERROR${std}: many.test 2 # UNPLANNED$
+ cat stdout | grep ^${mgn}ERROR${std}: order.test 5 # OUT-OF-ORDER
(expecting 1)$
# Diagnostic messages shouldn't be colorized.
cat stdout | grep ^# all\.test: Hi! I shouldn't be colorized!$
:
-*-*-*-
Sorry for the noise,
Stefano
Re: [PATCH] {testsuite-work} tests: fix typos, grammaros and other blunders in comments
On Monday 18 July 2011, Ralf Wildenhues wrote: * Stefano Lattarini wrote on Sat, Jul 16, 2011 at 02:30:40PM CEST: A patch doing various (mostly cosmetic) fixes to comments in tests. Before pushing, I'll allow 72 hours for comments and suggestsions. OK thanks. Subject: [PATCH] tests: fix typos, grammaros and other blunders in comments * tests/all.test: In comments, fix typos, grammaros and/or botched capitalization, and/or improve wording. * tests/alpha.test: Likewise. * tests/amopts-variable-expansion.test: Likewise. [...] The GCS now allows you to omit specifying all affected file names in the log entry, and just use something like All affected files changed. For patches such as this one, it makes sense, as there's neither copyright-relevant changes nor is it too important for later analysis whether some file was affected or not. Good to know. I've made this adjustment, and pushed. Thanks, Stefano
Re: [PATCH 1/5] {test-protocols} parallel-tests: make parsing of test results safer
* Stefano Lattarini wrote on Fri, Jul 15, 2011 at 12:36:17AM CEST: The new code for parsing the testsuite-generated `.log' files, as introduced in commit `v1.11-872-gc96b881', considers each `:test-result:' field anywhere in a `.log' file as a declaration of a test result, and accounts for it as such in the testsuite summary. Unfortunately this could easily cause spurious test failures being reported in the testsuite summary. This happened in practice with the Automake's own testsuite; for example: $ make check TESTS='check12-p.test'; echo exit: $? ... PASS: check12-p.test = 4 of 5 tests failed See tests/test-suite.log Please report to bug-autom...@gnu.org = make[2]: *** [test-suite.log] Error 1 make: *** [check-am] Error 2 exit: 2 This change introduces a new special `:test-result:' END, that, when seen, prevents the rest of the log file from being parsed. For more information, refer to the thread: http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00199.html * lib/am/check.am ($(TEST_SUITE_LOG)): Stop the parsing of a log file as soon as the special :test-result:END directive is seen. Related changes and enhancements. * lib/test-driver: Protect the rest of the log after the result lined with a :test-result:END directive. * tests/parallel-tests-no-spurious-summary.test: New test. * tests/test-driver-end-test-results.test: Likewise. * tests/Makefile.am (TESTS): Update. I'm still not sold on this. It is not as robust as a test protocol could be; also I haven't seen this approach being used in any other test suite environments. Whether some line is considered having a result or not depends on (possibly far-away) context, on whether aggregation with other results has happened. When passed through email, misquoting can change not only the interpretation of the misquoted text (which could be expected) but possibly also of later, correctly quoted (or not quoted) text. Please, this is really important: we need to research the other test protocols, what they do to be robust here. Don't NIH here, because the experience we have is not enough to not mess up this. Consider this research as part of the work needed for the SoC assignment, it is a very important part (as it is not easily corrected once released). Thanks, Ralf
Re: [PATCH 2/5] {test-protocols} parallel-tests: new recognized test result 'ERROR'
* Stefano Lattarini wrote on Fri, Jul 15, 2011 at 12:37:07AM CEST:
* lib/am/check.am ($(TEST_SUITE_LOG)): Recognize a new test result
`ERROR'. Use it when encountering unreadable test logs (previously
a simple `FAIL' was used in this situations).
* lib/test-driver: Set the global test result to `ERROR' when the
test exit status is 99. When doing colorized output, color `ERROR'
results in magenta.
* doc/automake.texi (Log files generation and test results
recording): Update by also listing `ERROR' among the list of valid
`:test-results:' arguments.
* NEWS: Update.
* tests/trivial-test-driver: Update.
* tests/parallel-tests.test: Likewise.
* tests/parallel-tests-harderror.test: Likewise.
* tests/parallel-tests-no-spurious-summary.test: Likewise.
* tests/test-driver-global-log.test: Likewise.
* tests/test-driver-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck2.test: Likewise.
* tests/test-driver-custom-multitest.test: Likewise.
* tests/test-driver-custom-no-html.test: Likewise.
* tests/test-driver-end-test-results.test: Likewise.
* tests/color.test: Likewise. Also, make stricter, and also test
from VPATH.
* tests/color2.test: Likewise, and improve syncing with color.test.
* tests/parallel-tests-exit-statuses.test: New test.
* tests/parallel-tests-console-output.test: Likewise.
* tests/Makefile.am (TESTS): Update.
--- a/NEWS
+++ b/NEWS
@@ -13,6 +13,10 @@ New in 1.11a:
* Changes to Automake-generated testsuite harnesses:
+ - The parallel-tests harness has a new test result 'ERROR', that can be
+used to signal e.g., unexpected or internal errors, or failures to set
+up test case scenarios.
The NEWS entry could be a bit more precise in that the ERROR state was
actually used before already with exit status 99, just that it is named
ERROR now. (Please check whether the semantics of 99 already were in a
stable release, otherwise this NEWS entry could of course advertise it
as new.)
A description of ERROR semantics needs to be part of automake.texi as
well.
Also, are you going to followup with Autoconf to rename Autotest's hard
failure state ERROR as well? We should agree on common naming and
semantics, so the two systems are not harder to learn than necessary.
I'm OK with this change once these issues are resolved.
As a minor detail however, please remove the '' greps, I've already
mentioned why I consider '=' in the output not a good idea.
Thanks,
Ralf
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -9248,10 +9248,10 @@ leading whitespace will not be ignored.
@c Keep this in sync with lib/am/check-am:$(TEST_SUITE_LOG).
The only recognized test results are currently @code{PASS}, @code{XFAIL},
-@code{SKIP}, @code{FAIL} and @code{XPASS}. These results, when declared
-with @code{:test-result:}, can be optionally followed by text holding
-the name and/or a brief description of the corresponding test; the
-@option{parallel-tests} harness will ignore such extra text when
+@code{SKIP}, @code{FAIL}, @code{XPASS} and @code{ERROR}. These results,
+when declared with @code{:test-result:}, can be optionally followed by
+text holding the name and/or a brief description of the corresponding
+test; the @option{parallel-tests} harness will ignore such extra text when
generating @file{test-suite.log} and preparing the testsuite summary.
Also, @code{:test-result:} can be used with a special ``pseudo-result''
@code{END}, that will instruct the testsuite harness to stop scanning
Re: [PATCH 4/5] {test-protocols} tests defs: new auxiliary function 'count_test_results'
* Stefano Lattarini wrote on Fri, Jul 15, 2011 at 12:38:57AM CEST: * tests/defs (count_test_results): New function. * tests/check11.test: Use it. * tests/test-driver-custom-multitest.test: Likewise. * tests/test-driver-custom-multitest-recheck.test: Likewise. * tests/test-driver-custom-multitest-recheck2.test: Likewise. * tests/parallel-tests-log-override-recheck.test: Likewise. * tests/parallel-tests-log-override-recheck.test: Likewise. * tests/parallel-tests-no-spurious-summary.test: Likewise, and slightly improve debugging output. * tests/parallel-tests.test: Make use of `count_test_results'. Also, make grepping of make check output slightly stricter * tests/parallel-tests9.test: Likewise. * tests/parallel-tests-log-override-2.test: Likewise, and throw in a small optimization. OK once dependent changes are. Thanks, Ralf
Re: [PATCH 7/5] tap: some preparatory refactoring (2)
* Stefano Lattarini wrote on Mon, Jul 18, 2011 at 10:28:33AM CEST: * lib/tap-driver (console_output): Renamed ... (report): ... to this, and extended to appropriately register the test results when needed. (testsuite_error, handle_tap_comment, handle_tap_test, handle_tap_plan): Adjusted accordingly. OK once dependent changes are. Thanks, Ralf
Re: [PATCH 5/5] {test-protocols} tap: add experimental TAP-aware driver
* Stefano Lattarini wrote on Mon, Jul 18, 2011 at 10:17:01AM CEST:
On Friday 15 July 2011, Stefano Lattarini wrote:
* doc/automake.texi (Using the TAP test protocol): New section.
(Overview of Custom Test Drivers Support): Minor updates.
* lib/tap-driver: New script, TAP-aware test driver for Automake;
implemented in perl and based on TAP::Parser.
* lib/Makefile.am (dist_script_DATA): Add it.
[...]
And consider this squashed in:
The squash-in is OK (but I'm not done with the original patch yet).
Thanks,
Ralf
--- a/tests/tap-color.test
+++ b/tests/tap-color.test
@@ -52,7 +52,8 @@ AUTOMAKE_OPTIONS = color-tests
AM_TEST_LOG_DRIVER_FLAGS = --comments
TEST_LOG_COMPILER = cat
TEST_LOG_DRIVER = $(PERL) $(srcdir)/tap-driver
-TESTS = all.test skip.test bailout.test plan-errors.test
+TESTS = all.test skip.test bail.test badplan.test noplan.test \
+few.test many.test order.test
END
cat all.test 'END'
@@ -66,15 +67,41 @@ ok 5 - zardoz # TODO
END
cat skip.test 'END'
-1..0 # Whole script not run
+1..0 # SKIP whole script
END
-cat errors.test 'END'
+cat bail.test 'END'
1..1
ok 1
Bail out!
END
+cat badplan.test 'END'
+# foo
+1..1
+ok 1
+END
+
+cat noplan.test 'END'
+ok 1
+END
+
+cat few.test 'END'
+1..2
+ok 1
+END
+
+cat many.test 'END'
+1..1
+ok 1
+ok 2
+END
+
+cat order.test 'END'
+1..1
+ok 5
+END
+
$ACLOCAL
$AUTOCONF
$AUTOMAKE
@@ -92,9 +119,15 @@ test_color ()
cat stdout | grep ^${blu}SKIP${std}: all\.test 3 - baz # SKIP sk$
cat stdout | grep ^${red}FAIL${std}: all\.test 4 - quux$
cat stdout | grep ^${red}XPASS${std}: all\.test 5 - zardoz # TODO$
- cat stdout | grep ^${blu}SKIP${std}: skip\.test # Whole script not run$
- cat stdout | grep ^${grn}PASS${std}: bailout\.test 1$
- cat stdout | grep ^${mgn}ERROR${std}: all\.test - Bail out!$
+ cat stdout | grep ^${blu}SKIP${std}: skip\.test - whole script$
+ cat stdout | grep ^${grn}PASS${std}: bail\.test 1$
+ cat stdout | grep ^${mgn}ERROR${std}: bail\.test - Bail out!$
+ cat stdout | grep ^${mgn}ERROR${std}: badplan\.test - test plan in middle
of output$
+ cat stdout | grep ^${mgn}ERROR${std}: noplan\.test - missing test plan$
+ cat stdout | grep ^${mgn}ERROR${std}: few.test - too few tests run
(expected 2, got 1)$
+ cat stdout | grep ^${mgn}ERROR${std}: many.test - too many tests run
(expected 1, got 2)$
+ cat stdout | grep ^${mgn}ERROR${std}: many.test 2 # UNPLANNED$
+ cat stdout | grep ^${mgn}ERROR${std}: order.test 5 # OUT-OF-ORDER
(expecting 1)$
# Diagnostic messages shouldn't be colorized.
cat stdout | grep ^# all\.test: Hi! I shouldn't be colorized!$
Re: [PATCH 8/5] tap: support colorization of testsuite progress output
* Stefano Lattarini wrote on Mon, Jul 18, 2011 at 10:30:56AM CEST:
* lib/tap-driver (%COLORS): New variable (definition extracted
from `lib/am/check.am:$(am__tty_colors)', with some obvious
adjustments.
(report): Adjust to colorize console output when required,
using ...
(decorate_result): ... this new function.
(colored): New function, used by the one above.
* tests/tap-summary.test: Also run the checks when `color-tests'
is in use.
* tests/Makefile.am (XFAIL_TESTS): Remove `tap-color.test'.
OK once dependent changes are in, and with nits below addressed.
Thanks,
Ralf
--- a/lib/tap-driver
+++ b/lib/tap-driver
+# Keep this in sync with `lib/am/check.am:$(am__tty_colors)'.
+my %COLOR = (
+ red = '[0;31m',
I'm sure perl has a way to encode ESC without a literal ESC.
+ grn = '[0;32m',
+ lgn = '[1;32m',
+ blu = '[1;34m',
+ mgn = '[0;35m',
+ brg = '[1m',
+ std = '[m',
+);
@@ -211,17 +222,39 @@ sub stringify_test_result ($)
+sub decorate_result ($)
+{
+ return $_[0] unless $cfg{color-tests};
+ # Best way to simulate a 'switch' construct here.
Please don't, that only obfuscates the code. automake.in uses long
if ... else lists. If you don't like that, use a map.
+ for (@_)
+{
+ $_ eq ERROR and return colored ('mgn', $_);
+ $_ eq PASS and return colored ('grn', $_);
+ $_ eq XPASS and return colored ('red', $_);
+ $_ eq FAIL and return colored ('red', $_);
+ $_ eq XFAIL and return colored ('lgn', $_);
+ $_ eq SKIP and return colored ('blu', $_);
+ return $_; # Don't colorize unknown stuff.
+}
+}
+
sub report ($;$)
{
my ($msg, $result, $explanation) = (undef, @_);
if ($result =~ /^(?:X?(?:PASS|FAIL)|SKIP|ERROR)/)
{
- $msg = $result: $test_script_name;
+ $msg = : $test_script_name;
add_test_result $result;
}
elsif ($result eq #)
{
- $msg = # $test_script_name:;
+ $msg = $test_script_name:;
}
else
{
@@ -229,10 +262,11 @@ sub report ($;$)
}
$msg .= $explanation if defined $explanation;
$msg .= \n;
- print OLDOUT $msg;
+ # Output on console might be colorized.
+ print OLDOUT decorate_result ($result) . $msg;
# Log the result in the log file too, to help debugging (this is
# especially true when said result is a TAP error or Bail out!).
- print $msg;
+ print $result . $msg;
}
sub testuite_error ($)
[...]
