Re: [PATCH v4 3/3] parallel-tests: allow each test to have multiple results

2011-06-16 Thread Ralf Wildenhues 
* Ralf Wildenhues wrote on Fri, Jun 17, 2011 at 08:37:59AM CEST:
> * Stefano Lattarini wrote on Thu, Jun 16, 2011 at 10:03:59AM CEST:
> > The implementation makes use of a custom reStructuredText field
> > `:am-testcase-result:'.
> 
> Why an Automake-specific name here?  This is user facing, it should
> be ':testcase-result:'.  But 'testcase' is not a word, at least not
> a good one.  Please check what other test suite codes use, but I'd
> just use ':test-result:' or ':result:' (if that is not ambiguous).
> 
> Other third-party code might want to make use of this too.

You might want to check how other test suite drivers call this
(dejagnu for example).  No need to invent new names for existing
concepts.

Re: [PATCH v4 3/3] parallel-tests: allow each test to have multiple results

2011-06-16 Thread Ralf Wildenhues 
* Stefano Lattarini wrote on Thu, Jun 16, 2011 at 10:03:59AM CEST:
> With this change, we improve the code creating the `test-suite.log'
> global log and the console testsuite summary to make it   able to
> grasp multiple results per test script.  This is required in order
> to introduce the planned support for test protocols, like TAP and
> SubUnit, which can run multiple testcases per test script.
> The implementation makes use of a custom reStructuredText field
> `:am-testcase-result:'.

Why an Automake-specific name here?  This is user facing, it should
be ':testcase-result:'.  But 'testcase' is not a word, at least not
a good one.  Please check what other test suite codes use, but I'd
just use ':test-result:' or ':result:' (if that is not ambiguous).

Other third-party code might want to make use of this too.

Actually, why not s/testcase/test/g globally in all your text.

I haven't gone over this in detail yet, so just a few random remarks
below.

And by the way, thanks for all your work on this so far!

Cheers,
Ralf

> * lib/check.am ($(TEST_SUITE_LOG)): When processing .log files,
> recognize a testcase result report only if it is declared with
> the custom `:am-testcase-result:' reStructuredText field placed
> at the beginning of a line.  Extend and add explanatory comments.
> (recheck, recheck-html): Add explanatory comments.
> * lib/pt-driver: Write an appropriate `:am-testcase-result:'
> reStructuredText field in the generated log file.  Use a
> reStructuredText transition to better separate the test outcome
> report from the test registered output.  Improve comments.
> * tests/test-driver-custom-xfail-tests.test: Adapt.
> * tests/parallel-tests-empty-testlogs.test: New test.
> * tests/parallel-tests-recheck-override.test: Likewise.
> * tests/parallel-tests2.test: Extend and keep more in-sync with ...
> * tests/test-driver-custom-html.test: ... this new related test.
> * tests/test-driver-custom-no-html.test: New test.
> * tests/test-driver-custom-multitest.test: Likewise.
> * tests/test-driver-custom-multitest-recheck.test: Likewise.
> * tests/test-driver-custom-multitest-recheck2.test: Likewise.
> * tests/ostp-driver: New file, used by the last four tests above.

tests/simple-driver, or maybe trivial-test-driver (as "simple driver" is
already used in the manual for something different) or something.  ostp
is just confusing, and you've argued against 8+3 conventions all the
time.  ;->

> * tests/Makefile.am (TESTS): Update.
> (EXTRA_DIST): Distribute `ostp-driver'.
> (test-driver-custom-multitest.log): Depend on `ostp-driver'.
> (test-driver-custom-multitest-recheck.log): Likewise.
> (test-driver-custom-multitest-recheck2.log): Likewise.
> (test-driver-custom-html.log): Likewise.
> * doc/automake.texi (API for Custom Test Drivers): Update (still
> in Texinfo comments only).

I don't like TODO and comments stuff in finished patches documentation
if we can help it (and here, we surely can).

> --- a/doc/automake.texi
> +++ b/doc/automake.texi
> @@ -9052,8 +9052,9 @@ if the exact interpretation of the associated semantics 
> can change
>  between a test driver and another, and even be a no-op in some drivers).
>  
>  @b{TODO!}  @i{Options and flags that the driver must handle.  Generation
> -of ``.log'' files.  Console output the driver is expected to produce.
> -Support for colored output, XFAIL_TESTS, and DISABLE_HARD_ERRORS}
> +of ``.log'' files, and format they must obey.  Console output the driver

@file{.log}

You should almost never need to use ``...'' in texinfo, at least not for
any entity that has a meaning in code.

> +is expected to produce.  Support for colored output, XFAIL_TESTS, and
> +DISABLE_HARD_ERRORS.}
>  
>  @c
>  @c The driver script should follow a simple protocol in order to really
> @@ -9075,6 +9076,29 @@ Support for colored output, XFAIL_TESTS, and 
> DISABLE_HARD_ERRORS}
>  @cdriver can use temporary files if it needs to, only it should clean
>  @cthem up properly).
>  @c
> +@c  * The result of each testcase run by a test script/program *must*
> +@cbe registered in the test log using a custom reStructuredText
> +@cfield ``am-testcase-result''.  For example, if the test script

See above.

> +@cexecutes two test cases, one successful and one failing, and skip
> +@canother test case, the driver should end up writing the following
> +@cin the test log:
> +@c  :am-testcase-result: PASS [passed testcase name or details]
> +@c  :am-testcase-result: FAIL [failed testcase name or details]
> +@c  :am-testcase-result: SKIP [skipped testcase name or details]

is the [...] literal or metasyntactic?  Please use proper notation here.

> +@cThe above lines (each of which *must* be followed by a blank line
> +@cin order for the HTML output generation to work) are used only
> +@cwhen generating the `test-suite.log' from the individual test

See above.

> +@clogs, and can be placed in any order and position within the logs
> +@

Re: [PATCH v4 1/3] parallel-tests: add auxiliary script 'pt-driver', refactor

2011-06-16 Thread Ralf Wildenhues 
* Stefano Lattarini wrote on Thu, Jun 16, 2011 at 10:00:31AM CEST:
> This refactoring should cause no API of functionality change,
> and is meant only to simplify the future implementation of TAP
> and SubUnit testsuite drivers.  More precisely, our roadmap is
> to move most of the "testsuite driving" features out of the
> Automake-generated Makefiles, and into external scripts with
> well-defined interfaces.  This will allow the user to define
> its own personalized testsuite drivers, and will also offer us
> a framework upon which to implement our new TAP and SubUnit
> drivers, all in a very unobtrusive way and retaining an high
> degree of code reuse and backward-compatibility.

I generally like the direction this is taking.  The point of best
separation between which code goes into Makefile.in and which into
the driver scripts can be fine-tuned when we have more than one such
script.

Actually, yes, before deciding on this for real I really do want to see
a nontrivial other driver script.  There is no point in hardcoding
too much in several driver scripts if it all needs to be the same
anyway.

Please measure the time overhead your changes introduce into the current
code, for a trivial testsuite (say, 50 tests running 'true'), and a
nontrivial one like Automake's and one with faster tests.  Thanks.

I've already mentioned that I don't like the name of the script.  ;-)
(You could basically replace 'pt' everywhere else with 'test', too,
as the non-parallel test setup stuff does not allow for a driver either,
but that is not something we announce in all our names either.)

> * lib/pt-driver: New auxiliary script.
> * lib/Makefile.am (dist_SCRIPT_DATA): Add it.
> * automake.in (handle_tests): Require the new auxiliary script
> `pt-driver', and define new makefile variable `$(am__pt_driver)',

So this could just be $(am__test_driver).

> used to call it.  Perform new substitution on `DRIVER' when
> processing the `check2.am' file.
> * lib/check.am (am__tty_colors): Define new shell variable
> `$am__color_tests'.
> (am__rst_section): Removed, its role taken over by the new
> `pt-driver' script.
> (am__test_driver_flags): New variable, contains the command
> line options passed to `pt-driver'.
> (am__check_pre): Do not deal with temporary files and exit
> traps anymore, as the `pt-driver' script takes care of that now.
> Define shell variable `$am__enable_hard_errors', used by
> `$(am__test_driver_flags)'.  Reorder so that we don't need to
> save and restore the value of the `TERM' environment variable
> anymore.
> Other related adjustments.
> (am__check_post): Remove, as its role has been completely taken
> over by the `pt-driver' script.
> * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
> ?!GENERIC?%OBJ%): Call the test script through the Automake
> substituted `%DRIVER%', and honor the command-line options
> in `$(am__test_driver_flags)'.  Do not call the obsoleted
> `$(am__check_post)' anymore.
> * tests/check.test: Adjust.
> * tests/check2.test : Likewise.
> * tests/check3.test : Likewise.
> * tests/check4.test : Likewise.
> * tests/check10.test: Likewise.
> * tests/color.test: Likewise.
> * tests/color2.test: Likewise.
> * tests/comment9.test: Likewise.
> * tests/dejagnu.test: Likewise.
> * tests/exeext4.test: Likewise.
> * tests/maken3.test: Likewise.
> * tests/maken4.test: Likewise.
> * tests/parallel-tests-interrupt.test: Likewise.
> * tests/posixsubst-tests.test: Likewise.
> * tests/repeated-options.test: Likewise.
> * tests/check-no-pt-driver.test: New test.
> * tests/parallel-tests-pt-driver.test: Likewise.
> * tests/Makefile.am (TESTS): Update.
> * NEWS: Update.


> index 054c62d..f3116c8 100644
> --- a/lib/am/check2.am
> +++ b/lib/am/check2.am

> @@ -17,7 +17,9 @@
>  ## From a test file to a log file.
>  ?GENERIC?%EXT%.log:
>  ?!GENERIC?%OBJ%: %SOURCE%
> - @p='%SOURCE%'; $(am__check_pre) %COMPILE% "$$tst" $(am__check_post)
> + @p='%SOURCE%'; $(am__check_pre) \
> + %DRIVER% $(am__test_driver_flags) -- \
> + %COMPILE% "$$tst"

Erm, was there not a need to keep post flags for redirection?  I must
confess I haven't gone through all gnulib threads from the last weeks
yet, but I vaguely remember related issues coming up there.

> --- /dev/null
> +++ b/lib/pt-driver
> @@ -0,0 +1,129 @@
> +#! /bin/sh
> +# pt-driver - basic driver script for the `parallel-tests' mode.

> +# Make unconditional expansion of undefined variables an error.  This
> +# helps a lot in preventing typo-related bugs.
> +set -u

Not sure how older shells handle this, but for development it should be
ok.  We haven't used it elsewhere.

> +print_usage ()
> +{
> +  cat < +Usage:
> +  pt-driver [--help|--version] --test-name=NAME --log-file=PATH
> +  [--expect-failure={yes|no}] [--color-tests={yes|no}]
> +  [--enable-hard-errors={yes|no}] [--] TEST-SCRIPT
> +The \`--test-name' and \`--log-file' options are mandatory.
> +END

echo "\
Usage ...

"?

(Old shells will create (and sometimes fo

Re: [PATCH] check: don't use multi-line coloring for the report

2011-06-16 Thread Ralf Wildenhues 
Hello,

* Bert Wesarg wrote on Thu, Jun 16, 2011 at 08:19:23PM CEST:
> the parallel part is a little trickier. Because the line printing is
> done by awk. I would like to know, whether it is portable to use the
> printf function of awk. It is POSIX, but you may know that this
> doesn't count much.

True.  In case of doubt, try Solaris awk, that's fairly old.

> I couldn't find any prior usage in automake and
> autoconf either. Nor does the autoconf manual list printf as a
> functions in the traditional awk.

Then I would prefer to do without.  I don't see why you would need it
for your change at all, however.  All the coloring is done from echo
statements even in the parallel case, and you can put the resets in
there as well, methinks.

Thanks,
Ralf

Re: [PATCH] {maint} Deprecate de-ansification support, it should go away in automake 1.12

2011-06-16 Thread Stefano Lattarini 
On Thursday 16 June 2011, Stefano Lattarini wrote:
> Hi Bob.
> 
> On Thursday 16 June 2011, Bob Friesenhahn wrote:
> > On Wed, 15 Jun 2011, Stefano Lattarini wrote:
> > >>
> > > Problem: it seems that there are few (and luckily only few) active 
> > > packages
> > > that still use the `ansi2knr' option:
> > > 1. The latest version (8c) of the libjpeg implementation from IJG 
> > > (Independent
> > >JPEG Group, ); see the `configure.ac' file from 
> > > the
> > >archive 
> > 
> > I created most of the libjpeg build configury.  The official 
> > maintainer (Guido Vollbeding) added the extra AM_INIT_AUTOMAKE 
> > options.
> > 
> > > So, maybe it would be worthwhile to directly inform the maintainers of the
> > > packages mentioned above that still use de-ANSI-fication feature about its
> > > oncoming deprecation and removal.  WDYT?
> > 
> > If they do not hear about it, I am sure that they will learn once the 
> > option stops working.
> > 
> OK, then I'll save myself the trouble.
> 
> Thanks,
>   Stefano
> 
Well, we've informed those projects anyway in the end; hope that it'll
be helpful.

Regards,
  Stefano

Re: [PATCH] check: don't use multi-line coloring for the report

2011-06-16 Thread Bert Wesarg 
Hi,

the parallel part is a little trickier. Because the line printing is
done by awk. I would like to know, whether it is portable to use the
printf function of awk. It is POSIX, but you may know that this
doesn't count much. I couldn't find any prior usage in automake and
autoconf either. Nor does the autoconf manual list printf as a
functions in the traditional awk.

Bert

[FYI] maint merged into various branches

2011-06-16 Thread Stefano Lattarini 
I've merged maint into the following branches:
 - master
 - java-work
 - parallel-tests-maint
 - python-work
 - yacc-work

Then I've merged master into testsuite-work, yacc-work into
ylwrap-refactor, and pushed.

Regards,
  Stefano

Re: [PATCH] check: don't use multi-line coloring for the report

2011-06-16 Thread Bert Wesarg 
On Thu, Jun 16, 2011 at 09:41, Stefano Lattarini
 wrote:
> I verified that the problem you reported is really present, and
> I agree it would be nice to fix it.
>
> However, your patch fixes the problem only fot the old "serial"
> testsuite harness, not for the new "parallel" one.  It would be
> nice to fix the problem for both (especially because most
> Automake-using GNU projects employs the new "parallel" harness
> now).  Are you interested in improving your patch to do so?  If
> yes, note that ideally you should also add yourself to the
> THANKS file, and write a ChangeLog entry describing the problem
> you've fixed and the changes you made to the files (if you don't,
> I can write it for you before pushing the patch).

The first part is no problem, but maybe you must iterate over my
ChangeLog entry.

Bert

>
> Regards,
>  Stefano
>

Re: [PATCH] check: don't use multi-line coloring for the report

2011-06-16 Thread Stefano Lattarini 
On Thursday 16 June 2011, Bert Wesarg wrote:
> On Thu, Jun 16, 2011 at 09:41, Stefano Lattarini
>  wrote:
> > I verified that the problem you reported is really present, and
> > I agree it would be nice to fix it.
> >
> > However, your patch fixes the problem only fot the old "serial"
> > testsuite harness, not for the new "parallel" one.  It would be
> > nice to fix the problem for both (especially because most
> > Automake-using GNU projects employs the new "parallel" harness
> > now).  Are you interested in improving your patch to do so?  If
> > yes, note that ideally you should also add yourself to the
> > THANKS file, and write a ChangeLog entry describing the problem
> > you've fixed and the changes you made to the files (if you don't,
> > I can write it for you before pushing the patch).
> 
> The first part is no problem, but maybe you must iterate over my
> ChangeLog entry.
>
Don't worry about that, nobody is expected to write a GCS-compliant
ChangeLog entry the first time(s) ;-)  In the worst case, I'll just
rewrite it on your behalf before pushing.

Thanks again,
  Stefano

[FYI] {testsuite-work} tests: make test 'self-check-reexec.test' more portable

2011-06-16 Thread Stefano Lattarini 
* tests/self-check-reexec.test: Rewrite not to require a Korn
Shell able to grok variable expansions such as `${.sh.version}';
Solaris 10 /bin/ksh fails on this for example.  Instead, just
require bash and a shell that is not bash.
---
 ChangeLog|8 ++
 tests/self-check-reexec.test |   52 +-
 2 files changed, 39 insertions(+), 21 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index cce35e5..399e855 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2011-06-16  Stefano Lattarini  
+
+   tests: make test 'self-check-reexec.test' more portable
+   * tests/self-check-reexec.test: Rewrite not to require a Korn
+   Shell able to grok variable expansions such as `${.sh.version}';
+   Solaris 10 /bin/ksh fails on this for example.  Instead, just
+   require bash and a shell that is not bash.
+
 2011-06-14  Stefano Lattarini  
Jim Meyering  
 
diff --git a/tests/self-check-reexec.test b/tests/self-check-reexec.test
index a35c143..2a0fb33 100755
--- a/tests/self-check-reexec.test
+++ b/tests/self-check-reexec.test
@@ -18,47 +18,57 @@
 # Check that automatic re-execution of test script with the
 # configure-time $SHELL.
 
+required=bash
 . ./defs || Exit 1
 
+unset AM_TESTS_REEXEC BASH_VERSION || :
+
 # This is not optimal, but it's much better than writing wrapper
 # scripts acting as "fake" shells.
-sh -c ': ${.sh.version}' \
-  && skip_ "sh is a Korn Shell"
-ksh -c 'test -n "${.sh.version}"' || \
-  skip_ "ksh is not a fully-fledged Korn Shell"
+
+bash -c 'test -n "$BASH_VERSION"' || skip_ "bash shell not found"
+
+for shell in /bin/sh /bin/ksh sh ksh dash pdksh __no_shell__; do
+  test $shell = __no_shell__ && skip_ "can't find any non-bash shell"
+  $shell -c 'test -n "$BASH_VERSION"' && continue
+  break
+done
+echo "shell='$shell'"
 
 cwd=`pwd` || Exit 99
 
 cp ../defs .
-unset AM_TESTS_REEXEC || :
 
 #
 # Check how to default, force or prevent a re-execution.
 #
 
-cat > run-with-ksh.test <<'END'
+cat > run-with-bash.test <<'END'
 #!/bin/false
 . ./defs
-# Subshell required to prevent some shells (e.g., older bash) from
-# only complaining on stderr but then exiting with exit status 0.
-(test -n "${.sh.version}" && echo ${.sh.version})
+# Ensure that the script gets re-executed with bash.  Also ensure that
+# non-standard syntax used after the inclusion of `./defs' doesn't cause
+# non-bash shells to fail.
+# Subshell required to prevent some shells (e.g., Solaris 10 /bin/sh)
+# from only complaining on stderr but then exiting with exit status 0.
+(foo=abac && test xbxc = ${foo//a/x} && test -n "$BASH_VERSION")
 END
 
 sed -e "s|^testbuilddir=.*|testbuilddir='$cwd'|" \
--e 's|^SHELL=.*$|SHELL=ksh; export SHELL|' \
+-e 's|^SHELL=.*$|SHELL=bash; export SHELL|' \
 < ../defs-static >defs-static
 
-sh -x run-with-ksh.test
+$shell -x run-with-bash.test
 
-AM_TESTS_REEXEC=''sh run-with-ksh.test
-AM_TESTS_REEXEC=yes   sh run-with-ksh.test
-AM_TESTS_REEXEC=y sh run-with-ksh.test
-AM_TESTS_REEXEC=true  sh run-with-ksh.test
-AM_TESTS_REEXEC=1 sh run-with-ksh.test
-AM_TESTS_REEXEC=nosh run-with-ksh.test && Exit 1
-AM_TESTS_REEXEC=n sh run-with-ksh.test && Exit 1
-AM_TESTS_REEXEC=false sh run-with-ksh.test && Exit 1
-AM_TESTS_REEXEC=0 sh run-with-ksh.test && Exit 1
+AM_TESTS_REEXEC=''$shell run-with-bash.test
+AM_TESTS_REEXEC=yes   $shell run-with-bash.test
+AM_TESTS_REEXEC=y $shell run-with-bash.test
+AM_TESTS_REEXEC=true  $shell run-with-bash.test
+AM_TESTS_REEXEC=1 $shell run-with-bash.test
+AM_TESTS_REEXEC=no$shell run-with-bash.test && Exit 1
+AM_TESTS_REEXEC=n $shell run-with-bash.test && Exit 1
+AM_TESTS_REEXEC=false $shell run-with-bash.test && Exit 1
+AM_TESTS_REEXEC=0 $shell run-with-bash.test && Exit 1
 
 #
 # Check message about the re-execution.
@@ -80,7 +90,7 @@ sed -e "s|^testbuilddir=.*|testbuilddir='$cwd/sub'|" \
 
 ./dummy.test a b | grep "^dummy: exec $SHELL \\./dummy\\.test a b$"
 
-for am_sh in sh ksh; do
+for am_sh in $shell bash; do
   $am_sh dummy.test a b c \
 | grep "^dummy: exec $SHELL dummy\\.test a b c$"
   $am_sh ./dummy.test a b c \
-- 
1.7.2.3

Re: [PATCH] {testsuite-work} test defs: fix ksh-related portability bug in warning messages

2011-06-16 Thread Stefano Lattarini 
On Tuesday 14 June 2011, Stefano Lattarini wrote:
> [adding automake-patches, dropping bug-coreutils and bug-autoconf]
> 
> On Tuesday 14 June 2011, Jim Meyering wrote:
> > 
> > From 67811abfda04dafdde114a50556f0e1d2e9b6270 Mon Sep 17 00:00:00 2001
> > From: Jim Meyering 
> > Date: Tue, 14 Jun 2011 09:59:14 +0200
> > Subject: [PATCH] tests: accommodate HP-UX and Solaris shells
> > 
> > [SNIP]
> > 
> Thanks, I'll soonish apply a similar patch (see attachement) for Automake's
> `tests/defs' if there is no objection.
> 
> Regards,
>   Stefano
> 
I've pushed the patch now.

Regards,
  Stefano

Re: bug#8846: coreutils-8.12 on HP-UX 11.31: 3 of 365 tests failed

2011-06-16 Thread Stefano Lattarini 
On Tuesday 14 June 2011, Stefano Lattarini wrote:
> [adding automake-patches on CC:]
> 
> Reference to original thread(s), mostly duplicated:
>  
>  
> 
> Reference to last relevant message there:
>  
>  
>  
> On Tuesday 14 June 2011, Eric Blake wrote:
> > On 06/13/2011 04:29 PM, Stefano Lattarini wrote:
> > > If this work, then using a bare `>&2' *at the end of TESTS_ENVIRONMENT* 
> > > and
> > 
> > You meant a bare `9>&2',
> >
> Yes, sorry.
> 
> > but yes that does seem to be workable for what we want!
> > 
> > > *without a following semicolon* might give a portable workaround, as if 
> > > I'm
> > > not mistaken POSIX mandates that redirections can be specified anywere on
> > > the command line, and are to be evaluated from left to right.
> > 
> > Yes, all shells support these as equivalent:
> > 
> > 9>&2 sh k
> > sh k 9>&2
> > 
> > > 
> > > UPDATE: Yes, it seems to work.  I'll add a testcase to the 'maint' branch 
> > > in
> > > case you and Jim decide to go with this solution (and you can confirm 
> > > that it
> > > really works).
> > 
> > Cool!  Definitely worth documenting in the automake manual, as owner of
> > TESTS_ENVIRONMENT and as a client of init.sh functionality, as well as
> > your proposed automake testcase addition to ensure we don't break it.
> > 
>  
> I'll then push the attached patch to automake master soonish (by tomorrow
> or so) if there is no objection by then.
> 
I've pushed the patch now.

Regards,
  Stefano

Re: [PATCH] {maint} Warnings about primary/prefix mismatch fixed and extended.

2011-06-16 Thread Stefano Lattarini 
On Monday 13 June 2011, Stefano Lattarini wrote:
> On Monday 13 June 2011, Stefano Lattarini wrote:
> > 
> > I've pushed the patch now.
> > 
> > Regards,
> >   Stefano
> > 
> But the test `primary-prefix-invalid-couples.test' is really too slow.
> What about the attached follow-up patch to improve the situation?  It
> slightly reduces coverage, but on my system bring the test execution
> time from six minutes to fifteen seconds, so it's a good trade-off
> IMHO.
> 
> I'll push in 48 hours if there is no objection by then.
> 
Pushed now.

Regards,
  Stefano

Re: [PATCH] {testsuite-work} tests: remove 'test_prefer_config_shell' from the environment

2011-06-16 Thread Stefano Lattarini 
On Thursday 16 June 2011, Stefano Lattarini wrote:
> On Monday 13 June 2011, Stefano Lattarini wrote:
> > On Monday 06 June 2011, Stefano Lattarini wrote:
> > > With the previous commit, the user could prefer the use of $SHELL
> > > over /bin/sh in some tests checking the Automake-provided shell
> > > scripts by manually exporting `test_prefer_config_shell' to "yes"
> > > in the environment.  With this commit, we ensure that such tests
> > > *always* and *transparently* run using both $SHELL and /bin/sh
> > > to execute the checked scripts.  The `test_prefer_config_shell'
> > > variable becomes an internal detail, and is no more meant to be
> > > manually defined or overridden.
> > > 
> > > * tests/defs-static.in: Check that `test_prefer_config_shell' is
> > > not exported in the environment.  Error out if this is the case.
> > >
> > [HUGE SNIP]
> > 
> > > diff --git a/tests/defs-static.in b/tests/defs-static.in
> > > index 53d9f93..17ed284 100644
> > > --- a/tests/defs-static.in
> > > +++ b/tests/defs-static.in
> > > @@ -70,7 +70,9 @@ fi
> > >  # Check that the environment is properly sanitized.
> > >  # Having variables exported to the empty string is OK, since our code
> > >  # treats such variables as if they were unset.
> > > -for var in me required parallel_tests original_AUTOMAKE 
> > > original_ACLOCAL; do
> > > +for var in me required parallel_tests test_prefer_config_shell \
> > > +   original_AUTOMAKE original_ACLOCAL
> > > +do
> > >if eval "test x\"\$$var\" != x" && env | grep "^$var=" >/dev/null; then
> > >  echo "$argv0: variable \`$var' is set in the environment:" \
> > >   "this is unsafe" >&2
> > > 
> > Hmpf, this should have been complemented by a proper addition to
> > AM_TEST_ENVIRONMENT in tests/Makefile.am.  I will sooninsh apply
> > the attached follow-up patch if there is no objection.
> > 
> > Regards,
> >   Stefano
> > 
> Pushed now.
> 
> Regards,
>   Stefano
> 
Aargh!  I had forgotten to regenerate the Makefile.ins before pushing!

Fixed with the attached patch.

(/me starting to think whether it wouln't be better to remove the
autotools-generated files from the repository ...)

Sorry for the noise,
  Stefano
From f4057070950a5991a9adbe2e8bc8e78aa16d1895 Mon Sep 17 00:00:00 2001
Message-Id: 
From: Stefano Lattarini 
Date: Thu, 16 Jun 2011 10:25:59 +0200
Subject: [PATCH] regenerate tests/Makefile.in

---
 tests/Makefile.in |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/tests/Makefile.in b/tests/Makefile.in
index 39984b6..30965f4 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -406,6 +406,7 @@ AM_TESTS_ENVIRONMENT = \
   test x"$$me" = x || unset me; \
   test x"$$required" = x || unset required; \
   test x"$$parallel_tests" = x || unset parallel_tests; \
+  test x"$$test_prefer_config_shell" || unset test_prefer_config_shell; \
   test x"$$original_AUTOMAKE" = x || unset original_AUTOMAKE; \
   test x"$$original_ACLOCAL" = x || unset original_ACLOCAL; \
   AM_TESTS_REEXEC=no; export AM_TESTS_REEXEC; \
-- 
1.7.2.3

Re: [PATCH] {testsuite-work} [PATCH] tests: test mdate-sh with /bin/sh too

2011-06-16 Thread Stefano Lattarini 
On Monday 13 June 2011, Stefano Lattarini wrote:
> I will push this in 48 hours if there is no objection.
> 
> Regards,
>   Stefano
> 
Pushed now.

Regards,
  Stefano

Re: [PATCH] {testsuite-work} tests: extend tests on `--add-missing' and `--copy' a bit

2011-06-16 Thread Stefano Lattarini 
On Monday 13 June 2011, Stefano Lattarini wrote:
> On Monday 13 June 2011, Stefano Lattarini wrote:
> > References:
> >  
> >  
> > 
> > I've pushed this patch now.
> > 
> > Regards,
> >   Stefano
> > 
> I'd like also to add the attached patch, which offers a small increase
> in coverage of possible use cases.  I'll push in 48 hours if there is no
> objection by then.
> 
> Regards,
>   Stefano
> 
Pushed now.

Regards,
  Stefano

Re: [PATCH] {testsuite-work} tests: autogenerate list of wrapped tests for `lib/' shell scripts

2011-06-16 Thread Stefano Lattarini 
On Monday 13 June 2011, Stefano Lattarini wrote:
> On Tuesday 07 June 2011, Stefano Lattarini wrote:
> > On Tuesday 07 June 2011, Peter Rosin wrote:
> > > > +
> > > > +# FIXME: this list might probably be made auto-generated, like
> > > > +# $(parallel_tests) is...  But would that worth the hassle?
> > > > +config_shell_tests = \
> > > > +  compile-w.shtst \
> > > > +  compile2-w.shtst \
> > > > +  compile3-w.shtst \
> > > > +  compile4-w.shtst \
> > > > +  compile5-w.shtst \
> > > > +  compile6-w.shtst \
> > > > +  instsh2-w.shtst \
> > > > +  instsh3-w.shtst \
> > > > +  mkinst3-w.shtst \
> > > > +  missing-w.shtst \
> > > > +  missing2-w.shtst \
> > > > +  missing3-w.shtst \
> > > > +  missing5-w.shtst
> > > 
> > > I expected to see the ar-lib tests here? Or did you not add those
> > > since the AM_PROG_AR macro hasn't been merged? But would it hurt
> > > to test ar-lib with $SHELL prior to that merge?
> > >
> > No, this is just a stupid oversight on my part, sorry.  Which probably
> > shows that we indeed should make this list automatically-generated
> > after all.  But that's for a follow-up patch IMHO.
> > 
> > Thanks,
> >   Stefano
> > 
> And here it is.  I'll push in 48 hours if there is no objection by then.
> 
Pushed now.

Regards,
  Stefano

Re: [PATCH] {testsuite-work} tests: remove 'test_prefer_config_shell' from the environment

2011-06-16 Thread Stefano Lattarini 
On Monday 13 June 2011, Stefano Lattarini wrote:
> On Monday 06 June 2011, Stefano Lattarini wrote:
> > With the previous commit, the user could prefer the use of $SHELL
> > over /bin/sh in some tests checking the Automake-provided shell
> > scripts by manually exporting `test_prefer_config_shell' to "yes"
> > in the environment.  With this commit, we ensure that such tests
> > *always* and *transparently* run using both $SHELL and /bin/sh
> > to execute the checked scripts.  The `test_prefer_config_shell'
> > variable becomes an internal detail, and is no more meant to be
> > manually defined or overridden.
> > 
> > * tests/defs-static.in: Check that `test_prefer_config_shell' is
> > not exported in the environment.  Error out if this is the case.
> >
> [HUGE SNIP]
> 
> > diff --git a/tests/defs-static.in b/tests/defs-static.in
> > index 53d9f93..17ed284 100644
> > --- a/tests/defs-static.in
> > +++ b/tests/defs-static.in
> > @@ -70,7 +70,9 @@ fi
> >  # Check that the environment is properly sanitized.
> >  # Having variables exported to the empty string is OK, since our code
> >  # treats such variables as if they were unset.
> > -for var in me required parallel_tests original_AUTOMAKE original_ACLOCAL; 
> > do
> > +for var in me required parallel_tests test_prefer_config_shell \
> > +   original_AUTOMAKE original_ACLOCAL
> > +do
> >if eval "test x\"\$$var\" != x" && env | grep "^$var=" >/dev/null; then
> >  echo "$argv0: variable \`$var' is set in the environment:" \
> >   "this is unsafe" >&2
> > 
> Hmpf, this should have been complemented by a proper addition to
> AM_TEST_ENVIRONMENT in tests/Makefile.am.  I will sooninsh apply
> the attached follow-up patch if there is no objection.
> 
> Regards,
>   Stefano
> 
Pushed now.

Regards,
  Stefano

Re: [PATCH] tests: few fixlets and improvements

2011-06-16 Thread Stefano Lattarini 
On Monday 13 June 2011, Stefano Lattarini wrote:
> A simple patch with few minor improvements to four random test cases.
> 
> I'll push in 48 hours if there is no objection by then.
> 
> Regards,
>   Stefano
> 
> -*-*-*-
> 
> tests: few fixlets and improvements
> 
> * tests/cond31.test ($required): Remove `cc', it's not really
> needed.
> * tests/confh.test: Call autoheader too.  The lack of this call
> wasn't causing spurious failures because the automatic remake
> rules were somehow invoking it on our behalf.
> * tests/fn99subdir.test: Use $subdirname throughout, instead of
> ${subdirname}, for consistency with the rest of the testsuite.
> Avoid an unnecessary subshell, which could also cause spurious
> passes, being guarded by a trailing `|| Exit 1', which neutralize
> the `errexit' flag.  Remove an unnecessary `|| Exit 1' guard.
> * tests/insh2.test: Rewrite to avoid hackish Makefile.in munging,
> and to also run configure and make.
> ---
>  ChangeLog |   16 
>  tests/cond31.test |1 -
>  tests/confh.test  |1 +
>  tests/fn99subdir.test |   36 +++-
>  tests/insh2.test  |   15 +++
>  5 files changed, 47 insertions(+), 22 deletions(-)
> 
I've pushed the patch now (to 'testsuite-works').

Regards,
  Stefano

Re: [PATCH] {maint} Deprecate de-ansification support, it should go away in automake 1.12

2011-06-16 Thread Stefano Lattarini 
Hi Bob.

On Thursday 16 June 2011, Bob Friesenhahn wrote:
> On Wed, 15 Jun 2011, Stefano Lattarini wrote:
> >>
> > Problem: it seems that there are few (and luckily only few) active packages
> > that still use the `ansi2knr' option:
> > 1. The latest version (8c) of the libjpeg implementation from IJG 
> > (Independent
> >JPEG Group, ); see the `configure.ac' file from the
> >archive 
> 
> I created most of the libjpeg build configury.  The official 
> maintainer (Guido Vollbeding) added the extra AM_INIT_AUTOMAKE 
> options.
> 
> > So, maybe it would be worthwhile to directly inform the maintainers of the
> > packages mentioned above that still use de-ANSI-fication feature about its
> > oncoming deprecation and removal.  WDYT?
> 
> If they do not hear about it, I am sure that they will learn once the 
> option stops working.
> 
OK, then I'll save myself the trouble.

Thanks,
  Stefano

[PATCH v4 3/3] parallel-tests: allow each test to have multiple results

2011-06-16 Thread Stefano Lattarini 
With this change, we improve the code creating the `test-suite.log'
global log and the console testsuite summary to make it able to
grasp multiple results per test script.  This is required in order
to introduce the planned support for test protocols, like TAP and
SubUnit, which can run multiple testcases per test script.
The implementation makes use of a custom reStructuredText field
`:am-testcase-result:'.

* lib/check.am ($(TEST_SUITE_LOG)): When processing .log files,
recognize a testcase result report only if it is declared with
the custom `:am-testcase-result:' reStructuredText field placed
at the beginning of a line.  Extend and add explanatory comments.
(recheck, recheck-html): Add explanatory comments.
* lib/pt-driver: Write an appropriate `:am-testcase-result:'
reStructuredText field in the generated log file.  Use a
reStructuredText transition to better separate the test outcome
report from the test registered output.  Improve comments.
* tests/test-driver-custom-xfail-tests.test: Adapt.
* tests/parallel-tests-empty-testlogs.test: New test.
* tests/parallel-tests-recheck-override.test: Likewise.
* tests/parallel-tests2.test: Extend and keep more in-sync with ...
* tests/test-driver-custom-html.test: ... this new related test.
* tests/test-driver-custom-no-html.test: New test.
* tests/test-driver-custom-multitest.test: Likewise.
* tests/test-driver-custom-multitest-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck2.test: Likewise.
* tests/ostp-driver: New file, used by the last four tests above.
* tests/Makefile.am (TESTS): Update.
(EXTRA_DIST): Distribute `ostp-driver'.
(test-driver-custom-multitest.log): Depend on `ostp-driver'.
(test-driver-custom-multitest-recheck.log): Likewise.
(test-driver-custom-multitest-recheck2.log): Likewise.
(test-driver-custom-html.log): Likewise.
* doc/automake.texi (API for Custom Test Drivers): Update (still
in Texinfo comments only).
---
 ChangeLog|   38 
 doc/automake.texi|   28 +++-
 lib/Automake/tests/Makefile.in   |   16 +-
 lib/am/check.am  |   41 -
 lib/pt-driver|   16 ++-
 tests/Makefile.am|   11 +
 tests/Makefile.in|   29 ++-
 tests/ostp-driver|   94 +
 tests/parallel-tests-empty-testlogs.test |   86 +
 tests/parallel-tests2.test   |   43 +++--
 tests/test-driver-custom-html.test   |  104 ++
 tests/test-driver-custom-multitest-recheck.test  |  223 ++
 tests/test-driver-custom-multitest-recheck2.test |  172 +
 tests/test-driver-custom-multitest.test  |  191 ++
 tests/test-driver-custom-no-html.test|   67 +++
 tests/test-driver-custom-xfail-tests.test|   27 ++-
 16 files changed, 1142 insertions(+), 44 deletions(-)
 create mode 100644 tests/ostp-driver
 create mode 100755 tests/parallel-tests-empty-testlogs.test
 create mode 100755 tests/test-driver-custom-html.test
 create mode 100755 tests/test-driver-custom-multitest-recheck.test
 create mode 100755 tests/test-driver-custom-multitest-recheck2.test
 create mode 100755 tests/test-driver-custom-multitest.test
 create mode 100755 tests/test-driver-custom-no-html.test

diff --git a/ChangeLog b/ChangeLog
index d84355a..be213fb 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,43 @@
 2011-06-15  Stefano Lattarini  
 
+   parallel-tests: allow each test to have multiple results
+   With this change, we improve the code creating the `test-suite.log'
+   global log and the console testsuite summary to make it able to
+   grasp multiple results per test script.  This is required in order
+   to introduce the planned support for test protocols, like TAP and
+   SubUnit, which can run multiple testcases per test script.
+   The implementation makes use of a custom reStructuredText field
+   `:am-testcase-result:'.
+   * lib/check.am ($(TEST_SUITE_LOG)): When processing .log files,
+   recognize a testcase result report only if it is declared with
+   the custom `:am-testcase-result:' reStructuredText field placed
+   at the beginning of a line.  Extend and add explanatory comments.
+   (recheck, recheck-html): Add explanatory comments.
+   * lib/pt-driver: Write an appropriate `:am-testcase-result:'
+   reStructuredText field in the generated log file.  Use a
+   reStructuredText transition to better separate the test outcome
+   report from the test registered output.  Improve comments.
+   * tests/test-driver-custom-xfail-tests.test: Adapt.
+   * tests/parallel-tests-empty-testlogs.test: New test.
+   * tests/parallel-tests-recheck-override.test: Likewise.
+   * tests/parallel-tests2.test: Extend and keep more in

[PATCH v4 2/3] parallel-tests: allow custom driver scripts

2011-06-16 Thread Stefano Lattarini 
Allow suffix-based definition of custom "driver script" for the
test scripts.  These driver scripts will be responsible of
launching the tests (or their corresponding $(LOG_COMPILER), if
they have an associated one), interpreting and displaying the
test results, and writing the `.log' files.

This new API should allow easy and flexible use of different test
protocols in the future; in particular, we plan to use it will be
used to implement TAP and SubUnit harnesses.

* automake.in (handle_tests): Define default for $(LOG_DRIVER),
and, for any registered text extension `', define defaults
for $(_LOG_DRIVER).  Substitute %DRIVER% using these new
variables, instead of the old internal $(am__pt_wrap).  When
processing check2.am, also substitute %DRIVER_FLAGS%.
Require auxiliary script `pt-driver' only if no wrapper has been
explicitly defined for the test script kinds.
* am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
?!GENERIC?%OBJ%): Pass the %DRIVER_FLAGS% to the %DRIVER% call.
* doc/automake.texi (Custom Test Drivers): New section and node.
(Overview of Custom Test Drivers Support): New subsection.
(Declaring Custom Test Drivers in @file{Makefile.am}): Likewise.
(APIs for Custom Test Drivers): New subsection, still incomplete,
with sketchy descriptions only written in texinfo comments.
(Auxiliary Programs): Mention the new `pt-driver' script.
(Optional): Mention `pt-driver' in AC_CONFIG_AUX_DIR.  Since we
are at it, break the list of auxiliary script by placing one per
line, to simplify potential future additions of new scripts.
* tests/parallel-tests-no-pt-driver.test: New test.
* tests/test-driver-custom.test: Likewise.
* tests/test-driver-custom-xfail-tests.test: Likewise.
* tests/test-driver-fail.test: Likewise.
* tests/Makefile.am: Update.
* NEWS: Update.
---
 ChangeLog  |   36 ++
 NEWS   |   12 ++-
 automake.in|   38 +-
 doc/automake.texi  |  176 +++-
 lib/Automake/tests/Makefile.in |4 +-
 lib/am/check2.am   |4 +-
 tests/Makefile.am  |4 +
 tests/Makefile.in  |8 +-
 tests/test-driver-custom-no-pt-driver.test |   63 ++
 tests/test-driver-custom-xfail-tests.test  |  147 +++
 tests/test-driver-custom.test  |  140 ++
 tests/test-driver-fail.test|   65 ++
 12 files changed, 675 insertions(+), 22 deletions(-)
 create mode 100755 tests/test-driver-custom-no-pt-driver.test
 create mode 100755 tests/test-driver-custom-xfail-tests.test
 create mode 100755 tests/test-driver-custom.test
 create mode 100755 tests/test-driver-fail.test

diff --git a/ChangeLog b/ChangeLog
index 0771b24..d84355a 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,41 @@
 2011-06-15  Stefano Lattarini  
 
+   parallel-tests: allow custom driver scripts
+   Allow suffix-based definition of custom "driver script" for the
+   test scripts.  These driver scripts will be responsible of
+   launching the tests (or their corresponding $(LOG_COMPILER), if
+   they have an associated one), interpreting and displaying the
+   test results, and writing the `.log' files.
+   This new API should allow easy and flexible use of different test
+   protocols in the future; in particular, we plan to use it will be
+   used to implement TAP and SubUnit harnesses.
+   * automake.in (handle_tests): Define default for $(LOG_DRIVER),
+   and, for any registered text extension `', define defaults
+   for $(_LOG_DRIVER).  Substitute %DRIVER% using these new
+   variables, instead of the old internal $(am__pt_wrap).  When
+   processing check2.am, also substitute %DRIVER_FLAGS%.
+   Require auxiliary script `pt-driver' only if no wrapper has been
+   explicitly defined for the test script kinds.
+   * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
+   ?!GENERIC?%OBJ%): Pass the %DRIVER_FLAGS% to the %DRIVER% call.
+   * doc/automake.texi (Custom Test Drivers): New section and node.
+   (Overview of Custom Test Drivers Support): New subsection.
+   (Declaring Custom Test Drivers in @file{Makefile.am}): Likewise.
+   (APIs for Custom Test Drivers): New subsection, still incomplete,
+   with sketchy descriptions only written in texinfo comments.
+   (Auxiliary Programs): Mention the new `pt-driver' script.
+   (Optional): Mention `pt-driver' in AC_CONFIG_AUX_DIR.  Since we
+   are at it, break the list of auxiliary script by placing one per
+   line, to simplify potential future additions of new scripts.
+   * tests/parallel-tests-no-pt-driver.test: New test.
+   * tests/test-driver-custom.test: Likewise.
+   * tests/test-driver-custom-xfail-tests.test: Likewise.
+   * tests/test-

[PATCH v4 1/3] parallel-tests: add auxiliary script 'pt-driver', refactor

2011-06-16 Thread Stefano Lattarini 
This refactoring should cause no API of functionality change,
and is meant only to simplify the future implementation of TAP
and SubUnit testsuite drivers.  More precisely, our roadmap is
to move most of the "testsuite driving" features out of the
Automake-generated Makefiles, and into external scripts with
well-defined interfaces.  This will allow the user to define
its own personalized testsuite drivers, and will also offer us
a framework upon which to implement our new TAP and SubUnit
drivers, all in a very unobtrusive way and retaining an high
degree of code reuse and backward-compatibility.

* lib/pt-driver: New auxiliary script.
* lib/Makefile.am (dist_SCRIPT_DATA): Add it.
* automake.in (handle_tests): Require the new auxiliary script
`pt-driver', and define new makefile variable `$(am__pt_driver)',
used to call it.  Perform new substitution on `DRIVER' when
processing the `check2.am' file.
* lib/check.am (am__tty_colors): Define new shell variable
`$am__color_tests'.
(am__rst_section): Removed, its role taken over by the new
`pt-driver' script.
(am__test_driver_flags): New variable, contains the command
line options passed to `pt-driver'.
(am__check_pre): Do not deal with temporary files and exit
traps anymore, as the `pt-driver' script takes care of that now.
Define shell variable `$am__enable_hard_errors', used by
`$(am__test_driver_flags)'.  Reorder so that we don't need to
save and restore the value of the `TERM' environment variable
anymore.
Other related adjustments.
(am__check_post): Remove, as its role has been completely taken
over by the `pt-driver' script.
* am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
?!GENERIC?%OBJ%): Call the test script through the Automake
substituted `%DRIVER%', and honor the command-line options
in `$(am__test_driver_flags)'.  Do not call the obsoleted
`$(am__check_post)' anymore.
* tests/check.test: Adjust.
* tests/check2.test : Likewise.
* tests/check3.test : Likewise.
* tests/check4.test : Likewise.
* tests/check10.test: Likewise.
* tests/color.test: Likewise.
* tests/color2.test: Likewise.
* tests/comment9.test: Likewise.
* tests/dejagnu.test: Likewise.
* tests/exeext4.test: Likewise.
* tests/maken3.test: Likewise.
* tests/maken4.test: Likewise.
* tests/parallel-tests-interrupt.test: Likewise.
* tests/posixsubst-tests.test: Likewise.
* tests/repeated-options.test: Likewise.
* tests/check-no-pt-driver.test: New test.
* tests/parallel-tests-pt-driver.test: Likewise.
* tests/Makefile.am (TESTS): Update.
* NEWS: Update.
---
 ChangeLog   |   59 
 NEWS|6 ++
 automake.in |8 ++-
 lib/Automake/tests/Makefile.in  |   69 ---
 lib/Makefile.am |2 +-
 lib/Makefile.in |2 +-
 lib/am/check.am |   73 
 lib/am/check2.am|   10 ++-
 lib/pt-driver   |  129 +++
 tests/Makefile.am   |2 +
 tests/Makefile.in   |   71 ---
 tests/check-no-pt-driver.test   |   33 +
 tests/check.test|2 +
 tests/check10.test  |4 +
 tests/check2.test   |4 +
 tests/check3.test   |2 +-
 tests/check4.test   |9 ++-
 tests/color.test|   10 +++-
 tests/color2.test   |4 +-
 tests/comment9.test |2 +
 tests/dejagnu.test  |2 +
 tests/exeext4.test  |2 +-
 tests/maken3.test   |2 +-
 tests/maken4.test   |2 +-
 tests/parallel-tests-interrupt.test |   16 -
 tests/parallel-tests-pt-driver.test |   93 +
 tests/posixsubst-tests.test |2 +-
 tests/repeated-options.test |2 +-
 28 files changed, 481 insertions(+), 141 deletions(-)
 create mode 100755 lib/pt-driver
 create mode 100755 tests/check-no-pt-driver.test
 create mode 100755 tests/parallel-tests-pt-driver.test

diff --git a/ChangeLog b/ChangeLog
index aad41c7..0771b24 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,62 @@
+2011-06-15  Stefano Lattarini  
+
+   parallel-tests: add auxiliary script 'pt-driver', refactor
+   This refactoring should cause no API of functionality change,
+   and is meant only to simplify the future implementation of TAP
+   and SubUnit testsuite drivers.  More precisely, our roadmap is
+   to move most of the "testsuite driving" features out of the
+   Automake-generated Makefiles, and into external scripts with
+   well-defined interfaces.  This will allow the user to define
+   its own personalized testsuite drivers, and will also offer us
+   a framework upon which to implement our new TAP and SubUnit
+   drivers, all in a very unobtrusive way and ret

Re: [PATCH] check: don't use multi-line coloring for the report

2011-06-16 Thread Stefano Lattarini 
Hi Bert, thanks for the patch.

On Thursday 16 June 2011, Bert Wesarg wrote:
> less can't handle coloring which spans newlines because of performance
> reasons. Thus, color each line of the check report by its own.
> 
> ---
> 
> For reference, git had a similar problem and I talked to the less
> maintainer about the problem. Here is the resulting fix in git:
> 
> http://repo.or.cz/w/git.git/commit/374664478f204ab45bbd494ab21492f331d8b1f0
> ---
>  lib/am/check.am |   13 +++--
>  1 files changed, 7 insertions(+), 6 deletions(-)
> 
> diff --git a/lib/am/check.am b/lib/am/check.am
> index 97ecb68..63451b7 100644
> --- a/lib/am/check.am
> +++ b/lib/am/check.am
> @@ -401,14 +401,15 @@ check-TESTS: $(TESTS)
> fi; \
> dashes=`echo "$$dashes" | sed s/./=/g`; \
> if test "$$failed" -eq 0; then \
> - echo "$$grn$$dashes"; \
> + col="$$grn"; \
> else \
> - echo "$$red$$dashes"; \
> + col="$$red"; \
> fi; \
> -   echo "$$banner"; \
> -   test -z "$$skipped" || echo "$$skipped"; \
> -   test -z "$$report" || echo "$$report"; \
> -   echo "$$dashes$$std"; \
> +   echo "$${col}$$dashes$${std}"; \
> +   echo "$${col}$$banner$${std}"; \
> +   test -z "$$skipped" || echo "$${col}$$skipped$${std}"; \
> +   test -z "$$report" || echo "$${col}$$report$${std}"; \
> +   echo "$${col}$$dashes$${std}"; \
> test "$$failed" -eq 0; \
>   else :; fi
>  
 
I verified that the problem you reported is really present, and
I agree it would be nice to fix it.

However, your patch fixes the problem only fot the old "serial"
testsuite harness, not for the new "parallel" one.  It would be
nice to fix the problem for both (especially because most
Automake-using GNU projects employs the new "parallel" harness
now).  Are you interested in improving your patch to do so?  If
yes, note that ideally you should also add yourself to the
THANKS file, and write a ChangeLog entry describing the problem
you've fixed and the changes you made to the files (if you don't,
I can write it for you before pushing the patch). 

Regards,
  Stefano

Re: [PATCH] {maint} distcheck: add support for AM_DISTCHECK_CONFIGURE_FLAGS

2011-06-16 Thread Stefano Lattarini 
On Thursday 16 June 2011, Ralf Corsepius wrote:
> On 06/15/2011 11:56 PM, Stefano Lattarini wrote:
> > On Wednesday 15 June 2011, Ralf Corsepius wrote:
> >
> >> In other words: IMO, automake is right in encouraging users to avod
> >> DISTCHECK_CONFIGURE_FLAGS,
> >>
> > Actually, automake will discourage the use of AM_DISTCHECK_CONFIGURE_FLAGS
> > *by developers*, not of DISTCHECK_CONFIGURE_FLAGS by users.
> Pardon, I sense a misunderstanding.
> 
> I consider "developers" to be "automake's users" and was using the word 
> "users" in this sense.
>
Ah, sorry then; I usually employ the term "developers" or "package authors"
to indicate Automake's users, and "users" to indicate the users of Automake
based packages (they can be either final users, or maintainers of Linux
distros' packages or FreeBSD ports).

In the light of your precisation above, I can say that we are in violent
agreement then ;-)

Regards,
  Stefano