bug#8893: [PATCH 1/2] tests: make test runner a script, not a shell function (was: Automake patches for custom test drivers' support break coreutils testsuite)

2011-06-18 Thread Stefano Lattarini
On Saturday 18 June 2011, Jim Meyering wrote:
> Stefano Lattarini wrote:
> >> ...
> >> >
> >> > In order to work with the upcoming new Automake testsuite harness, 
> >> > coreutils
> >> > have two possibilities:
> >> >  1. move the `shell_or_perl_' subroutine's functionality into a real 
> >> > acript,
> >> > and define the LOG_COMPILER to point to it; or
> >> >  2. add a `.pl' extension to the perl test scripts, and define 
> >> > PL_LOG_COMPILER
> >> > appropriately (might be a little tricky, considering the hops that 
> >> > the
> >> > `shell_or_perl_' subroutine goes through in order to get the flags 
> >> > and
> >> > imports right).
> >>
> >> 1) sounds preferable.
> >>
> > But it has a serious drawback: the redirection `9>&2' placed at the end
> > of TESTS_ENVIRONMENT will be rendered useless by the final exec done
> > in the new `shell_or_perl' script (at least for with shells using the
> > `cloexec' flag on fds > 2); this will bring back the problems fixed by
> > commit `v8.12-82-g6b68745' :-(
> >
Well, no, that's not true; the "problematic" shells only set the 'close-on-exec'
flag on the file descriptors they themselves create, not on the ones they 
inherit
from their parents:

  $ ksh -c 'exec 9>&1; sh -c "echo foo >&9"'
  sh: 9: Bad file descriptor
  $ ksh -c 'sh -c "echo foo >&9"' 9>&1
  foo
  $ (exec 9>&1; ksh -c 'sh -c "echo foo >&9"')
  foo

I should have tested better before reporting an imaginary problem.

> > So I now think we should go with solution (2).
> 
> Ok.
> 
I've gone with the less invasive solution (1) instead.  See the attached
patch.  I will soon post a follow up that tries to compensate for the
extra forks I've introduced here by removing a couple of grep calls from
the `shell-or-perl' script (this is for Cygwin, I know that the overhead
on Unix is utterly irrelevant).

Regards,
  Stefano
From e4dd05df8ea1f0fc99649627f895a76ec776279a Mon Sep 17 00:00:00 2001
Message-Id: 
From: Stefano Lattarini 
Date: Sat, 18 Jun 2011 10:26:15 +0200
Subject: [PATCH 1/2] tests: make test runner a script, not a shell function

This changes implements a more correct and idiomatic use of the
features of the Automake-provided 'parallel-tests' harness.
Moreover, this change is required in order for the testsuite to
continue to work with the new testsuite harness that is planned
to be introduced in Automake 1.12 (which, as of the writing date,
is still under development and in alpha state).

* tests/shell-or-perl: New auxiliary script.
* tests/Makefile.am (EXTRA_DIST): Distribute it.
* tests/check.mk (TESTS_ENVIRONMENT): Remove definition of the
`shell_or_perl_' shell function, whose code has been moved in
the new script above (with few improvements and extensions). Do
not use it to run the test scripts.
(LOG_COMPILER): New, properly invoking `shell-or-perl'.
---
 tests/Makefile.am   |1 +
 tests/check.mk  |   27 +---
 tests/shell-or-perl |  111 +++
 3 files changed, 123 insertions(+), 16 deletions(-)
 create mode 100644 tests/shell-or-perl

diff --git a/tests/Makefile.am b/tests/Makefile.am
index a324dc1..f8fbd38 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -20,6 +20,7 @@ EXTRA_DIST =		\
   other-fs-tmpdir	\
   require-perl		\
   sample-test		\
+  shell-or-perl		\
   $(pr_data)
 
 root_tests =	\
diff --git a/tests/check.mk b/tests/check.mk
index 9db96af..d45c288 100644
--- a/tests/check.mk
+++ b/tests/check.mk
@@ -48,6 +48,16 @@ check-am: .built-programs
 && MAKEFLAGS= $(MAKE) -s built_programs.list)		\
   > $@-t && mv $@-t $@
 
+## `$f' is set by the Automake-generated test harness to the path of the
+## current test script stripped of VPATH components, and is used by the
+## shell-or-perl script to determine the name of the temporary files to be
+## used.  Note that $f is a shell variable, not a make macro, so the use of
+## `$$f' below is correct, and not a typo.
+LOG_COMPILER = \
+  $(SHELL) $(srcdir)/shell-or-perl \
+  --test-name "$$f" --srcdir '$(srcdir)' \
+  --shell '$(SHELL)' --perl '$(PERL)' --
+
 # Note that the first lines are statements.  They ensure that environment
 # variables that can perturb tests are unset or set to expected values.
 # The rest are envvar settings that propagate build-related Makefile
@@ -58,21 +68,6 @@ TESTS_ENVIRONMENT =\
   test -d "$$tmp__" && test -w "$$tmp__" || tmp__=.;	\
   . $(srcdir)/envvar-check;			\
   TMPDIR=$$tmp__; export TMPDIR;		\
-  shell_or_perl_() {\
-if grep '^\#!/usr/bin/perl' "$$1" > /dev/null; then			\
-  if $(PERL) -e 'use warnings' > /dev/null 2>&1; then		\
-	grep '^\#!/usr/bin/perl -T' "$$1" > /dev/null && T_=T || T_=;	\
-$(PERL) -w$$T_ -I$(srcdir) -MCoreutils -MCuSkip			\
-	  -M"CuTmpdir qw($$f)" -- "$$1";	\
-  else	\
-	echo 1>&2 "$$tst: configure did not find a usable version of Perl," \
-	  "so skipping this test";		\
-	(exit 77);\
-  fi;	\
-else	\
-  $(SHELL)

bug#8893: [PATCH 1/2] tests: make test runner a script, not a shell function (was: Automake patches for custom test drivers' support break coreutils testsuite)

2011-06-18 Thread Stefano Lattarini
On Saturday 18 June 2011, Stefano Lattarini wrote:
> On Saturday 18 June 2011, Jim Meyering wrote:
> > Stefano Lattarini wrote:
> > >> ...
> > >> >
> > >> > In order to work with the upcoming new Automake testsuite harness, 
> > >> > coreutils
> > >> > have two possibilities:
> > >> >  1. move the `shell_or_perl_' subroutine's functionality into a real 
> > >> > acript,
> > >> > and define the LOG_COMPILER to point to it; or
> > >> >  2. add a `.pl' extension to the perl test scripts, and define 
> > >> > PL_LOG_COMPILER
> > >> > appropriately (might be a little tricky, considering the hops that 
> > >> > the
> > >> > `shell_or_perl_' subroutine goes through in order to get the flags 
> > >> > and
> > >> > imports right).
> > >>
> > >> 1) sounds preferable.
> > >>
> > > But it has a serious drawback: the redirection `9>&2' placed at the end
> > > of TESTS_ENVIRONMENT will be rendered useless by the final exec done
> > > in the new `shell_or_perl' script (at least for with shells using the
> > > `cloexec' flag on fds > 2); this will bring back the problems fixed by
> > > commit `v8.12-82-g6b68745' :-(
> > >
> Well, no, that's not true; the "problematic" shells only set the 
> 'close-on-exec'
> flag on the file descriptors they themselves create, not on the ones they 
> inherit
> from their parents:
> 
>   $ ksh -c 'exec 9>&1; sh -c "echo foo >&9"'
>   sh: 9: Bad file descriptor
>   $ ksh -c 'sh -c "echo foo >&9"' 9>&1
>   foo
>   $ (exec 9>&1; ksh -c 'sh -c "echo foo >&9"')
>   foo
> 
> I should have tested better before reporting an imaginary problem.
> 
> > > So I now think we should go with solution (2).
> > 
> > Ok.
> > 
> I've gone with the less invasive solution (1) instead.  See the attached
> patch.  I will soon post a follow up that tries to compensate for the
> extra forks I've introduced here by removing a couple of grep calls from
> the `shell-or-perl' script
>
Here it is.

Regards,
  Stefano
From bb3cbef9eece619dd694a60e5a738e71e939f9aa Mon Sep 17 00:00:00 2001
Message-Id: 
In-Reply-To: 
References: 
From: Stefano Lattarini 
Date: Sat, 18 Jun 2011 17:57:53 +0200
Subject: [PATCH 2/2] tests: avoid extra forks in the testsuite

* tests/shell-or-perl: Prefer the `read' builtin over `grep' to
look at the shebang line of test scripts.  Since `read' is a
special builtin, it might abort the whole program upon failures,
so add extra sanity checks, verifying that the test script exists
and is readable, before trying to read from it.
---
 tests/shell-or-perl |   16 
 1 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/tests/shell-or-perl b/tests/shell-or-perl
index ff92009..8c92f87 100644
--- a/tests/shell-or-perl
+++ b/tests/shell-or-perl
@@ -84,12 +84,19 @@ test -z "$test_name" && test_name=$test_script
 #  Run the test script  #
 # - #
 
-if grep '^#!/usr/bin/perl' "$test_script" >/dev/null; then
+test -f "$test_script" && test -r "$test_script" \
+  || error_ "test script '$test_script' does not exist, or isn't readable"
+
+read shebang_line < "$test_script" \
+  || error_ "cannot read from the test script '$test_script'"
+
+case $shebang_line in
+'#!/usr/bin/perl'*)
   # The test is a perl script.
   if $cu_PERL -e 'use warnings' > /dev/null 2>&1; then
 # Perl is available, see if we must run the test with taint
 # mode on or not.
-grep '^#!/usr/bin/perl -T' "$test_script" >/dev/null && T_=T || T_=
+case $shebang_line in *\ -T*) T_=T;; *) T_=;; esac
 # Now run it.
 exec $cu_PERL -w$T_ -I"$srcdir" -MCoreutils -MCuSkip \
   -M"CuTmpdir qw($test_name)" \
@@ -99,10 +106,11 @@ if grep '^#!/usr/bin/perl' "$test_script" >/dev/null; then
 echo "$test_name: skip: no usable version of Perl found"
 exit 77
   fi
-else
+  ;;
+*)
   # Assume the test is a shell script.
   exec $cu_SHELL "$test_script" ${1+"$@"}
-fi
+esac
 
 # - #
 #  Not reached  #
-- 
1.7.2.3