Hi Stefano, * Stefano Lattarini wrote on Sun, Nov 14, 2010 at 02:29:33PM CET: > Another testsuite patch, based off of maint, but to be merged > in master only.
OK, but I have a question and a nit below. Thanks, Ralf > Extend and improve tests on DejaGnu support. > > * tests/dejagnu.test: Do not create useless dummy test script. > Add trailing `:' command. In heading comments, add reference > to ... > * tests/check12.test: ... this new "semantic" test, covering > concurrent use of dejagnu tests, simple tests and `check-local' > target. > * tests/dejagnu2.test: Make test more reliable, by avoid weak > grepping of make output. Prefer `cat' over `echo' to append > to configure.in. Quote literal dots in grep regexps. Prefer > `grep -c ...' over `grep ... | wc -l'. Make grepping of > automake stderr slighty stricter. Add trailing `:' command. slightly > * tests/dejagnu3.test: Prefer `cat' over `echo' to append to > configure.in. Check stderr of expected-to-fail "make" call. > Remove extra blank lines from Makefile.am. > * tests/dejagnu4.test: Prefer `cat' over `echo' to append to > configure.in. Prefer `mv -f' over plain `mv' when the target > file already exists. Avoid extra mkdir calls by creating more > directories at once. Better use of blank lines. Check that > the `*.log' and `*.sum' files are created by runtest also when > "make check" fails. > * tests/dejagnu7.test: Prefer `cat' over `echo' to append to > configure.in. Better use of blank lines. Add a trailing `:' > command. > * tests/dejagnu6.test: Likewise, and give the dejagnu test a > more descriptive name. > * tests/dejagnu5.test: Likewise. Also, simply define package > name to `$me' rather than using a non-obvious sed script to > extract it from `AC_INIT', and write the Makefile.am with only > one command. > * tests/Makefile.am (TESTS): Updated. > --- /dev/null > +++ b/tests/check12.test > @@ -0,0 +1,201 @@ > +# Test to make sure dejagnu tests, automake-style tests, and check-local > +# target can coexist. [...] > +## Go with the testcase execution. > + > +$ACLOCAL > +$AUTOCONF > +$AUTOMAKE --add-missing > + > +for vpath in : false; do > + > + if $vpath; then > + srcdir=.. > + mkdir build_ > + cd build_ > + else > + srcdir=. > + fi > + > + $srcdir/configure > + > + $MAKE check > + test -f hammer.log > + test -f hammer.sum > + test -f spanner.log > + test -f spanner.sum > + # This checks will be run only by the autogenerated `check12-p.test'. > + if test x"$parallel_tests" = x"yes"; then > + cat test-suite.log > + grep '^PASS: a\.test (exit: 0)' a.log > + grep '^PASS: b\.test (exit: 0)' b.log > + else > + : > + fi > + grep 'check-local succeded :-)' local.log > + > + cp -f config.status config-status.sav > + > + $MAKE distclean > + test ! -r hammer.log > + test ! -r hammer.sum > + test ! -r spanner.log > + test ! -r spanner.sum > + test ! -r test-suite.log > + test ! -r a.log > + test ! -r b.log > + test ! -r local.log > + > + mv -f config-status.sav config.status > + ./config.status > + > + NAIL=screw $MAKE check && Exit 1 > + test -f hammer.log > + test -f hammer.sum > + test -f spanner.log > + test -f spanner.sum > + grep 'FAIL: test_hammer' hammer.sum > + grep 'FAIL:' spanner.sum && Exit 1 > + > + B_EXIT_STATUS=1 $MAKE check && Exit 1 > + # This checks will be run only by the autogenerated `check12-p.test'. > + if test x"$parallel_tests" = x"yes"; then > + cat test-suite.log > + grep '^PASS: a\.test (exit: 0)' a.log > + grep '^FAIL: b\.test (exit: 1)' b.log > + grep '^FAIL: b\.test (exit: 1)' test-suite.log > + else :; fi > + > + CHECKLOCAL_EXIT_STATUS=1 $MAKE check && Exit 1 > + grep 'check-local failed :-(' local.log > + > + # Do not trust the exit status of `make -k'. > + NAIL=screw B_EXIT_STATUS=23 CHECKLOCAL_EXIT_STATUS=1 $MAKE -k check || : > + test -f hammer.log > + test -f hammer.sum > + test -f spanner.log > + test -f spanner.sum > + grep 'FAIL: test_hammer' hammer.sum > + grep 'FAIL:' spanner.sum && Exit 1 > + # This checks will be run only by the autogenerated `check12-p.test'. > + if test x"$parallel_tests" = x"yes"; then > + cat test-suite.log > + grep '^PASS: a\.test (exit: 0)' a.log > + grep '^FAIL: b\.test (exit: 23)' b.log > + grep '^FAIL: b\.test (exit: 23)' test-suite.log > + else :; fi > + grep 'check-local failed :-(' local.log > + > + $MAKE distcheck Can't you either omit this distcheck, or the vpath = : case above, without decreasing coverage? > + $MAKE distclean > + cd $srcdir > + > +done > + > +: