With this, you can run make -k check-coverage and on the next day, you might have useful coverage data. :-)
Well, if something failed, you might still need to follow up with make check-coverage-report Pausing for some coffee is still worthwhile at this point. For a reasonably sane edit-compile-cycle, use make -k recheck-coverage or limit the set of tests to be run make -k check-coverage TESTS="foo.test bar.test" make check-coverage-report The environment variable WANT_NO_THREADS is used to skip those tests that required ithreads. I thought of hooking clean-coverage to maintainer-clean-local, but that doesn't help much, as Devel::Cover invalidates coverage data for changed scripts as well. Cheers, Ralf Perl coverage support using Devel::Cover. This introduces makefile rules to run the testsuite with Perl coverage enabled. It skips tests that use perl ithreads, by unsetting AUTOMAKE_JOBS and setting WANT_NO_THREADS to make the threaded tests skip. * Makefile.am (PERL_COVERAGE_DB, PERL_COVERAGE_FLAGS) (PERL_COVER): New variables. (check-coverage, recheck-coverage, clean-coverage): New phony targets. (check-coverage-run, recheck-coverage-run): New phony helper targets. (clean-local): New, depend on clean-coverage. * lib/Automake/tests/Condition-t.pl: Skip if WANT_NO_THREADS is set. * lib/Automake/tests/DisjConditions-t.pl: Likewise. * tests/defs.in: New required entry 'perl-threads'. * tests/parallel-am.test: Use it to skip if WANT_NO_THREADS is set. * tests/parallel-am2.test: Likewise. * tests/parallel-am3.test: Likewise. diff --git a/HACKING b/HACKING index 64d55f9..c8ae47e 100644 --- a/HACKING +++ b/HACKING @@ -125,6 +125,9 @@ * Use `keep_testdirs=yes' to keep test directories for successful tests also. +* Use perl coverage information to ensure your new code is thoroughly + tested by your new tests. + ================================================================ = Release procedure diff --git a/Makefile.am b/Makefile.am index 66d8315..fc16b7a 100644 --- a/Makefile.am +++ b/Makefile.am @@ -121,6 +121,41 @@ recheck: dist-hook: cd $(distdir)/tests && chmod a+rx *.test + +# Perl coverage statistics. +PERL_COVERAGE_DB = $(abs_top_builddir)/cover_db +PERL_COVERAGE_FLAGS = -MDevel::Cover=-db,$(PERL_COVERAGE_DB),-silent,on,-summary,off +PERL_COVER = cover + +check-coverage-run recheck-coverage-run: all + $(mkinstalldirs) $(PERL_COVERAGE_DB) + PERL5OPT="$$PERL5OPT $(PERL_COVERAGE_FLAGS)"; export PERL5OPT; \ + WANT_NO_THREADS=yes; export WANT_NO_THREADS; unset AUTOMAKE_JOBS; \ + $(MAKE) $(AM_MAKEFLAGS) `echo $@ | sed 's/-coverage-run//'` + +check-coverage-report: + @if test ! -d "$(PERL_COVERAGE_DB)"; then \ + echo "No coverage database found in \`$(PERL_COVERAGE_DB)'." >&2; \ + echo "Please run \`make check-coverage' first" >&2; \ + exit 1; \ + fi + $(PERL_COVER) $(PERL_COVER_FLAGS) "$(PERL_COVERAGE_DB)" + +# We don't use direct dependencies here because we'd like to be able +# to invoke the report even after interrupted check-coverage. +check-coverage: check-coverage-run + $(MAKE) $(AM_MAKEFLAGS) check-coverage-report + +recheck-coverage: recheck-coverage-run + $(MAKE) $(AM_MAKEFLAGS) check-coverage-report + +clean-coverage: + rm -rf "$(PERL_COVERAGE_DB)" +clean-local: clean-coverage + +.PHONY: check-coverage recheck-coverage check-coverage-run \ + recheck-coverage-run check-coverage-report clean-coverage + # Some simple checks, and then ordinary check. These are only really # guaranteed to work on my machine. syntax_check_rules = \ diff --git a/lib/Automake/tests/Condition-t.pl b/lib/Automake/tests/Condition-t.pl index 06eb34e..0f1dde8 100644 --- a/lib/Automake/tests/Condition-t.pl +++ b/lib/Automake/tests/Condition-t.pl @@ -1,4 +1,5 @@ -# Copyright (C) 2001, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2003, 2008, 2009 Free Software Foundation, +# Inc. # # This file is part of GNU Automake. # @@ -18,7 +19,8 @@ BEGIN { use Config; if (eval { require 5.007_002; } # for CLONE support - && $Config{useithreads}) + && $Config{useithreads} + && !$ENV{WANT_NO_THREADS}) { require threads; import threads; diff --git a/lib/Automake/tests/DisjConditions-t.pl b/lib/Automake/tests/DisjConditions-t.pl index 2fe275b..eccdcd6 100644 --- a/lib/Automake/tests/DisjConditions-t.pl +++ b/lib/Automake/tests/DisjConditions-t.pl @@ -1,4 +1,5 @@ -# Copyright (C) 2001, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2003, 2008, 2009 Free Software Foundation, +# Inc. # # This file is part of GNU Automake. # @@ -18,7 +19,8 @@ BEGIN { use Config; if (eval { require 5.007_002; } # for CLONE support - && $Config{useithreads}) + && $Config{useithreads} + && !$ENV{WANT_NO_THREADS}) { require threads; import threads; diff --git a/tests/defs.in b/tests/defs.in index ecdf069..6eec344 100644 --- a/tests/defs.in +++ b/tests/defs.in @@ -170,6 +170,10 @@ do rm -f $priv_check_temp test $overwrite_status = 0 && exit 77 ;; + perl-threads) + # Skip with Devel::Cover: it cannot cope with threads. + test "$WANT_NO_THREADS" = yes && exit 77 + ;; python) # Python doesn't support --version, it has -V echo "$me: running python -V" diff --git a/tests/parallel-am.test b/tests/parallel-am.test index 4aef628..c46377b 100755 --- a/tests/parallel-am.test +++ b/tests/parallel-am.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -35,6 +35,7 @@ # # This test checks (0), (1), and (2). See sister tests for further coverage. +required=perl-threads . ./defs || Exit 1 set -e diff --git a/tests/parallel-am2.test b/tests/parallel-am2.test index 20225a0..d929740 100755 --- a/tests/parallel-am2.test +++ b/tests/parallel-am2.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,7 @@ # 4) warning and normal error output should be identical, in that duplicate # warnings should be omitted in the same way as without threads, +required=perl-threads . ./defs || Exit 1 set -e diff --git a/tests/parallel-am3.test b/tests/parallel-am3.test index 5db9c67..104376a 100755 --- a/tests/parallel-am3.test +++ b/tests/parallel-am3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -21,6 +21,7 @@ # with --add-missing, even with concurrent file requirements, and the # installation of aux files should be race-free, +required=perl-threads . ./defs || Exit 1 set -e