On Fri, Sep 12, 2014 at 04:42:25PM -0700, Mike Stump wrote:
> On Sep 12, 2014, at 9:32 AM, Jakub Jelinek <ja...@redhat.com> wrote:
> > Here is my latest version of the patch.
>
> I did a timing test:
Here is an updated version.
Changes since last version:
1) acats parallelized the same way (just, because it is in shell,
using mkdir instead of open with O_EXCL|O_CREAT);
also, as acats has pretty significant initial setup time
(up to a minute or so on not really fast box), I'm now
creating the /support stuff before spawning the parallel
jobs and let the parallel jobs use the same shared support
directory
2) I'm now using addprefix instead of patsubst where appropriate
3) I'm using $(if ...) instead of $(or ...) to make it usable
with make 3.80 (3.81 already supports or)
4) parallelization is performed for any kinds of RUNTESTFLAGS arguments now
5) struct-layout-1.exp apparently doesn't have to be performed serially,
and for gnu-encoding.exp I've used similar change as for go-test.exp
6) in libstdc++, abi.exp, pretty-printers.exp and xmethods.exp are performed
together with conformance.exp, so again parallelization for any
RUNTESTFLAGS flags; abi.exp and xmethods.exp are serially tested
by the first runtest instance to encounter them
Regtested on x86_64-linux, without the patch toplevel make -k check
took 8hrs3minutes (don't have time data for that run), without the
patch toplevel make -j48 -k check took:
real 40m21.984s
user 341m51.675s
sys 112m46.993s
and with the patch make -j48 -k check took:
real 32m22.066s
user 355m1.788s
sys 117m5.809s
I saw over 45 jobs running pretty much as the point where all the
testing was done, and test_summary run from the non-parallel testing
is the same as test_summary from the -j48 testing with the patch.
Is this version ok for trunk?
2014-09-14 Jakub Jelinek <ja...@redhat.com>
gcc/
* Makefile.in (dg_target_exps): Remove.
(check_gcc_parallelize): Change to just an upper bound number.
(check-%-subtargets): Always print the non-parallelized goals.
(check_p_vars, check_p_comma, check_p_subwork): Remove.
(check_p_count, check_p_numbers0, check_p_numbers1, check_p_numbers2,
check_p_numbers3, check_p_numbers4, check_p_numbers5,
check_p_numbers6): New variables.
(check_p_numbers): Set to sequence from 1 to 9999.
(check_p_subdirs): Set to sequence from 1 to minimum of
$(check_p_count) and either GCC_TEST_PARALLEL_SLOTS env var if set,
or 128.
(check-%, check-parallel-%): Rewritten so that for parallelized
testing each job runs all the *.exp files, with
GCC_RUNTEST_PARALLELIZE_DIR set in environment.
gcc/go/
* Make-lang.in (check_go_parallelize): Change to just an upper bound
number.
gcc/fortran/
* Make-lang.in (check_gfortran_parallelize): Change to just an upper
bound number.
gcc/cp/
* Make-lang.in (check_g++_parallelize): Change to just an upper bound
number.
gcc/objc/
* Make-lang.in (check_objc_parallelize): Change to just an upper
bound number.
gcc/ada/
* gcc-interface/Make-lang.in (check_acats_numbers0,
check_acats_numbers1, check_acats_numbers2, check_acats_numbers3,
check_acats_numbers4, check_acats_numbers5, check_acats_numbers6,
check_acats_numbers, check_acats_subdirs): New variables.
(check_acats_targets): Use $(check_acats_subdirs).
(check-acats, check-acats%): Rewritten so that for parallelized
testing each job runs all the chapters files, with
GCC_RUNTEST_PARALLELIZE_DIR set in environment. Prepare the support
directory sequentially and share it.
(check-acats-subtargets): Always print just check-acats.
gcc/testsuite/
* lib/gcc-defs.exp (gcc_parallel_test_run_p,
gcc_parallel_test_enable): New procedures. If
GCC_RUNTEST_PARALLELIZE_DIR is set in environment, override
runtest_file_p to invoke also gcc_parallel_test_run_p.
* g++.dg/guality/guality.exp (check_guality): Save/restore
test_counts array around the body of the procedure.
* gcc.dg/guality/guality.exp (check_guality): Likewise.
* g++.dg/plugin/plugin.exp: Run all the tests serially
by the first parallel runtest encountering it.
* gcc.dg/plugin/plugin.exp: Likewise.
* gcc.misc-tests/matrix1.exp: Likewise.
* gcc.misc-tests/dhry.exp: Likewise.
* gcc.misc-tests/acker1.exp: Likewise.
* gcc.misc-tests/linkage.exp: Likewise.
* gcc.misc-tests/mg.exp: Likewise.
* gcc.misc-tests/mg-2.exp: Likewise.
* gcc.misc-tests/sort2.exp: Likewise.
* gcc.misc-tests/sieve.exp: Likewise.
* gcc.misc-tests/options.exp: Likewise.
* gcc.misc-tests/help.exp: Likewise.
* go.test/go-test.exp (go-gc-tests): Use
gcc_parallel_test_enable {0, 1} around all handling of
each test.
* objc.dg/gnu-encoding/gnu-encoding.exp: Likewise.
* ada/acats/run_all.sh: Parallelize using mkdir inside of
a shared directory. Avoid forking and expr uses if
shell supports $(()).
libstdc++-v3/
* testsuite/Makefile.am (check_p_numbers0, check_p_numbers1,
check_p_numbers2, check_p_numbers3, check_p_numbers4,
check_p_numbers5, check_p_numbers6, check_p_numbers,
check_p_subdirs): New variables.
(check_DEJAGNU_normal_targets): Use check_p_subdirs.
(check-DEJAGNU): Rewritten so that for parallelized
testing each job runs all the *.exp files, with
GCC_RUNTEST_PARALLELIZE_DIR set in environment.
* testsuite/Makefile.in: Regenerated.
* testsuite/lib/libstdc++.exp (gcc_parallel_test_run_p,
gcc_parallel_test_enable): New procedures. If
GCC_RUNTEST_PARALLELIZE_DIR is set in environment, override
runtest_file_p to invoke also gcc_parallel_test_run_p.
* testsuite/libstdc++-abi/abi.exp: Run all the tests serially
by the first parallel runtest encountering it. Fix up path
of the extract_symvers script.
* testsuite/libstdc++-xmethods/xmethods.exp: Run all the tests
serially by the first parallel runtest encountering it. Run
dg-finish even in case of error.
--- gcc/Makefile.in.jj 2014-09-14 21:09:04.612498827 +0200
+++ gcc/Makefile.in 2014-09-14 21:30:27.509141045 +0200
@@ -513,34 +513,10 @@ xm_include_list=@xm_include_list@
xm_defines=@xm_defines@
lang_checks=
lang_checks_parallelized=
-dg_target_exps:=aarch64.exp,alpha.exp,arm.exp,avr.exp,bfin.exp,cris.exp
-dg_target_exps:=$(dg_target_exps),epiphany.exp,frv.exp,i386.exp,ia64.exp
-dg_target_exps:=$(dg_target_exps),m68k.exp,microblaze.exp,mips.exp,powerpc.exp
-dg_target_exps:=$(dg_target_exps),rx.exp,s390.exp,sh.exp,sparc.exp,spu.exp
-dg_target_exps:=$(dg_target_exps),tic6x.exp,xstormy16.exp
-# This lists a couple of test files that take most time during check-gcc.
-# When doing parallelized check-gcc, these can run in parallel with the
-# remaining tests. Each word in this variable stands for work for one
-# make goal and one extra make goal is added to handle all the *.exp
-# files not handled explicitly already. If multiple *.exp files
-# should be run in the same runtest invocation (usually if they aren't
-# very long running, but still should be split of from the check-parallel-$lang
-# remaining tests runtest invocation), they should be concatenated with commas.
-# Note that [a-zA-Z] wildcards need to have []s prefixed with \ (needed
-# by tcl) and as the *.exp arguments are mached both as is and with
-# */ prefixed to it in runtest_file_p, it is usually desirable to include
-# a subdirectory name.
-check_gcc_parallelize=execute.exp=execute/2* \
- execute.exp=execute/\[013-9a-fA-F\]* \
- execute.exp=execute/\[pP\]*,dg.exp \
-
execute.exp=execute/\[g-oq-zG-OQ-Z\]*,compile.exp=compile/2* \
- compile.exp=compile/\[9pP\]*,builtins.exp \
- compile.exp=compile/\[013-8a-oq-zA-OQ-Z\]* \
- dg-torture.exp,ieee.exp \
- vect.exp,unsorted.exp \
- guality.exp \
- struct-layout-1.exp,stackalign.exp \
- $(dg_target_exps)
+# Upper limit to which it is useful to parallelize this lang target.
+# It doesn't make sense to try e.g. 128 goals for small testsuites
+# like objc or go.
+check_gcc_parallelize=10000
lang_opt_files=@lang_opt_files@ $(srcdir)/c-family/c.opt $(srcdir)/common.opt
lang_specs_files=@lang_specs_files@
lang_tree_files=@lang_tree_files@
@@ -3631,27 +3607,34 @@ $(filter-out $(lang_checks_parallelized)
export TCL_LIBRARY ; fi ; \
$(RUNTEST) --tool $* $(RUNTESTFLAGS))
-$(patsubst %,%-subtargets,$(filter-out
$(lang_checks_parallelized),$(lang_checks))): check-%-subtargets:
+$(patsubst %,%-subtargets,$(lang_checks)): check-%-subtargets:
@echo check-$*
check_p_tool=$(firstword $(subst _, ,$*))
-check_p_vars=$(check_$(check_p_tool)_parallelize)
+check_p_count=$(check_$(check_p_tool)_parallelize)
check_p_subno=$(word 2,$(subst _, ,$*))
-check_p_comma=,
-check_p_subwork=$(subst $(check_p_comma), ,$(if $(check_p_subno),$(word
$(check_p_subno),$(check_p_vars))))
-check_p_numbers=1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
+check_p_numbers0:=1 2 3 4 5 6 7 8 9
+check_p_numbers1:=0 $(check_p_numbers0)
+check_p_numbers2:=$(foreach i,$(check_p_numbers0),$(addprefix
$(i),$(check_p_numbers1)))
+check_p_numbers3:=$(addprefix 0,$(check_p_numbers1)) $(check_p_numbers2)
+check_p_numbers4:=$(foreach i,$(check_p_numbers0),$(addprefix
$(i),$(check_p_numbers3)))
+check_p_numbers5:=$(addprefix 0,$(check_p_numbers3)) $(check_p_numbers4)
+check_p_numbers6:=$(foreach i,$(check_p_numbers0),$(addprefix
$(i),$(check_p_numbers5)))
+check_p_numbers:=$(check_p_numbers0) $(check_p_numbers2) $(check_p_numbers4)
$(check_p_numbers6)
check_p_subdir=$(subst _,,$*)
-check_p_subdirs=$(wordlist 1,$(words
$(check_$*_parallelize)),$(check_p_numbers))
+check_p_subdirs=$(wordlist 1,$(check_p_count),$(wordlist 1, \
+ $(if
$(GCC_TEST_PARALLEL_SLOTS),$(GCC_TEST_PARALLEL_SLOTS),128), \
+ $(check_p_numbers)))
# For parallelized check-% targets, this decides whether parallelization
# is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
# but optional --target_board or --extra_opts arguments). If desirable,
# recursive make is run with check-parallel-$lang{,1,2,3,4,5} etc. goals,
# which can be executed in parallel, as they are run in separate directories.
-# check-parallel-$lang{1,2,3,4,5} etc. goals invoke runtest with the longest
-# running *.exp files from the testsuite, as determined by
check_$lang_parallelize
-# variable. The check-parallel-$lang goal in that case invokes runtest with
-# all the remaining *.exp files not handled by the separate goals.
+# check-parallel-$lang{,1,2,3,4,5} etc. goals invoke runtest with
+# GCC_RUNTEST_PARALLELIZE_DIR var in the environment and runtest_file_p
+# dejaGNU procedure is overridden to additionally synchronize through
+# a $lang-parallel directory which tests will be run by which runtest instance.
# Afterwards contrib/dg-extract-results.sh is used to merge the sum and log
# files. If parallelization isn't desirable, only one recursive make
# is run with check-parallel-$lang goal and check_$lang_parallelize variable
@@ -3662,76 +3645,60 @@ check_p_subdirs=$(wordlist 1,$(words $(c
# to lang_checks_parallelized variable and define check_$lang_parallelize
# variable (see above check_gcc_parallelize description).
$(lang_checks_parallelized): check-% : site.exp
- @if [ -z "$(filter-out --target_board=%,$(filter-out
--extra_opts%,$(RUNTESTFLAGS)))" ] \
- && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
+ -rm -rf $(TESTSUITEDIR)/$*-parallel
+ @if [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
+ test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR) || true; \
+ test -d $(TESTSUITEDIR)/$*-parallel || mkdir
$(TESTSUITEDIR)/$*-parallel || true; \
+
GCC_RUNTEST_PARALLELIZE_DIR=`${PWD_COMMAND}`/$(TESTSUITEDIR)/$(check_p_tool)-parallel
; \
+ export GCC_RUNTEST_PARALLELIZE_DIR ; \
$(MAKE) TESTSUITEDIR="$(TESTSUITEDIR)" RUNTESTFLAGS="$(RUNTESTFLAGS)"
\
check-parallel-$* \
$(patsubst %,check-parallel-$*_%, $(check_p_subdirs)); \
- for file in $(TESTSUITEDIR)/$*/$* \
- $(patsubst %,$(TESTSUITEDIR)/$*%/$*,$(check_p_subdirs));\
+ sums= ; logs= ; \
+ for dir in $(TESTSUITEDIR)/$* \
+ $(patsubst %,$(TESTSUITEDIR)/$*%,$(check_p_subdirs));\
do \
- mv -f $$file.sum $$file.sum.sep; mv -f $$file.log $$file.log.sep; \
+ if [ -d $$dir ]; then \
+ mv -f $$dir/$*.sum $$dir/$*.sum.sep; mv -f $$dir/$*.log
$$dir/$*.log.sep; \
+ sums="$$sums $$dir/$*.sum.sep"; logs="$$logs $$dir/$*.log.sep"; \
+ fi; \
done; \
- $(SHELL) $(srcdir)/../contrib/dg-extract-results.sh \
- $(TESTSUITEDIR)/$*/$*.sum.sep \
- $(patsubst %,$(TESTSUITEDIR)/$*%/$*.sum.sep,$(check_p_subdirs)) \
+ $(SHELL) $(srcdir)/../contrib/dg-extract-results.sh $$sums \
> $(TESTSUITEDIR)/$*/$*.sum; \
- $(SHELL) $(srcdir)/../contrib/dg-extract-results.sh -L \
- $(TESTSUITEDIR)/$*/$*.log.sep \
- $(patsubst %,$(TESTSUITEDIR)/$*%/$*.log.sep,$(check_p_subdirs)) \
+ $(SHELL) $(srcdir)/../contrib/dg-extract-results.sh -L $$logs \
> $(TESTSUITEDIR)/$*/$*.log; \
+ rm -rf $(TESTSUITEDIR)/$*-parallel || true; \
else \
$(MAKE) TESTSUITEDIR="$(TESTSUITEDIR)" RUNTESTFLAGS="$(RUNTESTFLAGS)"
\
check_$*_parallelize= check-parallel-$*; \
fi
-# Just print the parallelized subtargets for those that want to split
-# the testing across machines.
-$(patsubst %,%-subtargets,$(lang_checks_parallelized)): check-%-subtargets:
- @echo check-parallel-$* \
- $(patsubst %,check-parallel-$*_%, $(check_p_subdirs))
-
-# In the if [ -n "$(check_p_subno)" ] case runtest should be given the name of
-# the given *.exp file(s). See comment above check_gcc_parallelize variable
-# for details on the content of these variables.
-#
-# In the elif [ -n "$(check_p_vars)" ] case runtest should be given
-# names of all the *.exp files for this tool that aren't already handled by
-# other goals. First it finds all the *.exp files for this tool, then
-# prunes those already specified in check_$lang_parallelize or duplicates.
-#
-# Otherwise check-$lang isn't parallelized and runtest is invoked just with
-# the $(RUNTESTFLAGS) arguments.
check-parallel-% : site.exp
-test -d plugin || mkdir plugin
-test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir
$(TESTSUITEDIR)/$(check_p_subdir)
-(rootme=`${PWD_COMMAND}`; export rootme; \
srcdir=`cd ${srcdir}; ${PWD_COMMAND}` ; export srcdir ; \
- cd $(TESTSUITEDIR)/$(check_p_subdir); \
- rm -f tmp-site.exp; \
- sed '/set tmpdir/ s|testsuite$$|$(TESTSUITEDIR)/$(check_p_subdir)|' \
+ if [ -n "$(check_p_subno)" ] \
+ && [ -n "$$GCC_RUNTEST_PARALLELIZE_DIR" ] \
+ && [ -f $(TESTSUITEDIR)/$(check_p_tool)-parallel/finished ]; then \
+ rm -rf $(TESTSUITEDIR)/$(check_p_subdir); \
+ else \
+ cd $(TESTSUITEDIR)/$(check_p_subdir); \
+ rm -f tmp-site.exp; \
+ sed '/set tmpdir/ s|testsuite$$|$(TESTSUITEDIR)/$(check_p_subdir)|' \
< ../../site.exp > tmp-site.exp; \
- $(SHELL) $${srcdir}/../move-if-change tmp-site.exp site.exp; \
- EXPECT=${EXPECT} ; export EXPECT ; \
- if [ -f $${rootme}/../expect/expect ] ; then \
- TCL_LIBRARY=`cd .. ; cd $${srcdir}/../tcl/library ; ${PWD_COMMAND}`
; \
- export TCL_LIBRARY ; fi ; \
- runtestflags= ; \
- if [ -n "$(check_p_subno)" ] ; then \
- runtestflags="$(check_p_subwork)"; \
- elif [ -n "$(check_p_vars)" ] ; then \
- parts="`echo ' $(strip $(subst $(check_p_comma), ,$(check_p_vars))) '
\
- | sed 's/=[^ ]* / /g'`"; \
- for part in `find $$srcdir/testsuite/$(check_p_tool)* -name \*.exp` ;
do \
- part=`basename $$part` ; \
- case " $$parts $$runtestflags " in \
- *" $$part "*) ;; \
- *) runtestflags="$$runtestflags $$part" ;; \
- esac ; \
- done ; \
- fi ; \
- $(RUNTEST) --tool $(check_p_tool) $(RUNTESTFLAGS) $$runtestflags)
+ $(SHELL) $${srcdir}/../move-if-change tmp-site.exp site.exp; \
+ EXPECT=${EXPECT} ; export EXPECT ; \
+ if [ -f $${rootme}/../expect/expect ] ; then \
+ TCL_LIBRARY=`cd .. ; cd $${srcdir}/../tcl/library ; ${PWD_COMMAND}`
; \
+ export TCL_LIBRARY ; \
+ fi ; \
+ $(RUNTEST) --tool $(check_p_tool) $(RUNTESTFLAGS); \
+ if [ -n "$$GCC_RUNTEST_PARALLELIZE_DIR" ] ; then \
+ touch $${rootme}/$(TESTSUITEDIR)/$(check_p_tool)-parallel/finished;
\
+ fi ; \
+ fi )
# QMTest targets
--- gcc/go/Make-lang.in.jj 2014-09-14 21:09:05.744493198 +0200
+++ gcc/go/Make-lang.in 2014-09-14 21:25:17.767673249 +0200
@@ -131,11 +131,7 @@ go.srcman: doc/gccgo.1
lang_checks += check-go
lang_checks_parallelized += check-go
-check_go_parallelize = go-test.exp=*/test/\[0-57-9a-bd-hj-qs-zA-Z\]* \
- go-test.exp=*/test/c* \
- go-test.exp=*/test/i* \
- go-test.exp=*/test/r* \
- go-test.exp=*/test/6*
+check_go_parallelize = 10
# Install hooks.
--- gcc/fortran/Make-lang.in.jj 2014-09-14 21:09:04.560499080 +0200
+++ gcc/fortran/Make-lang.in 2014-09-14 21:25:17.768673222 +0200
@@ -168,12 +168,7 @@ check-fortran-subtargets : check-gfortra
lang_checks += check-gfortran
lang_checks_parallelized += check-gfortran
# For description see comment above check_gcc_parallelize in gcc/Makefile.in.
-check_gfortran_parallelize = dg.exp=gfortran.dg/\[adAD\]* \
- dg.exp=gfortran.dg/\[bcBC\]* \
- dg.exp=gfortran.dg/\[nopNOP\]* \
- dg.exp=gfortran.dg/\[isuvISUV\]* \
- dg.exp=gfortran.dg/\[efhkqrxzEFHKQRXZ\]* \
- dg.exp=gfortran.dg/\[0-9gjlmtwyGJLMTWY\]*
+check_gfortran_parallelize = 10000
# GFORTRAN documentation.
GFORTRAN_TEXI = \
--- gcc/cp/Make-lang.in.jj 2014-09-14 21:09:05.729493272 +0200
+++ gcc/cp/Make-lang.in 2014-09-14 21:25:17.768673222 +0200
@@ -156,11 +156,7 @@ check-c++-subtargets : check-g++-subtarg
lang_checks += check-g++
lang_checks_parallelized += check-g++
# For description see comment above check_gcc_parallelize in gcc/Makefile.in.
-check_g++_parallelize = old-deja.exp \
- dg.exp=g++.dg/[0-9A-Za-bd-su-z]* \
- dg.exp=g++.dg/[ct]* \
- dg.exp=c-c++-common/*,dg-torture.exp
-
+check_g++_parallelize = 10000
#
# Install hooks:
# cc1plus is installed elsewhere as part of $(COMPILERS).
--- gcc/ada/gcc-interface/Make-lang.in.jj 2014-08-04 10:43:27.000000000
+0200
+++ gcc/ada/gcc-interface/Make-lang.in 2014-09-15 14:38:30.250714169 +0200
@@ -862,7 +862,18 @@ check-ada-subtargets: check-acats-subtar
ACATSDIR = $(TESTSUITEDIR)/ada/acats
-check_acats_targets = $(patsubst %,check-acats%, 0 1 2)
+check_acats_numbers0:=1 2 3 4 5 6 7 8 9
+check_acats_numbers1:=0 $(check_acats_numbers0)
+check_acats_numbers2:=$(foreach i,$(check_acats_numbers0),$(addprefix
$(i),$(check_acats_numbers1)))
+check_acats_numbers3:=$(addprefix 0,$(check_acats_numbers1))
$(check_acats_numbers2)
+check_acats_numbers4:=$(foreach i,$(check_acats_numbers0),$(addprefix
$(i),$(check_acats_numbers3)))
+check_acats_numbers5:=$(addprefix 0,$(check_acats_numbers3))
$(check_acats_numbers4)
+check_acats_numbers6:=$(foreach i,$(check_acats_numbers0),$(addprefix
$(i),$(check_acats_numbers5)))
+check_acats_numbers:=$(check_acats_numbers0) $(check_acats_numbers2)
$(check_acats_numbers4) $(check_acats_numbers6)
+check_acats_subdirs=$(wordlist 1, \
+ $(if
$(GCC_TEST_PARALLEL_SLOTS),$(GCC_TEST_PARALLEL_SLOTS),128), \
+ $(check_acats_numbers))
+check_acats_targets = $(addprefix check-acats, $(check_acats_subdirs))
check-acats:
@test -d $(ACATSDIR) || mkdir -p $(ACATSDIR); \
@@ -870,35 +881,45 @@ check-acats:
EXPECT=$(EXPECT); export EXPECT; \
if [ -z "$(CHAPTERS)" ] && [ "$(filter -j, $(MFLAGS))" = "-j" ]; \
then \
+ rm -rf $(ACATSDIR)-parallel; \
+ mkdir $(ACATSDIR)-parallel; \
+ ( testdir=`cd ${srcdir}/${ACATSDIR} && ${PWD_COMMAND}`; \
+ export testdir; \
+ cd $(ACATSDIR) && $(SHELL) $${testdir}/run_acats NONE ) \
+ || exit 1; \
+ GCC_RUNTEST_PARALLELIZE_DIR=$$rootme/$(ACATSDIR)-parallel; \
+ export GCC_RUNTEST_PARALLELIZE_DIR; \
$(MAKE) $(check_acats_targets); \
- for idx in 0 1 2; do \
+ rm -rf $(ACATSDIR)-parallel; \
+ acats_sums= ; acats_logs= ; \
+ for idx in "" $(check_acats_subdirs); do \
+ [ -d $(ACATSDIR)$$idx ] || continue; \
mv -f $(ACATSDIR)$$idx/acats.sum $(ACATSDIR)$$idx/acats.sum.sep; \
mv -f $(ACATSDIR)$$idx/acats.log $(ACATSDIR)$$idx/acats.log.sep; \
+ acats_sums="$$acats_sums $(ACATSDIR)$$idx/acats.sum.sep"; \
+ acats_logs="$$acats_logs $(ACATSDIR)$$idx/acats.log.sep"; \
done; \
$(SHELL) $(srcdir)/../contrib/dg-extract-results.sh \
- $(ACATSDIR)0/acats.sum.sep $(ACATSDIR)1/acats.sum.sep \
- $(ACATSDIR)2/acats.sum.sep > $(ACATSDIR)/acats.sum; \
+ $$acats_sums > $(ACATSDIR)/acats.sum; \
$(SHELL) $(srcdir)/../contrib/dg-extract-results.sh -L \
- $(ACATSDIR)0/acats.log.sep $(ACATSDIR)1/acats.log.sep \
- $(ACATSDIR)2/acats.log.sep > $(ACATSDIR)/acats.log; \
+ $$acats_logs > $(ACATSDIR)/acats.log; \
exit 0; \
fi; \
testdir=`cd ${srcdir}/${ACATSDIR} && ${PWD_COMMAND}`; \
export testdir; cd $(ACATSDIR) && $(SHELL) $${testdir}/run_acats
$(CHAPTERS)
check-acats-subtargets:
- @echo $(check_acats_targets)
+ @echo check-acats
# Parallelized check-acats
$(check_acats_targets): check-acats%:
+ if [ -f $$GCC_RUNTEST_PARALLELIZE_DIR/finished ]; then \
+ rm -rf $(ACATSDIR)$*; exit 0; \
+ fi; \
test -d $(ACATSDIR)$* || mkdir -p $(ACATSDIR)$*; \
testdir=`cd ${srcdir}/${ACATSDIR} && ${PWD_COMMAND}`; \
- case "$*" in \
- 0) chapters="`cd $$testdir/tests; echo [a-b]* c[0-4]*`";; \
- 1) chapters="`cd $$testdir/tests; echo c[5-9ab]*`";; \
- 2) chapters="`cd $$testdir/tests; echo c[c-z]* [d-z]*`";; \
- esac; \
- export testdir; cd $(ACATSDIR)$* && $(SHELL) $${testdir}/run_acats
$$chapters
+ export testdir; cd $(ACATSDIR)$* && $(SHELL) $${testdir}/run_acats
+ touch $$GCC_RUNTEST_PARALLELIZE_DIR/finished
.PHONY: check-acats $(check_acats_targets)
--- gcc/testsuite/gcc.misc-tests/sieve.exp.jj 2014-09-14 21:09:05.668493574
+0200
+++ gcc/testsuite/gcc.misc-tests/sieve.exp 2014-09-14 21:25:17.773673082
+0200
@@ -20,9 +20,21 @@ if { ![info exists PERF_TEST] || "$PERF_
}
load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file. The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p sieve] {
+ return
+}
+gcc_parallel_test_enable 0
prebase
set actions run
set compiler_output "^$"
set program_output "^$"
postbase sieve.c $run $groups
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/sort2.exp.jj 2014-09-14 21:09:05.524494295
+0200
+++ gcc/testsuite/gcc.misc-tests/sort2.exp 2014-09-14 21:25:17.773673082
+0200
@@ -20,9 +20,21 @@ if { ![info exists PERF_TEST] || "$PERF_
}
load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file. The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p sort2] {
+ return
+}
+gcc_parallel_test_enable 0
prebase
set actions run
set compiler_output "^$"
set program_output "^$"
postbase sort2.c $run $groups
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/matrix1.exp.jj 2014-09-14 21:09:05.544494191
+0200
+++ gcc/testsuite/gcc.misc-tests/matrix1.exp 2014-09-14 21:25:17.772673116
+0200
@@ -20,9 +20,21 @@ if { ![info exists PERF_TEST] || "$PERF_
}
load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file. The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p matrix1] {
+ return
+}
+gcc_parallel_test_enable 0
prebase
set actions run
set compiler_output "^$"
set program_output "^$"
postbase matrix1.c $run $groups
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/dhry.exp.jj 2014-09-14 21:09:05.596493933
+0200
+++ gcc/testsuite/gcc.misc-tests/dhry.exp 2014-09-14 21:25:17.772673116
+0200
@@ -20,9 +20,21 @@ if { ![info exists PERF_TEST] || "$PERF_
}
load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file. The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p dhry] {
+ return
+}
+gcc_parallel_test_enable 0
prebase
set actions run
set compiler_output "^$"
set program_output "^$"
postbase dhry.c $run $groups
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/mg.exp.jj 2014-09-14 21:09:05.580494013
+0200
+++ gcc/testsuite/gcc.misc-tests/mg.exp 2014-09-14 21:25:17.772673116 +0200
@@ -17,8 +17,20 @@
# Test the -MG flag.
load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file. The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p mg] {
+ return
+}
+gcc_parallel_test_enable 0
prebase
set actions none-of-the-above
set compiler_output "mg.o ?: .*mg.c \[ \\\\\n\]*nonexist.h"
postbase mg.c "" "" "-MM -MG"
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/options.exp.jj 2014-09-14 21:09:05.563494097
+0200
+++ gcc/testsuite/gcc.misc-tests/options.exp 2014-09-14 21:25:17.773673082
+0200
@@ -19,6 +19,17 @@
# match the patterns COMPILER_PATTERN, AS_PATTERN and LD_PATTERN,
# respectively.
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file. The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p options] {
+ return
+}
+gcc_parallel_test_enable 0
+
proc check_for_all_options {language gcc_options compiler_pattern as_pattern
ld_pattern} {
set filename test-[pid]
set fd [open $filename.c w]
@@ -52,3 +63,5 @@ proc check_for_all_options {language gcc
}
check_for_all_options c {--coverage} {-fprofile-arcs -ftest-coverage} {}
{-lgcov}
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/help.exp.jj 2014-09-14 21:09:05.555494137
+0200
+++ gcc/testsuite/gcc.misc-tests/help.exp 2014-09-14 21:25:17.774673051
+0200
@@ -18,6 +18,16 @@
# documented in --help, and that the various --help* options work.
load_lib options.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file. The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p help] {
+ return
+}
+gcc_parallel_test_enable 0
# Document --version. Ideally, there should be no undocumented switches
# in --help.
@@ -78,3 +88,5 @@ check_for_options c "--help=joined,^sepa
check_for_options c "--help=joined,undocumented" "" "" ""
# Listing only excludes gives empty results.
check_for_options c "--help=^joined,^separate" "" "" ""
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/linkage.exp.jj 2014-09-14 21:09:05.624493790
+0200
+++ gcc/testsuite/gcc.misc-tests/linkage.exp 2014-09-14 21:25:17.772673116
+0200
@@ -18,6 +18,17 @@
# was written by Rob Savoye. (r...@cygnus.com)
# All the other tests driven by that file have since been moved elsewhere.
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file. The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p linkage] {
+ return
+}
+gcc_parallel_test_enable 0
+
if { [isnative] && ![is_remote host] } then {
set lines [gcc_target_compile "$srcdir/$subdir/linkage-x.c" "linkage-x.o"
object {additional_flags="-w"}]
if ![string match "" $lines] then {
@@ -117,3 +128,5 @@ if { [isnative] && ![is_remote host] } t
file delete "linkage-x.o"
}
}
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/acker1.exp.jj 2014-09-14 21:09:05.586493983
+0200
+++ gcc/testsuite/gcc.misc-tests/acker1.exp 2014-09-14 21:25:17.772673116
+0200
@@ -20,9 +20,21 @@ if { ![info exists PERF_TEST] || "$PERF_
}
load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file. The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p acker1] {
+ return
+}
+gcc_parallel_test_enable 0
prebase
set actions run
set compiler_output "^$"
set program_output "^$"
postbase acker1.c $run $groups
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/mg-2.exp.jj 2014-09-14 21:09:05.570494062
+0200
+++ gcc/testsuite/gcc.misc-tests/mg-2.exp 2014-09-14 21:25:17.773673082
+0200
@@ -17,8 +17,20 @@
# Test the -MG flag with a system header file.
load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file. The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p mg-2] {
+ return
+}
+gcc_parallel_test_enable 0
prebase
set actions none-of-the-above
set compiler_output "mg-2.o ?: .*mg-2.c \[ \\\\\n\]*nonexist.h"
postbase mg-2.c "" "" "-MM -MG"
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/g++.dg/guality/guality.exp.jj 2014-09-14 21:09:04.767498056
+0200
+++ gcc/testsuite/g++.dg/guality/guality.exp 2014-09-14 21:25:17.768673222
+0200
@@ -14,6 +14,11 @@ if { [istarget "powerpc-ibm-aix*"] } {
}
proc check_guality {args} {
+ # Don't count check_guality as PASS, or FAIL etc., that would make
+ # the total PASS count dependent on how many parallel runtest invocations
+ # ran guality.exp. So save the counts first and restore them afterwards.
+ global test_counts
+ array set saved_test_counts [array get test_counts]
set result [eval check_compile guality_check executable $args "-g -O0"]
set lines [lindex $result 0]
set output [lindex $result 1]
@@ -23,6 +28,7 @@ proc check_guality {args} {
set ret [string match "*1 PASS, 0 FAIL, 0 UNRESOLVED*" $execout]
}
remote_file build delete $output
+ array get test_counts [array get saved_test_counts]
return $ret
}
--- gcc/testsuite/g++.dg/plugin/plugin.exp.jj 2014-09-14 21:09:04.824497773
+0200
+++ gcc/testsuite/g++.dg/plugin/plugin.exp 2014-09-14 21:25:17.769673197
+0200
@@ -44,6 +44,15 @@ if $tracelevel then {
# Load support procs.
load_lib plugin-support.exp
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file. The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p plugin] {
+ return
+}
+gcc_parallel_test_enable 0
+
# Specify the plugin source file and the associated test files in a list.
# plugin_test_list={ {plugin1 test1 test2 ...} {plugin2 test1 ...} ... }
set plugin_test_list [list \
@@ -68,3 +77,5 @@ foreach plugin_test $plugin_test_list {
set plugin_input_tests [lreplace $plugin_test 0 0]
plugin-test-execute $plugin_src $plugin_input_tests
}
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.dg/guality/guality.exp.jj 2014-09-14 21:09:05.362495088
+0200
+++ gcc/testsuite/gcc.dg/guality/guality.exp 2014-09-14 21:25:17.769673197
+0200
@@ -14,6 +14,11 @@ if { [istarget "powerpc-ibm-aix*"] } {
}
proc check_guality {args} {
+ # Don't count check_guality as PASS, or FAIL etc., that would make
+ # the total PASS count dependent on how many parallel runtest invocations
+ # ran guality.exp. So save the counts first and restore them afterwards.
+ global test_counts
+ array set saved_test_counts [array get test_counts]
set result [eval check_compile guality_check executable $args "-g -O0"]
set lines [lindex $result 0]
set output [lindex $result 1]
@@ -23,6 +28,7 @@ proc check_guality {args} {
set ret [string match "*1 PASS, 0 FAIL, 0 UNRESOLVED*" $execout]
}
remote_file build delete $output
+ array set test_counts [array get saved_test_counts]
return $ret
}
--- gcc/testsuite/gcc.dg/plugin/plugin.exp.jj 2014-09-14 21:09:05.470494558
+0200
+++ gcc/testsuite/gcc.dg/plugin/plugin.exp 2014-09-14 21:25:17.770673173
+0200
@@ -44,6 +44,15 @@ if $tracelevel then {
# Load support procs.
load_lib plugin-support.exp
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file. The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p plugin] {
+ return
+}
+gcc_parallel_test_enable 0
+
# Specify the plugin source file and the associated test files in a list.
# plugin_test_list={ {plugin1 test1 test2 ...} {plugin2 test1 ...} ... }
set plugin_test_list [list \
@@ -80,3 +89,5 @@ dg-runtest [lsort [glob -nocomplain $src
# All done.
dg-finish
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/go.test/go-test.exp.jj 2014-09-14 21:09:05.292495444
+0200
+++ gcc/testsuite/go.test/go-test.exp 2014-09-14 21:25:17.770673173 +0200
@@ -489,6 +489,12 @@ proc go-gc-tests { } {
close $fd
+ # runtest_file_p is already run above, and the code below can run
+ # runtest_file_p again, make sure everything for this test is
+ # performed if the above runtest_file_p decided this runtest
+ # instance should execute the test
+ gcc_parallel_test_enable 0
+
set go_compile_args ""
set go_execute_args ""
if { [regexp "// run (\[^|&>2\].*)\$" $test_line match progargs] \
@@ -1164,6 +1170,7 @@ proc go-gc-tests { } {
set go_compile_args ""
set go_execute_args ""
set TORTURE_OPTIONS [list { -O2 -g }]
+ gcc_parallel_test_enable 1
}
set dg-do-what-default ${saved-dg-do-what-default}
--- gcc/testsuite/ada/acats/run_all.sh.jj 2013-06-01 10:28:28.100138045
+0200
+++ gcc/testsuite/ada/acats/run_all.sh 2014-09-15 15:33:15.793502393 +0200
@@ -18,6 +18,21 @@ target_run () {
# End of customization section.
+# Perform arithmetic evaluation on the ARGs, and store the result in the
+# global $as_val. Take advantage of shells that can avoid forks. The arguments
+# must be portable across $(()) and expr.
+if (eval "test \$(( 1 + 1 )) = 2") 2>/dev/null; then :
+ eval 'as_fn_arith ()
+ {
+ as_val=$(( $* ))
+ }'
+else
+ as_fn_arith ()
+ {
+ as_val=`expr "$@" || test $? -eq 1`
+ }
+fi # as_fn_arith
+
display_noeol () {
printf "$@"
printf "$@" >> $dir/acats.sum
@@ -90,6 +105,18 @@ display `type gnatmake`
gnatls -v >> $dir/acats.log
display ""
+if [ -n "$GCC_RUNTEST_PARALLELIZE_DIR" ]; then
+ dir_support=$dir/../acats/support
+
+ rm -rf $dir/run
+ mv $dir/tests $dir/tests.$$ 2> /dev/null
+ rm -rf $dir/tests.$$ &
+ mkdir -p $dir/run
+
+ cp -pr $dir/../acats/tests $dir/
+else
+ dir_support=$dir/support
+
display " === acats support ==="
display_noeol "Generating support files..."
@@ -186,6 +213,9 @@ target_gnatmake -c -gnato -gnatE *.adb >
display " done."
display ""
+
+fi
+
display " === acats tests ==="
if [ $# -eq 0 ]; then
@@ -197,8 +227,14 @@ fi
glob_countn=0
glob_countok=0
glob_countu=0
+par_count=0
+par_countm=0
+par_last=
for chapter in $chapters; do
+ # Used to generate support once and finish after that.
+ [ "$chapter" = "NONE" ] && continue
+
display Running chapter $chapter ...
if [ ! -d $dir/tests/$chapter ]; then
@@ -209,13 +245,43 @@ for chapter in $chapters; do
cd $dir/tests/$chapter
ls *.a *.ada *.adt *.am *.dep 2> /dev/null | sed -e 's/\(.*\)\..*/\1/g' | \
- cut -c1-7 | sort | uniq | comm -23 - $dir/support/norun.lst \
+ cut -c1-7 | sort | uniq | comm -23 - $dir_support/norun.lst \
> $dir/tests/$chapter/${chapter}.lst
countn=`wc -l < $dir/tests/$chapter/${chapter}.lst`
- glob_countn=`expr $glob_countn + $countn`
- counti=0
+ as_fn_arith $glob_countn + $countn
+ glob_countn=$as_val
for i in `cat $dir/tests/$chapter/${chapter}.lst`; do
- counti=`expr $counti + 1`
+
+ # If running multiple run_all.sh jobs in parallel, decide
+ # if we should run this test in the current instance.
+ if [ -n "$GCC_RUNTEST_PARALLELIZE_DIR" ]; then
+ case "$i" in
+ # Ugh, some tests have inter-test dependencies, those
+ # tests have to be scheduled on the same parallel instance
+ # as previous test.
+ ce2108f | ce2108h | ce3112d) ;;
+ # All others can be hopefully scheduled freely.
+ *)
+ as_fn_arith $par_countm + 1
+ par_countm=$as_val
+ [ $par_countm -eq 10 ] && par_countm=0
+ if [ $par_countm -eq 1 ]; then
+ as_fn_arith $par_count + 1
+ par_count=$as_val
+ if mkdir $GCC_RUNTEST_PARALLELIZE_DIR/$par_count; then
+ par_last=1
+ else
+ par_last=
+ fi
+ fi;;
+ esac
+ if [ -z "$par_last" ]; then
+ as_fn_arith $glob_countn - 1
+ glob_countn=$as_val
+ continue
+ fi
+ fi
+
extraflags="-gnat95"
grep $i $testdir/overflow.lst > /dev/null 2>&1
if [ $? -eq 0 ]; then
@@ -254,7 +320,7 @@ for chapter in $chapters; do
echo "BUILD $main" >> $dir/acats.log
EXTERNAL_OBJECTS=""
case $i in
- cxb30*) EXTERNAL_OBJECTS="$dir/support/cxb30040.o
$dir/support/cxb30060.o $dir/support/cxb30130.o $dir/support/cxb30131.o";;
+ cxb30*) EXTERNAL_OBJECTS="$dir_support/cxb30040.o
$dir_support/cxb30060.o $dir_support/cxb30130.o $dir_support/cxb30131.o";;
ca1020e) rm -f ca1020e_func1.adb ca1020e_func2.adb ca1020e_proc1.adb
ca1020e_proc2.adb > /dev/null 2>&1;;
ca14028) rm -f ca14028_func2.ads ca14028_func3.ads ca14028_proc1.ads
ca14028_proc3.ads > /dev/null 2>&1;;
esac
@@ -265,7 +331,7 @@ for chapter in $chapters; do
continue
fi
- target_gnatmake $extraflags -I$dir/support $main >> $dir/acats.log 2>&1
+ target_gnatmake $extraflags -I$dir_support $main >> $dir/acats.log 2>&1
if [ $? -ne 0 ]; then
display "FAIL: $i"
failed="${failed}${i} "
@@ -290,12 +356,15 @@ for chapter in $chapters; do
failed="${failed}${i} "
else
log "UNSUPPORTED: $i"
- glob_countn=`expr $glob_countn - 1`
- glob_countu=`expr $glob_countu + 1`
+ as_fn_arith $glob_countn - 1
+ glob_countn=$as_val
+ as_fn_arith $glob_countu + 1
+ glob_countu=$as_val
fi
else
log "PASS: $i"
- glob_countok=`expr $glob_countok + 1`
+ as_fn_arith $glob_countok + 1
+ glob_countok=$as_val
fi
clean_dir
done
--- gcc/testsuite/lib/gcc-defs.exp.jj 2014-09-14 21:09:04.880497500 +0200
+++ gcc/testsuite/lib/gcc-defs.exp 2014-09-14 22:10:41.116165494 +0200
@@ -188,6 +188,81 @@ if { [info procs runtest_file_p] == "" }
}
}
+if { [info exists env(GCC_RUNTEST_PARALLELIZE_DIR)] \
+ && [info procs runtest_file_p] != [list] \
+ && [info procs gcc_parallelize_saved_runtest_file_p] == [list] } then {
+ global gcc_runtest_parallelize_counter
+ global gcc_runtest_parallelize_counter_minor
+ global gcc_runtest_parallelize_enable
+ global gcc_runtest_parallelize_dir
+ global gcc_runtest_parallelize_last
+
+ set gcc_runtest_parallelize_counter 0
+ set gcc_runtest_parallelize_counter_minor 0
+ set gcc_runtest_parallelize_enable 1
+ set gcc_runtest_parallelize_dir [getenv GCC_RUNTEST_PARALLELIZE_DIR]
+ set gcc_runtest_parallelize_last 0
+
+ proc gcc_parallel_test_run_p { testcase } {
+ global gcc_runtest_parallelize_counter
+ global gcc_runtest_parallelize_counter_minor
+ global gcc_runtest_parallelize_enable
+ global gcc_runtest_parallelize_dir
+ global gcc_runtest_parallelize_last
+
+ if { $gcc_runtest_parallelize_enable == 0 } {
+ return 1
+ }
+
+ # Only test the filesystem every 10th iteration
+ incr gcc_runtest_parallelize_counter_minor
+ if { $gcc_runtest_parallelize_counter_minor == 10 } {
+ set gcc_runtest_parallelize_counter_minor 0
+ }
+ if { $gcc_runtest_parallelize_counter_minor != 1 } {
+ #verbose -log "gcc_parallel_test_run_p $testcase
$gcc_runtest_parallelize_counter $gcc_runtest_parallelize_last"
+ return $gcc_runtest_parallelize_last
+ }
+
+ set path $gcc_runtest_parallelize_dir/$gcc_runtest_parallelize_counter
+
+ if {![catch {open $path {RDWR CREAT EXCL} 0600} fd]} {
+ close $fd
+ set gcc_runtest_parallelize_last 1
+ #verbose -log "gcc_parallel_test_run_p $testcase
$gcc_runtest_parallelize_counter 1"
+ incr gcc_runtest_parallelize_counter
+ return 1
+ }
+ set gcc_runtest_parallelize_last 0
+ #verbose -log "gcc_parallel_test_run_p $testcase
$gcc_runtest_parallelize_counter 0"
+ incr gcc_runtest_parallelize_counter
+ return 0
+ }
+
+ proc gcc_parallel_test_enable { val } {
+ global gcc_runtest_parallelize_enable
+ set gcc_runtest_parallelize_enable $val
+ }
+
+ rename runtest_file_p gcc_parallelize_saved_runtest_file_p
+ proc runtest_file_p { runtests testcase } {
+ if ![gcc_parallelize_saved_runtest_file_p $runtests $testcase] {
+ return 0
+ }
+ return [gcc_parallel_test_run_p $testcase]
+ }
+
+} else {
+
+ proc gcc_parallel_test_run_p { testcase } {
+ return 1
+ }
+
+ proc gcc_parallel_test_enable { val } {
+ }
+
+}
+
# Like dg-options, but adds to the default options rather than replacing them.
proc dg-additional-options { args } {
--- gcc/testsuite/objc.dg/gnu-encoding/gnu-encoding.exp.jj 2014-09-14
21:09:05.237495674 +0200
+++ gcc/testsuite/objc.dg/gnu-encoding/gnu-encoding.exp 2014-09-14
22:09:58.788381473 +0200
@@ -58,7 +58,9 @@ if { $status == 0 } then {
continue
}
+ gcc_parallel_test_enable 0
dg-runtest $src "" $DEFAULT_CFLAGS
+ gcc_parallel_test_enable 1
}
} else {
warning "Could not execute
objc.dg/gnu-encoding/struct-layout-encoding-1 generator"
--- gcc/objc/Make-lang.in.jj 2014-09-14 21:09:05.807492885 +0200
+++ gcc/objc/Make-lang.in 2014-09-14 21:25:17.771673151 +0200
@@ -96,7 +96,7 @@ lang_checks += check-objc
# The following allows you to do 'make check-objc -j2'. The
# execute.exp tests will be run in parallel with all the other ones.
lang_checks_parallelized += check-objc
-check_objc_parallelize = gnu-encoding.exp execute.exp exceptions.exp
+check_objc_parallelize = 6
#
# Install hooks:
--- libstdc++-v3/testsuite/libstdc++-abi/abi.exp.jj 2014-05-20
16:36:47.000000000 +0200
+++ libstdc++-v3/testsuite/libstdc++-abi/abi.exp 2014-09-15
14:23:06.308253078 +0200
@@ -18,6 +18,12 @@
# this test. Or, hey, if we don't support this kind of symbol
# versioning test: don't run it.
+# Only run whole abi.exp in one instance, serially.
+if ![gcc_parallel_test_run_p abi] {
+ return
+}
+gcc_parallel_test_enable 0
+
if { [string match "*-*-darwin*" $target_triplet] } {
set lib $blddir/src/.libs/libstdc++.dylib
} else {
@@ -30,6 +36,7 @@ v3-build_support
if { (${v3-symver} == 0) || ![info exists baseline_dir] \
|| ![file exists $baseline_dir] \
|| ![file exists $lib] } {
+ gcc_parallel_test_enable 1
return
}
@@ -42,12 +49,13 @@ if ![file exists $baseline_file] {
set baseline_file [file join $baseline_dir "baseline_symbols.txt"]
}
if ![file exists $baseline_file] {
+ gcc_parallel_test_enable 1
return
}
send_log "Checking $lib against $baseline_file\n"
# Figure out what symbols are defined by the active build of the library.
-remote_exec "build" "$objdir/../scripts/extract_symvers" \
+remote_exec "build" "$blddir/scripts/extract_symvers" \
[list $lib "current_symbols.txt"]
# Build the abi_check program.
@@ -62,3 +70,5 @@ set result [${tool}_load "./abi_check" \
[list "--check-verbose" "current_symbols.txt" \
"baseline_symbols.txt"]]
[lindex $result 0] "libstdc++-abi/abi_check"
+
+gcc_parallel_test_enable 1
--- libstdc++-v3/testsuite/libstdc++-xmethods/xmethods.exp.jj 2014-09-10
21:57:14.000000000 +0200
+++ libstdc++-v3/testsuite/libstdc++-xmethods/xmethods.exp 2014-09-15
15:31:50.277960368 +0200
@@ -16,6 +16,12 @@
load_lib gdb-test.exp
+# Only run whole xmethods.exp in one instance, serially.
+if ![gcc_parallel_test_run_p xmethods] {
+ return
+}
+gcc_parallel_test_enable 0
+
dg-init
v3-build_support
@@ -30,7 +36,9 @@ if ![info exists ::env(GUALITY_GDB_NAME)
}
if {! [gdb_version_check_xmethods]} {
+ dg-finish
unsupported "xmethods.exp"
+ gcc_parallel_test_enable 1
return
}
@@ -49,3 +57,4 @@ if [info exists guality_gdb_name] {
}
dg-finish
+gcc_parallel_test_enable 1
--- libstdc++-v3/testsuite/Makefile.am.jj 2014-09-14 21:08:57.067536354
+0200
+++ libstdc++-v3/testsuite/Makefile.am 2014-09-15 15:27:57.632110277 +0200
@@ -101,26 +101,37 @@ new-abi-baseline:
@test ! -f $*/site.exp || mv $*/site.exp $*/site.bak
@mv $*/site.exp.tmp $*/site.exp
-check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6
7 8 9 10)
+check_p_numbers0:=1 2 3 4 5 6 7 8 9
+check_p_numbers1:=0 $(check_p_numbers0)
+check_p_numbers2:=$(foreach i,$(check_p_numbers0),$(addprefix
$(i),$(check_p_numbers1)))
+check_p_numbers3:=$(addprefix 0,$(check_p_numbers1)) $(check_p_numbers2)
+check_p_numbers4:=$(foreach i,$(check_p_numbers0),$(addprefix
$(i),$(check_p_numbers3)))
+check_p_numbers5:=$(addprefix 0,$(check_p_numbers3)) $(check_p_numbers4)
+check_p_numbers6:=$(foreach i,$(check_p_numbers0),$(addprefix
$(i),$(check_p_numbers5)))
+check_p_numbers:=$(check_p_numbers0) $(check_p_numbers2) $(check_p_numbers4)
$(check_p_numbers6)
+check_p_subdirs=$(wordlist 1,$(if
$(GCC_TEST_PARALLEL_SLOTS),$(GCC_TEST_PARALLEL_SLOTS),128),$(check_p_numbers))
+check_DEJAGNU_normal_targets = $(addprefix
check-DEJAGNUnormal,$(check_p_subdirs))
$(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%: normal%/site.exp
# Run the testsuite in normal mode.
check-DEJAGNU $(check_DEJAGNU_normal_targets): check-DEJAGNU%: site.exp
AR="$(AR)"; export AR; \
RANLIB="$(RANLIB)"; export RANLIB; \
- if [ -z "$*$(filter-out --target_board=%, $(RUNTESTFLAGS))" ] \
- && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
+ if [ -z "$*" ] && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
+ rm -rf normal-parallel || true; \
+ mkdir normal-parallel; \
$(MAKE) $(AM_MAKEFLAGS) $(check_DEJAGNU_normal_targets); \
- for idx in 0 1 2 3 4 5 6 7 8 9 10; do \
- mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \
- mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \
+ rm -rf normal-parallel || true; \
+ for idx in $(check_p_subdirs); do \
+ if [ -d normal$$idx ]; then \
+ mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \
+ mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \
+ fi; \
done; \
- mv -f libstdc++.sum libstdc++.sum.sep; \
- mv -f libstdc++.log libstdc++.log.sep; \
$(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh \
- libstdc++.sum.sep normal[0-9]*/libstdc++.sum.sep > libstdc++.sum; \
+ normal[0-9]*/libstdc++.sum.sep > libstdc++.sum; \
$(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh -L \
- libstdc++.log.sep normal[0-9]*/libstdc++.log.sep > libstdc++.log; \
+ normal[0-9]*/libstdc++.log.sep > libstdc++.log; \
exit 0; \
fi; \
srcdir=`$(am__cd) $(srcdir) && pwd`; export srcdir; \
@@ -128,47 +139,20 @@ check-DEJAGNU $(check_DEJAGNU_normal_tar
runtest=$(RUNTEST); \
if [ -z "$$runtest" ]; then runtest=runtest; fi; \
tool=libstdc++; \
- dirs=; \
- case "$*" in \
- normal0) \
- if $(SHELL) -c "$$runtest --version" > /dev/null 2>&1; then \
- $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
- $(RUNTESTFLAGS) abi.exp prettyprinters.exp; \
- else echo "WARNING: could not find \`runtest'" 1>&2; :;\
- fi; \
- dirs="`cd $$srcdir; echo [013-9][0-9]_*/*`";; \
- normal1) \
- dirs="`cd $$srcdir; echo [ab]* de* [ep]*/*`";; \
- normal2) \
- dirs="`cd $$srcdir; echo 2[01]_*/*`";; \
- normal3) \
- dirs="`cd $$srcdir; echo 22_*/*`";; \
- normal4) \
- dirs="`cd $$srcdir; echo 23_*/[a-km-tw-z]*`";; \
- normal5) \
- dirs="`cd $$srcdir; echo 23_*/[luv]*`";; \
- normal6) \
- dirs="`cd $$srcdir; echo 2[459]_*/*`";; \
- normal7) \
- dirs="`cd $$srcdir; echo 26_*/* 28_*/[c-z]*`";; \
- normal8) \
- dirs="`cd $$srcdir; echo 27_*/*`";; \
- normal9) \
- dirs="`cd $$srcdir; echo 28_*/[ab]*`";; \
- normal10) \
- dirs="`cd $$srcdir; echo t*/*`";; \
- esac; \
- if [ -n "$*" ]; then cd "$*"; fi; \
+ if [ -n "$*" ]; then \
+ if [ -f normal-parallel/finished ]; then rm -rf "$*"; exit 0; fi; \
+ GCC_RUNTEST_PARALLELIZE_DIR=`${PWD_COMMAND}`/normal-parallel; \
+ export GCC_RUNTEST_PARALLELIZE_DIR; \
+ cd "$*"; \
+ fi; \
if $(SHELL) -c "$$runtest --version" > /dev/null 2>&1; then \
- if [ -n "$$dirs" ]; then \
- $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
- $(RUNTESTFLAGS) \
- "conformance.exp=`echo $$dirs | sed 's/ /* /g;s/$$/*/'`"; \
- else \
- $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
+ $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
$(RUNTESTFLAGS); \
+ if [ -n "$*" ]; then \
+ touch $$GCC_RUNTEST_PARALLELIZE_DIR/finished; \
fi; \
- else echo "WARNING: could not find \`runtest'" 1>&2; :;\
+ else \
+ echo "WARNING: could not find \`runtest'" 1>&2; :;\
fi
check-am:
--- libstdc++-v3/testsuite/lib/libstdc++.exp.jj 2014-09-14 21:08:57.278535267
+0200
+++ libstdc++-v3/testsuite/lib/libstdc++.exp 2014-09-15 13:42:23.822366707
+0200
@@ -1823,3 +1823,78 @@ proc check_v3_target_little_endian { } {
}
set additional_prunes ""
+
+if { [info exists env(GCC_RUNTEST_PARALLELIZE_DIR)] \
+ && [info procs runtest_file_p] != [list] \
+ && [info procs gcc_parallelize_saved_runtest_file_p] == [list] } then {
+ global gcc_runtest_parallelize_counter
+ global gcc_runtest_parallelize_counter_minor
+ global gcc_runtest_parallelize_enable
+ global gcc_runtest_parallelize_dir
+ global gcc_runtest_parallelize_last
+
+ set gcc_runtest_parallelize_counter 0
+ set gcc_runtest_parallelize_counter_minor 0
+ set gcc_runtest_parallelize_enable 1
+ set gcc_runtest_parallelize_dir [getenv GCC_RUNTEST_PARALLELIZE_DIR]
+ set gcc_runtest_parallelize_last 0
+
+ proc gcc_parallel_test_run_p { testcase } {
+ global gcc_runtest_parallelize_counter
+ global gcc_runtest_parallelize_counter_minor
+ global gcc_runtest_parallelize_enable
+ global gcc_runtest_parallelize_dir
+ global gcc_runtest_parallelize_last
+
+ if { $gcc_runtest_parallelize_enable == 0 } {
+ return 1
+ }
+
+ # Only test the filesystem every 10th iteration
+ incr gcc_runtest_parallelize_counter_minor
+ if { $gcc_runtest_parallelize_counter_minor == 10 } {
+ set gcc_runtest_parallelize_counter_minor 0
+ }
+ if { $gcc_runtest_parallelize_counter_minor != 1 } {
+ #verbose -log "gcc_parallel_test_run_p $testcase
$gcc_runtest_parallelize_counter $gcc_runtest_parallelize_last"
+ return $gcc_runtest_parallelize_last
+ }
+
+ set path $gcc_runtest_parallelize_dir/$gcc_runtest_parallelize_counter
+
+ if {![catch {open $path {RDWR CREAT EXCL} 0600} fd]} {
+ close $fd
+ set gcc_runtest_parallelize_last 1
+ #verbose -log "gcc_parallel_test_run_p $testcase
$gcc_runtest_parallelize_counter 1"
+ incr gcc_runtest_parallelize_counter
+ return 1
+ }
+ set gcc_runtest_parallelize_last 0
+ #verbose -log "gcc_parallel_test_run_p $testcase
$gcc_runtest_parallelize_counter 0"
+ incr gcc_runtest_parallelize_counter
+ return 0
+ }
+
+ proc gcc_parallel_test_enable { val } {
+ global gcc_runtest_parallelize_enable
+ set gcc_runtest_parallelize_enable $val
+ }
+
+ rename runtest_file_p gcc_parallelize_saved_runtest_file_p
+ proc runtest_file_p { runtests testcase } {
+ if ![gcc_parallelize_saved_runtest_file_p $runtests $testcase] {
+ return 0
+ }
+ return [gcc_parallel_test_run_p $testcase]
+ }
+
+} else {
+
+ proc gcc_parallel_test_run_p { testcase } {
+ return 1
+ }
+
+ proc gcc_parallel_test_enable { val } {
+ }
+
+}
--- libstdc++-v3/testsuite/Makefile.in.jj 2014-09-14 21:08:57.170535802
+0200
+++ libstdc++-v3/testsuite/Makefile.in 2014-09-15 15:29:57.323521363 +0200
@@ -301,7 +301,16 @@ lists_of_files = \
extract_symvers = $(glibcxx_builddir)/scripts/extract_symvers
baseline_subdir := $(shell $(CXX) $(baseline_subdir_switch))
-check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6
7 8 9 10)
+check_p_numbers0:=1 2 3 4 5 6 7 8 9
+check_p_numbers1:=0 $(check_p_numbers0)
+check_p_numbers2:=$(foreach i,$(check_p_numbers0),$(addprefix
$(i),$(check_p_numbers1)))
+check_p_numbers3:=$(addprefix 0,$(check_p_numbers1)) $(check_p_numbers2)
+check_p_numbers4:=$(foreach i,$(check_p_numbers0),$(addprefix
$(i),$(check_p_numbers3)))
+check_p_numbers5:=$(addprefix 0,$(check_p_numbers3)) $(check_p_numbers4)
+check_p_numbers6:=$(foreach i,$(check_p_numbers0),$(addprefix
$(i),$(check_p_numbers5)))
+check_p_numbers:=$(check_p_numbers0) $(check_p_numbers2) $(check_p_numbers4)
$(check_p_numbers6)
+check_p_subdirs=$(wordlist 1,$(if
$(GCC_TEST_PARALLEL_SLOTS),$(GCC_TEST_PARALLEL_SLOTS),128),$(check_p_numbers))
+check_DEJAGNU_normal_targets = $(addprefix
check-DEJAGNUnormal,$(check_p_subdirs))
# Runs the testsuite, but in compile only mode.
# Can be used to test sources with non-GNU FE's at various warning
@@ -559,19 +568,21 @@ $(check_DEJAGNU_normal_targets): check-D
check-DEJAGNU $(check_DEJAGNU_normal_targets): check-DEJAGNU%: site.exp
AR="$(AR)"; export AR; \
RANLIB="$(RANLIB)"; export RANLIB; \
- if [ -z "$*$(filter-out --target_board=%, $(RUNTESTFLAGS))" ] \
- && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
+ if [ -z "$*" ] && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
+ rm -rf normal-parallel || true; \
+ mkdir normal-parallel; \
$(MAKE) $(AM_MAKEFLAGS) $(check_DEJAGNU_normal_targets); \
- for idx in 0 1 2 3 4 5 6 7 8 9 10; do \
- mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \
- mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \
+ rm -rf normal-parallel || true; \
+ for idx in $(check_p_subdirs); do \
+ if [ -d normal$$idx ]; then \
+ mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \
+ mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \
+ fi; \
done; \
- mv -f libstdc++.sum libstdc++.sum.sep; \
- mv -f libstdc++.log libstdc++.log.sep; \
$(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh \
- libstdc++.sum.sep normal[0-9]*/libstdc++.sum.sep > libstdc++.sum; \
+ normal[0-9]*/libstdc++.sum.sep > libstdc++.sum; \
$(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh -L \
- libstdc++.log.sep normal[0-9]*/libstdc++.log.sep > libstdc++.log; \
+ normal[0-9]*/libstdc++.log.sep > libstdc++.log; \
exit 0; \
fi; \
srcdir=`$(am__cd) $(srcdir) && pwd`; export srcdir; \
@@ -579,47 +590,20 @@ check-DEJAGNU $(check_DEJAGNU_normal_tar
runtest=$(RUNTEST); \
if [ -z "$$runtest" ]; then runtest=runtest; fi; \
tool=libstdc++; \
- dirs=; \
- case "$*" in \
- normal0) \
- if $(SHELL) -c "$$runtest --version" > /dev/null 2>&1; then \
- $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
- $(RUNTESTFLAGS) abi.exp prettyprinters.exp; \
- else echo "WARNING: could not find \`runtest'" 1>&2; :;\
- fi; \
- dirs="`cd $$srcdir; echo [013-9][0-9]_*/*`";; \
- normal1) \
- dirs="`cd $$srcdir; echo [ab]* de* [ep]*/*`";; \
- normal2) \
- dirs="`cd $$srcdir; echo 2[01]_*/*`";; \
- normal3) \
- dirs="`cd $$srcdir; echo 22_*/*`";; \
- normal4) \
- dirs="`cd $$srcdir; echo 23_*/[a-km-tw-z]*`";; \
- normal5) \
- dirs="`cd $$srcdir; echo 23_*/[luv]*`";; \
- normal6) \
- dirs="`cd $$srcdir; echo 2[459]_*/*`";; \
- normal7) \
- dirs="`cd $$srcdir; echo 26_*/* 28_*/[c-z]*`";; \
- normal8) \
- dirs="`cd $$srcdir; echo 27_*/*`";; \
- normal9) \
- dirs="`cd $$srcdir; echo 28_*/[ab]*`";; \
- normal10) \
- dirs="`cd $$srcdir; echo t*/*`";; \
- esac; \
- if [ -n "$*" ]; then cd "$*"; fi; \
+ if [ -n "$*" ]; then \
+ if [ -f normal-parallel/finished ]; then rm -rf "$*"; exit 0; fi; \
+ GCC_RUNTEST_PARALLELIZE_DIR=`${PWD_COMMAND}`/normal-parallel; \
+ export GCC_RUNTEST_PARALLELIZE_DIR; \
+ cd "$*"; \
+ fi; \
if $(SHELL) -c "$$runtest --version" > /dev/null 2>&1; then \
- if [ -n "$$dirs" ]; then \
- $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
- $(RUNTESTFLAGS) \
- "conformance.exp=`echo $$dirs | sed 's/ /* /g;s/$$/*/'`"; \
- else \
- $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
+ $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
$(RUNTESTFLAGS); \
+ if [ -n "$*" ]; then \
+ touch $$GCC_RUNTEST_PARALLELIZE_DIR/finished; \
fi; \
- else echo "WARNING: could not find \`runtest'" 1>&2; :;\
+ else \
+ echo "WARNING: could not find \`runtest'" 1>&2; :;\
fi
check-am:
Jakub
