Re: [PATCH] gcc parallel make check
On 15-09-14 18:05, Jakub Jelinek wrote:
libstdc++-v3/
* testsuite/Makefile.am (check_p_numbers0, check_p_numbers1,
check_p_numbers2, check_p_numbers3, check_p_numbers4,
check_p_numbers5, check_p_numbers6, check_p_numbers,
check_p_subdirs): New variables.
(check_DEJAGNU_normal_targets): Use check_p_subdirs.
(check-DEJAGNU): Rewritten so that for parallelized
testing each job runs all the *.exp files, with
GCC_RUNTEST_PARALLELIZE_DIR set in environment.
* testsuite/Makefile.in: Regenerated.
* testsuite/lib/libstdc++.exp (gcc_parallel_test_run_p,
gcc_parallel_test_enable): New procedures. If
GCC_RUNTEST_PARALLELIZE_DIR is set in environment, override
runtest_file_p to invoke also gcc_parallel_test_run_p.
* testsuite/libstdc++-abi/abi.exp: Run all the tests serially
by the first parallel runtest encountering it. Fix up path
of the extract_symvers script.
* testsuite/libstdc++-xmethods/xmethods.exp: Run all the tests
serially by the first parallel runtest encountering it. Run
dg-finish even in case of error.
When comparing test results of patch builds with test results of reference
builds, the only differences I'm seeing are random differences in amount of
'UNSUPPORTED: prettyprinter.exp'.
This patch fixes that by ensuring that we print that unsupported message only
once.
The resulting test result comparison diff is:
...
--- without/FAIL 2014-11-24 17:46:32.202673282 +0100
+++ with/FAIL 2014-11-25 13:45:15.636131571 +0100
libstdc++-v3/testsuite/libstdc++.sum:UNSUPPORTED: prettyprinters.exp
-libstdc++-v3/testsuite/libstdc++.sum:UNSUPPORTED: prettyprinters.exp
-libstdc++-v3/testsuite/libstdc++.sum:UNSUPPORTED: prettyprinters.exp
-libstdc++-v3/testsuite/libstdc++.sum:UNSUPPORTED: prettyprinters.exp
-libstdc++-v3/testsuite/libstdc++.sum:UNSUPPORTED: prettyprinters.exp
libstdc++-v3/testsuite/libstdc++.sum:UNSUPPORTED: xmethods.exp
...
Furthermore, the patch adds a dg-finish in case the prettyprinters.exp file is
unsupported, which AFAIU is also required in that case.
Bootstrapped and reg-tested on x86_64.
OK for trunk/stage3?
Thanks,
- Tom
2014-11-25 Tom de Vries t...@codesourcery.com
* testsuite/libstdc++-prettyprinters/prettyprinters.exp: Add missing
dg-finish. Only print unsupported message once.
---
libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp | 7 +++
1 file changed, 7 insertions(+)
diff --git a/libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp b/libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp
index a57660f..e5be5b5 100644
--- a/libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp
+++ b/libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp
@@ -30,7 +30,14 @@ if ![info exists ::env(GUALITY_GDB_NAME)] {
}
if {! [gdb_version_check]} {
+dg-finish
+# Only print unsupported message in one instance.
+if ![gcc_parallel_test_run_p prettyprinters] {
+ return
+}
+gcc_parallel_test_enable 0
unsupported prettyprinters.exp
+gcc_parallel_test_enable 1
return
}
--
1.9.1
Re: [PATCH] gcc parallel make check
On Tue, Nov 25, 2014 at 03:27:40PM +0100, Tom de Vries wrote:
This patch fixes that by ensuring that we print that unsupported message only
once.
The resulting test result comparison diff is:
2014-11-25 Tom de Vries t...@codesourcery.com
* testsuite/libstdc++-prettyprinters/prettyprinters.exp: Add missing
dg-finish. Only print unsupported message once.
LGTM.
--- a/libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp
+++ b/libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp
@@ -30,7 +30,14 @@ if ![info exists ::env(GUALITY_GDB_NAME)] {
}
if {! [gdb_version_check]} {
+dg-finish
+# Only print unsupported message in one instance.
+if ![gcc_parallel_test_run_p prettyprinters] {
+ return
+}
+gcc_parallel_test_enable 0
unsupported prettyprinters.exp
+gcc_parallel_test_enable 1
return
}
--
1.9.1
Jakub
Re: [PATCH] gcc parallel make check
On Nov 25, 2014, at 12:15 PM, Jakub Jelinek ja...@redhat.com wrote: On Tue, Nov 25, 2014 at 03:27:40PM +0100, Tom de Vries wrote: This patch fixes that by ensuring that we print that unsupported message only once. The resulting test result comparison diff is: 2014-11-25 Tom de Vries t...@codesourcery.com * testsuite/libstdc++-prettyprinters/prettyprinters.exp: Add missing dg-finish. Only print unsupported message once. LGTM. That is the approval. :-)
Re: [PATCH] gcc parallel make check
On 15-09-14 18:05, Jakub Jelinek wrote:
libstdc++-v3/
* testsuite/Makefile.am (check_p_numbers0, check_p_numbers1,
check_p_numbers2, check_p_numbers3, check_p_numbers4,
check_p_numbers5, check_p_numbers6, check_p_numbers,
check_p_subdirs): New variables.
(check_DEJAGNU_normal_targets): Use check_p_subdirs.
(check-DEJAGNU): Rewritten so that for parallelized
testing each job runs all the *.exp files, with
GCC_RUNTEST_PARALLELIZE_DIR set in environment.
* testsuite/Makefile.in: Regenerated.
* testsuite/lib/libstdc++.exp (gcc_parallel_test_run_p,
gcc_parallel_test_enable): New procedures. If
GCC_RUNTEST_PARALLELIZE_DIR is set in environment, override
runtest_file_p to invoke also gcc_parallel_test_run_p.
* testsuite/libstdc++-abi/abi.exp: Run all the tests serially
by the first parallel runtest encountering it. Fix up path
of the extract_symvers script.
* testsuite/libstdc++-xmethods/xmethods.exp: Run all the tests
serially by the first parallel runtest encountering it. Run
dg-finish even in case of error.
When comparing test results of patch builds with test results of reference
builds, the only differences I'm seeing are random differences in amount of
'UNSUPPORTED: prettyprinter.exp'.
This patch fixes that by ensuring that we print that unsupported message only
once.
The resulting test result comparison diff is:
...
--- without/FAIL 2014-11-24 17:46:32.202673282 +0100
+++ with/FAIL 2014-11-25 13:45:15.636131571 +0100
libstdc++-v3/testsuite/libstdc++.sum:UNSUPPORTED: prettyprinters.exp
-libstdc++-v3/testsuite/libstdc++.sum:UNSUPPORTED: prettyprinters.exp
-libstdc++-v3/testsuite/libstdc++.sum:UNSUPPORTED: prettyprinters.exp
-libstdc++-v3/testsuite/libstdc++.sum:UNSUPPORTED: prettyprinters.exp
-libstdc++-v3/testsuite/libstdc++.sum:UNSUPPORTED: prettyprinters.exp
libstdc++-v3/testsuite/libstdc++.sum:UNSUPPORTED: xmethods.exp
...
Furthermore, the patch adds a dg-finish in case the prettyprinters.exp file is
unsupported, which AFAIU is also required in that case.
Bootstrapped and reg-tested on x86_64.
OK for trunk/stage3?
Thanks,
- Tom
2014-11-25 Tom de Vries t...@codesourcery.com
* testsuite/libstdc++-prettyprinters/prettyprinters.exp: Add missing
dg-finish. Only print unsupported message once.
---
libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp | 7 +++
1 file changed, 7 insertions(+)
diff --git a/libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp b/libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp
index a57660f..e5be5b5 100644
--- a/libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp
+++ b/libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp
@@ -30,7 +30,14 @@ if ![info exists ::env(GUALITY_GDB_NAME)] {
}
if {! [gdb_version_check]} {
+dg-finish
+# Only print unsupported message in one instance.
+if ![gcc_parallel_test_run_p prettyprinters] {
+ return
+}
+gcc_parallel_test_enable 0
unsupported prettyprinters.exp
+gcc_parallel_test_enable 1
return
}
--
1.9.1
Re: [PATCH] gcc parallel make check
On Tue, Nov 25, 2014 at 03:27:40PM +0100, Tom de Vries wrote:
This patch fixes that by ensuring that we print that unsupported message only
once.
The resulting test result comparison diff is:
2014-11-25 Tom de Vries t...@codesourcery.com
* testsuite/libstdc++-prettyprinters/prettyprinters.exp: Add missing
dg-finish. Only print unsupported message once.
LGTM.
--- a/libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp
+++ b/libstdc++-v3/testsuite/libstdc++-prettyprinters/prettyprinters.exp
@@ -30,7 +30,14 @@ if ![info exists ::env(GUALITY_GDB_NAME)] {
}
if {! [gdb_version_check]} {
+dg-finish
+# Only print unsupported message in one instance.
+if ![gcc_parallel_test_run_p prettyprinters] {
+ return
+}
+gcc_parallel_test_enable 0
unsupported prettyprinters.exp
+gcc_parallel_test_enable 1
return
}
--
1.9.1
Jakub
Re: [PATCH] gcc parallel make check
On Nov 25, 2014, at 12:15 PM, Jakub Jelinek ja...@redhat.com wrote: On Tue, Nov 25, 2014 at 03:27:40PM +0100, Tom de Vries wrote: This patch fixes that by ensuring that we print that unsupported message only once. The resulting test result comparison diff is: 2014-11-25 Tom de Vries t...@codesourcery.com * testsuite/libstdc++-prettyprinters/prettyprinters.exp: Add missing dg-finish. Only print unsupported message once. LGTM. That is the approval. :-)
Re: [PATCH] gcc parallel make check
Hi Jakub,
On 15 September 2014 18:05, Jakub Jelinek ja...@redhat.com wrote:
[...]
# For parallelized check-% targets, this decides whether parallelization
# is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
# but optional --target_board or --extra_opts arguments). If desirable,
# recursive make is run with check-parallel-$lang{,1,2,3,4,5} etc. goals,
# which can be executed in parallel, as they are run in separate directories.
-# check-parallel-$lang{1,2,3,4,5} etc. goals invoke runtest with the longest
-# running *.exp files from the testsuite, as determined by
check_$lang_parallelize
-# variable. The check-parallel-$lang goal in that case invokes runtest with
-# all the remaining *.exp files not handled by the separate goals.
+# check-parallel-$lang{,1,2,3,4,5} etc. goals invoke runtest with
+# GCC_RUNTEST_PARALLELIZE_DIR var in the environment and runtest_file_p
+# dejaGNU procedure is overridden to additionally synchronize through
+# a $lang-parallel directory which tests will be run by which runtest
instance.
# Afterwards contrib/dg-extract-results.sh is used to merge the sum and log
# files. If parallelization isn't desirable, only one recursive make
# is run with check-parallel-$lang goal and check_$lang_parallelize variable
@@ -3662,76 +3645,60 @@ check_p_subdirs=$(wordlist 1,$(words $(c
# to lang_checks_parallelized variable and define check_$lang_parallelize
# variable (see above check_gcc_parallelize description).
$(lang_checks_parallelized): check-% : site.exp
- @if [ -z $(filter-out --target_board=%,$(filter-out
--extra_opts%,$(RUNTESTFLAGS))) ] \
Since you removed this test, the comment above is not longer accurate:
setting RUNTESTFLAGS to whatever value no longer disables
parallelization.
Which leads me to discuss a bug I faced after you committed this
change: I am testing a patch which bring a series of new tests.
$ RUNTESTFLAGS=my.exp make -jN check (in fact the 'make -j' is
embedded in a larger build script)
my.exp contains the following construct which is often used in the testsuite:
==
foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
# If we're only testing specific files and this isn't one of them,
skip it.
if ![runtest_file_p $runtests $src] then {
continue
}
c-torture-execute $src $additional_flags
gcc-dg-runtest $src $additional_flags
}
==
Note that gcc-dg-runtest calls runtest_file_p too.
What I observed is that if I use -j1, all my .c files get tested,
while with N2 some of them are silently skipped.
It took me a while to figure out that it's because gcc-dg-runtest
calls runtest_file_p, which means that runtest_file_p is called twice
when the 1st invocation returns 1, and only once when the 1st
invocation returns 0.
For example, if we have pid0, pid1 the concurrent runtest processes,
and file0.c, file1.c, the testcases, then:
* pid0 decides to keep file0.c file1.c file2.c file3.c file4.c. Since
the above loop calls runtest_file_p twice for each, we reach the
minor counter of 10.
* in the mean time, pid1 decides to skip file0.c, file1.c ... file9.c
since it calls runtest_file_p only once for each
* pid1 increments its parallel counter to 1, and create the new testing subdir
* pid1 decides to keep file10, file11, file12, file13 and file14
(again, 2 calls to runtest_file_p per testcase)
* pid0 increments its parallel counter to 1, and decides it has to skip it
* pid0 thus decides to skip file5, file6, file7, ... file14, calling
runtest_file_p once for each
* etc...
In the end, we have ignored file5...file9
I'm not sure why you have made special cases for some of the existing
*.exp when you forced them to disable parallelization.
Was it to handle such cases?
I'm not sure about the next step:
- should I modify my .exp file?
- should you modify gcc_parallel_test_run_p?
Even if I have to modify my .exp file, I think this is error prone,
and others could introduce a similar construct in the future.
Thanks,
Christophe.
Re: [PATCH] gcc parallel make check
On Fri, Oct 10, 2014 at 04:09:39PM +0200, Christophe Lyon wrote:
my.exp contains the following construct which is often used in the testsuite:
==
foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
# If we're only testing specific files and this isn't one of them,
skip it.
if ![runtest_file_p $runtests $src] then {
continue
}
c-torture-execute $src $additional_flags
gcc-dg-runtest $src $additional_flags
}
==
Note that gcc-dg-runtest calls runtest_file_p too.
Such my.exp is invalid, you need to guarantee gcc_parallel_test_run_p
is run the same number of times in all instances unless
gcc_parallel_test_enable has been disabled.
See the patches I've posted when adding the fine-grained parallelization,
e.g. go testsuite has been fixed that way, etc.
So, in your above example, you'd need:
gcc_parallel_test_enable 0
line before c-torture-execute and
gcc_parallel_test_enable 1
line after gcc-dg-runtest. That way, if runtest_file_p says the test should
be scheduled by current instance, all the subtests will be run there.
If my.exp is part of gcc/testsuite, I'm sorry for missing it, if it is
elsewhere, just fix it up.
Note, there are #verbose lines in gcc_parallel_test_run_p, you can uncomment
them and through sed on the log files verify that each instance performs the
same parallelization checks (same strings).
Jakub
Re: [PATCH] gcc parallel make check
On 10 October 2014 16:19, Jakub Jelinek ja...@redhat.com wrote:
On Fri, Oct 10, 2014 at 04:09:39PM +0200, Christophe Lyon wrote:
my.exp contains the following construct which is often used in the testsuite:
==
foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
# If we're only testing specific files and this isn't one of them,
skip it.
if ![runtest_file_p $runtests $src] then {
continue
}
c-torture-execute $src $additional_flags
gcc-dg-runtest $src $additional_flags
}
==
Note that gcc-dg-runtest calls runtest_file_p too.
Such my.exp is invalid, you need to guarantee gcc_parallel_test_run_p
is run the same number of times in all instances unless
gcc_parallel_test_enable has been disabled.
Thanks for your prompt answer.
Is this documented somewhere, so that such cases do not happen in the future?
See the patches I've posted when adding the fine-grained parallelization,
e.g. go testsuite has been fixed that way, etc.
So, in your above example, you'd need:
gcc_parallel_test_enable 0
line before c-torture-execute and
gcc_parallel_test_enable 1
line after gcc-dg-runtest. That way, if runtest_file_p says the test should
be scheduled by current instance, all the subtests will be run there.
If my.exp is part of gcc/testsuite, I'm sorry for missing it, if it is
elsewhere, just fix it up.
It's in a patch which has been under review for quite some time
(started before your change), that's why you missed it.
Note, there are #verbose lines in gcc_parallel_test_run_p, you can uncomment
them and through sed on the log files verify that each instance performs the
same parallelization checks (same strings).
Yep, I saw those and also added other traces of my own :-)
What about my remark about:
# For parallelized check-% targets, this decides whether parallelization
# is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
# but optional --target_board or --extra_opts arguments). If desirable,
I think it should be removed from gcc/Makefile.in
Thanks,
Christophe.
Re: [PATCH] gcc parallel make check
On Fri, Oct 10, 2014 at 04:50:47PM +0200, Christophe Lyon wrote:
On 10 October 2014 16:19, Jakub Jelinek ja...@redhat.com wrote:
On Fri, Oct 10, 2014 at 04:09:39PM +0200, Christophe Lyon wrote:
my.exp contains the following construct which is often used in the
testsuite:
==
foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
# If we're only testing specific files and this isn't one of them,
skip it.
if ![runtest_file_p $runtests $src] then {
continue
}
c-torture-execute $src $additional_flags
gcc-dg-runtest $src $additional_flags
}
==
Note that gcc-dg-runtest calls runtest_file_p too.
Such my.exp is invalid, you need to guarantee gcc_parallel_test_run_p
is run the same number of times in all instances unless
gcc_parallel_test_enable has been disabled.
Thanks for your prompt answer.
Is this documented somewhere, so that such cases do not happen in the future?
Feel free to submit a documentation patch.
It's in a patch which has been under review for quite some time
(started before your change), that's why you missed it.
Ah, ok.
What about my remark about:
# For parallelized check-% targets, this decides whether parallelization
# is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
# but optional --target_board or --extra_opts arguments). If desirable,
I think it should be removed from gcc/Makefile.in
Only the and RUNTESTFLAGS ... arguments part of that. Patch preapproved.
Jakub
Re: [PATCH] gcc parallel make check
Hi Jakub,
On 15 September 2014 18:05, Jakub Jelinek ja...@redhat.com wrote:
[...]
# For parallelized check-% targets, this decides whether parallelization
# is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
# but optional --target_board or --extra_opts arguments). If desirable,
# recursive make is run with check-parallel-$lang{,1,2,3,4,5} etc. goals,
# which can be executed in parallel, as they are run in separate directories.
-# check-parallel-$lang{1,2,3,4,5} etc. goals invoke runtest with the longest
-# running *.exp files from the testsuite, as determined by
check_$lang_parallelize
-# variable. The check-parallel-$lang goal in that case invokes runtest with
-# all the remaining *.exp files not handled by the separate goals.
+# check-parallel-$lang{,1,2,3,4,5} etc. goals invoke runtest with
+# GCC_RUNTEST_PARALLELIZE_DIR var in the environment and runtest_file_p
+# dejaGNU procedure is overridden to additionally synchronize through
+# a $lang-parallel directory which tests will be run by which runtest
instance.
# Afterwards contrib/dg-extract-results.sh is used to merge the sum and log
# files. If parallelization isn't desirable, only one recursive make
# is run with check-parallel-$lang goal and check_$lang_parallelize variable
@@ -3662,76 +3645,60 @@ check_p_subdirs=$(wordlist 1,$(words $(c
# to lang_checks_parallelized variable and define check_$lang_parallelize
# variable (see above check_gcc_parallelize description).
$(lang_checks_parallelized): check-% : site.exp
- @if [ -z $(filter-out --target_board=%,$(filter-out
--extra_opts%,$(RUNTESTFLAGS))) ] \
Since you removed this test, the comment above is not longer accurate:
setting RUNTESTFLAGS to whatever value no longer disables
parallelization.
Which leads me to discuss a bug I faced after you committed this
change: I am testing a patch which bring a series of new tests.
$ RUNTESTFLAGS=my.exp make -jN check (in fact the 'make -j' is
embedded in a larger build script)
my.exp contains the following construct which is often used in the testsuite:
==
foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
# If we're only testing specific files and this isn't one of them,
skip it.
if ![runtest_file_p $runtests $src] then {
continue
}
c-torture-execute $src $additional_flags
gcc-dg-runtest $src $additional_flags
}
==
Note that gcc-dg-runtest calls runtest_file_p too.
What I observed is that if I use -j1, all my .c files get tested,
while with N2 some of them are silently skipped.
It took me a while to figure out that it's because gcc-dg-runtest
calls runtest_file_p, which means that runtest_file_p is called twice
when the 1st invocation returns 1, and only once when the 1st
invocation returns 0.
For example, if we have pid0, pid1 the concurrent runtest processes,
and file0.c, file1.c, the testcases, then:
* pid0 decides to keep file0.c file1.c file2.c file3.c file4.c. Since
the above loop calls runtest_file_p twice for each, we reach the
minor counter of 10.
* in the mean time, pid1 decides to skip file0.c, file1.c ... file9.c
since it calls runtest_file_p only once for each
* pid1 increments its parallel counter to 1, and create the new testing subdir
* pid1 decides to keep file10, file11, file12, file13 and file14
(again, 2 calls to runtest_file_p per testcase)
* pid0 increments its parallel counter to 1, and decides it has to skip it
* pid0 thus decides to skip file5, file6, file7, ... file14, calling
runtest_file_p once for each
* etc...
In the end, we have ignored file5...file9
I'm not sure why you have made special cases for some of the existing
*.exp when you forced them to disable parallelization.
Was it to handle such cases?
I'm not sure about the next step:
- should I modify my .exp file?
- should you modify gcc_parallel_test_run_p?
Even if I have to modify my .exp file, I think this is error prone,
and others could introduce a similar construct in the future.
Thanks,
Christophe.
Re: [PATCH] gcc parallel make check
On Fri, Oct 10, 2014 at 04:09:39PM +0200, Christophe Lyon wrote:
my.exp contains the following construct which is often used in the testsuite:
==
foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
# If we're only testing specific files and this isn't one of them,
skip it.
if ![runtest_file_p $runtests $src] then {
continue
}
c-torture-execute $src $additional_flags
gcc-dg-runtest $src $additional_flags
}
==
Note that gcc-dg-runtest calls runtest_file_p too.
Such my.exp is invalid, you need to guarantee gcc_parallel_test_run_p
is run the same number of times in all instances unless
gcc_parallel_test_enable has been disabled.
See the patches I've posted when adding the fine-grained parallelization,
e.g. go testsuite has been fixed that way, etc.
So, in your above example, you'd need:
gcc_parallel_test_enable 0
line before c-torture-execute and
gcc_parallel_test_enable 1
line after gcc-dg-runtest. That way, if runtest_file_p says the test should
be scheduled by current instance, all the subtests will be run there.
If my.exp is part of gcc/testsuite, I'm sorry for missing it, if it is
elsewhere, just fix it up.
Note, there are #verbose lines in gcc_parallel_test_run_p, you can uncomment
them and through sed on the log files verify that each instance performs the
same parallelization checks (same strings).
Jakub
Re: [PATCH] gcc parallel make check
On 10 October 2014 16:19, Jakub Jelinek ja...@redhat.com wrote:
On Fri, Oct 10, 2014 at 04:09:39PM +0200, Christophe Lyon wrote:
my.exp contains the following construct which is often used in the testsuite:
==
foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
# If we're only testing specific files and this isn't one of them,
skip it.
if ![runtest_file_p $runtests $src] then {
continue
}
c-torture-execute $src $additional_flags
gcc-dg-runtest $src $additional_flags
}
==
Note that gcc-dg-runtest calls runtest_file_p too.
Such my.exp is invalid, you need to guarantee gcc_parallel_test_run_p
is run the same number of times in all instances unless
gcc_parallel_test_enable has been disabled.
Thanks for your prompt answer.
Is this documented somewhere, so that such cases do not happen in the future?
See the patches I've posted when adding the fine-grained parallelization,
e.g. go testsuite has been fixed that way, etc.
So, in your above example, you'd need:
gcc_parallel_test_enable 0
line before c-torture-execute and
gcc_parallel_test_enable 1
line after gcc-dg-runtest. That way, if runtest_file_p says the test should
be scheduled by current instance, all the subtests will be run there.
If my.exp is part of gcc/testsuite, I'm sorry for missing it, if it is
elsewhere, just fix it up.
It's in a patch which has been under review for quite some time
(started before your change), that's why you missed it.
Note, there are #verbose lines in gcc_parallel_test_run_p, you can uncomment
them and through sed on the log files verify that each instance performs the
same parallelization checks (same strings).
Yep, I saw those and also added other traces of my own :-)
What about my remark about:
# For parallelized check-% targets, this decides whether parallelization
# is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
# but optional --target_board or --extra_opts arguments). If desirable,
I think it should be removed from gcc/Makefile.in
Thanks,
Christophe.
Re: [PATCH] gcc parallel make check
On Fri, Oct 10, 2014 at 04:50:47PM +0200, Christophe Lyon wrote:
On 10 October 2014 16:19, Jakub Jelinek ja...@redhat.com wrote:
On Fri, Oct 10, 2014 at 04:09:39PM +0200, Christophe Lyon wrote:
my.exp contains the following construct which is often used in the
testsuite:
==
foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
# If we're only testing specific files and this isn't one of them,
skip it.
if ![runtest_file_p $runtests $src] then {
continue
}
c-torture-execute $src $additional_flags
gcc-dg-runtest $src $additional_flags
}
==
Note that gcc-dg-runtest calls runtest_file_p too.
Such my.exp is invalid, you need to guarantee gcc_parallel_test_run_p
is run the same number of times in all instances unless
gcc_parallel_test_enable has been disabled.
Thanks for your prompt answer.
Is this documented somewhere, so that such cases do not happen in the future?
Feel free to submit a documentation patch.
It's in a patch which has been under review for quite some time
(started before your change), that's why you missed it.
Ah, ok.
What about my remark about:
# For parallelized check-% targets, this decides whether parallelization
# is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
# but optional --target_board or --extra_opts arguments). If desirable,
I think it should be removed from gcc/Makefile.in
Only the and RUNTESTFLAGS ... arguments part of that. Patch preapproved.
Jakub
Re: [PATCH] gcc parallel make check
On 09/12/2014 08:04 PM, Jakub Jelinek wrote:
I've been worried about the quick cases where
parallelization is not beneficial, like make check-gcc \
RUNTESTFLAGS=dg.exp=pr60123.c or similar, but one doesn't usually pass -jN
in that case.
I have -jN in my $MAKEFLAGS, so I've been running into this with my rgt
shell function:
rgt ()
{
( cd ~/m/$CANON/gcc/gcc;
make check-c++ ${1:+RUNTESTFLAGS=$*} )
}
If I say 'rgt dg.exp=var-templ1.C' the actual test results are lost in
the explosion of shell verbosity. Could we add some '@'s to more of the
rules, perhaps?
Jason
Re: [PATCH] gcc parallel make check
On Mon, Sep 22, 2014 at 11:21:14AM -0400, Jason Merrill wrote:
On 09/12/2014 08:04 PM, Jakub Jelinek wrote:
I've been worried about the quick cases where
parallelization is not beneficial, like make check-gcc \
RUNTESTFLAGS=dg.exp=pr60123.c or similar, but one doesn't usually pass -jN
in that case.
I have -jN in my $MAKEFLAGS, so I've been running into this with my rgt
shell function:
rgt ()
{
( cd ~/m/$CANON/gcc/gcc;
make check-c++ ${1:+RUNTESTFLAGS=$*} )
}
If I say 'rgt dg.exp=var-templ1.C' the actual test results are lost in the
explosion of shell verbosity. Could we add some '@'s to more of the rules,
perhaps?
I've been considering that too, but not sure what info people find valuable
and what they don't.
Jakub
Re: [PATCH] gcc parallel make check
On 09/22/2014 11:26 AM, Jakub Jelinek wrote:
On Mon, Sep 22, 2014 at 11:21:14AM -0400, Jason Merrill wrote:
If I say 'rgt dg.exp=var-templ1.C' the actual test results are lost in the
explosion of shell verbosity. Could we add some '@'s to more of the rules,
perhaps?
I've been considering that too, but not sure what info people find valuable
and what they don't.
I don't see much information in the ~128 repetitions of the
check-parallel rules with different numbers; the actual runtest command
is the same in all of them. Adding @ to all of the commands of the
check-parallel-% rule makes things much better for me:
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 6f251a5..be4c840 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -3674,10 +3674,10 @@ $(lang_checks_parallelized): check-% : site.exp
fi
check-parallel-% : site.exp
- -test -d plugin || mkdir plugin
- -test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
- test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir $(TESTSUITEDIR)/$(check_p_subdir)
- -(rootme=`${PWD_COMMAND}`; export rootme; \
+ -@test -d plugin || mkdir plugin
+ -@test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
+ @test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir $(TESTSUITEDIR)/$(check_p_subdir)
+ -@(rootme=`${PWD_COMMAND}`; export rootme; \
srcdir=`cd ${srcdir}; ${PWD_COMMAND}` ; export srcdir ; \
if [ -n $(check_p_subno) ] \
[ -n $$GCC_RUNTEST_PARALLELIZE_DIR ] \
Re: [PATCH] gcc parallel make check
On Mon, Sep 22, 2014 at 05:26:04PM +0200, Jakub Jelinek wrote: I've been considering that too, but not sure what info people find valuable and what they don't. The ten million Running blablablalba.exp ... messages on a very parallel run aren't helpful in my opinion. There might be more but that drowns out everything else :-) Segher
Re: [PATCH] gcc parallel make check
On Mon, Sep 22, 2014 at 10:44:06AM -0500, Segher Boessenkool wrote: On Mon, Sep 22, 2014 at 05:26:04PM +0200, Jakub Jelinek wrote: I've been considering that too, but not sure what info people find valuable and what they don't. The ten million Running blablablalba.exp ... messages on a very parallel run aren't helpful in my opinion. There might be more but that drowns out everything else :-) It has some value, it shows the actual progress. Sure, you can just watch the *.log files as they are populated and get better picture. I think the Running *.exp messages go from dejagnu, not from gcc testsuite changes. Jakub
Re: [PATCH] gcc parallel make check
On Mon, Sep 22, 2014 at 11:43:35AM -0400, Jason Merrill wrote:
On 09/22/2014 11:26 AM, Jakub Jelinek wrote:
On Mon, Sep 22, 2014 at 11:21:14AM -0400, Jason Merrill wrote:
If I say 'rgt dg.exp=var-templ1.C' the actual test results are lost in the
explosion of shell verbosity. Could we add some '@'s to more of the rules,
perhaps?
I've been considering that too, but not sure what info people find valuable
and what they don't.
I don't see much information in the ~128 repetitions of the check-parallel
rules with different numbers; the actual runtest command is the same in all
of them. Adding @ to all of the commands of the check-parallel-% rule makes
things much better for me:
LGTM (though, supposedly we want similar change in
libstdc++-v3/testsuite/Makefile.am).
Or, if people would really like to see the commands, we could print them
just once, using e.g.
-$(if $(check_p_subno),@)(rootme= ...
(then e.g. check-parallel-gcc goal would print the command, but
check-parallel-gcc-1 or check-parallel-gcc-112 would not).
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -3674,10 +3674,10 @@ $(lang_checks_parallelized): check-% : site.exp
fi
check-parallel-% : site.exp
- -test -d plugin || mkdir plugin
- -test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
- test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir
$(TESTSUITEDIR)/$(check_p_subdir)
- -(rootme=`${PWD_COMMAND}`; export rootme; \
+ -@test -d plugin || mkdir plugin
+ -@test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
+ @test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir
$(TESTSUITEDIR)/$(check_p_subdir)
+ -@(rootme=`${PWD_COMMAND}`; export rootme; \
srcdir=`cd ${srcdir}; ${PWD_COMMAND}` ; export srcdir ; \
if [ -n $(check_p_subno) ] \
[ -n $$GCC_RUNTEST_PARALLELIZE_DIR ] \
Jakub
Re: [PATCH] gcc parallel make check
On Mon, Sep 22, 2014 at 05:49:12PM +0200, Jakub Jelinek wrote: On Mon, Sep 22, 2014 at 10:44:06AM -0500, Segher Boessenkool wrote: On Mon, Sep 22, 2014 at 05:26:04PM +0200, Jakub Jelinek wrote: I've been considering that too, but not sure what info people find valuable and what they don't. The ten million Running blablablalba.exp ... messages on a very parallel run aren't helpful in my opinion. There might be more but that drowns out everything else :-) It has some value, it shows the actual progress. Sure, you can just watch the *.log files as they are populated and get better picture. I think the Running *.exp messages go from dejagnu, not from gcc testsuite changes. Hrm. Looking at the log files it seems there are not more of those messages at all since the changes. Maybe it just all got too fast! :-) Segher
Re: [PATCH] gcc parallel make check
On 09/22/2014 11:58 AM, Jakub Jelinek wrote:
LGTM (though, supposedly we want similar change in
libstdc++-v3/testsuite/Makefile.am).
Or, if people would really like to see the commands, we could print them
just once, using e.g.
-$(if $(check_p_subno),@)(rootme= ...
(then e.g. check-parallel-gcc goal would print the command, but
check-parallel-gcc-1 or check-parallel-gcc-112 would not).
So, like this?
commit c750897381a3f936e27cabd825cfa85ce936a6a9
Author: Jason Merrill ja...@redhat.com
Date: Mon Sep 22 11:44:00 2014 -0400
gcc/
* Makefile.in (check-parallel-%): Add @.
libstdc++-v3/
* testsuite/Makefile.am (%/site.exp): Add @.
(check-DEJAGNU): Likewise.
* testsuite/Makefile.in: Regenerate.
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 6f251a5..97b439a 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -3674,10 +3674,10 @@ $(lang_checks_parallelized): check-% : site.exp
fi
check-parallel-% : site.exp
- -test -d plugin || mkdir plugin
- -test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
- test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir $(TESTSUITEDIR)/$(check_p_subdir)
- -(rootme=`${PWD_COMMAND}`; export rootme; \
+ -@test -d plugin || mkdir plugin
+ -@test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
+ @test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir $(TESTSUITEDIR)/$(check_p_subdir)
+ -$(if $(check_p_subno),@)(rootme=`${PWD_COMMAND}`; export rootme; \
srcdir=`cd ${srcdir}; ${PWD_COMMAND}` ; export srcdir ; \
if [ -n $(check_p_subno) ] \
[ -n $$GCC_RUNTEST_PARALLELIZE_DIR ] \
diff --git a/libstdc++-v3/testsuite/Makefile.am b/libstdc++-v3/testsuite/Makefile.am
index e206aba..b4c9e85 100644
--- a/libstdc++-v3/testsuite/Makefile.am
+++ b/libstdc++-v3/testsuite/Makefile.am
@@ -91,9 +91,9 @@ new-abi-baseline:
${extract_symvers} ../src/.libs/libstdc++.so $${output})
%/site.exp: site.exp
- -test -d $* || mkdir $*
+ -@test -d $* || mkdir $*
@srcdir=`cd $(srcdir); ${PWD_COMMAND}`;
- objdir=`${PWD_COMMAND}`/$*; \
+ @objdir=`${PWD_COMMAND}`/$*; \
sed -e s|^set srcdir .*$$|set srcdir $$srcdir| \
-e s|^set objdir .*$$|set objdir $$objdir| \
site.exp $*/site.exp.tmp
@@ -115,7 +115,7 @@ $(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%: normal%/site.exp
# Run the testsuite in normal mode.
check-DEJAGNU $(check_DEJAGNU_normal_targets): check-DEJAGNU%: site.exp
- AR=$(AR); export AR; \
+ $(if $*,@)AR=$(AR); export AR; \
RANLIB=$(RANLIB); export RANLIB; \
if [ -z $* ] [ $(filter -j, $(MFLAGS)) = -j ]; then \
rm -rf normal-parallel || true; \
diff --git a/libstdc++-v3/testsuite/Makefile.in b/libstdc++-v3/testsuite/Makefile.in
index 59060b8..0fc26f4 100644
--- a/libstdc++-v3/testsuite/Makefile.in
+++ b/libstdc++-v3/testsuite/Makefile.in
@@ -553,9 +553,9 @@ new-abi-baseline:
${extract_symvers} ../src/.libs/libstdc++.so $${output})
%/site.exp: site.exp
- -test -d $* || mkdir $*
+ -@test -d $* || mkdir $*
@srcdir=`cd $(srcdir); ${PWD_COMMAND}`;
- objdir=`${PWD_COMMAND}`/$*; \
+ @objdir=`${PWD_COMMAND}`/$*; \
sed -e s|^set srcdir .*$$|set srcdir $$srcdir| \
-e s|^set objdir .*$$|set objdir $$objdir| \
site.exp $*/site.exp.tmp
@@ -566,7 +566,7 @@ $(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%: normal%/site.exp
# Run the testsuite in normal mode.
check-DEJAGNU $(check_DEJAGNU_normal_targets): check-DEJAGNU%: site.exp
- AR=$(AR); export AR; \
+ $(if $*,@)AR=$(AR); export AR; \
RANLIB=$(RANLIB); export RANLIB; \
if [ -z $* ] [ $(filter -j, $(MFLAGS)) = -j ]; then \
rm -rf normal-parallel || true; \
Re: [PATCH] gcc parallel make check
On Mon, Sep 22, 2014 at 12:21:08PM -0400, Jason Merrill wrote:
On 09/22/2014 11:58 AM, Jakub Jelinek wrote:
LGTM (though, supposedly we want similar change in
libstdc++-v3/testsuite/Makefile.am).
Or, if people would really like to see the commands, we could print them
just once, using e.g.
-$(if $(check_p_subno),@)(rootme= ...
(then e.g. check-parallel-gcc goal would print the command, but
check-parallel-gcc-1 or check-parallel-gcc-112 would not).
So, like this?
Ok, thanks.
commit c750897381a3f936e27cabd825cfa85ce936a6a9
Author: Jason Merrill ja...@redhat.com
Date: Mon Sep 22 11:44:00 2014 -0400
gcc/
* Makefile.in (check-parallel-%): Add @.
libstdc++-v3/
* testsuite/Makefile.am (%/site.exp): Add @.
(check-DEJAGNU): Likewise.
* testsuite/Makefile.in: Regenerate.
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 6f251a5..97b439a 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -3674,10 +3674,10 @@ $(lang_checks_parallelized): check-% : site.exp
fi
check-parallel-% : site.exp
- -test -d plugin || mkdir plugin
- -test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
- test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir
$(TESTSUITEDIR)/$(check_p_subdir)
- -(rootme=`${PWD_COMMAND}`; export rootme; \
+ -@test -d plugin || mkdir plugin
+ -@test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
+ @test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir
$(TESTSUITEDIR)/$(check_p_subdir)
+ -$(if $(check_p_subno),@)(rootme=`${PWD_COMMAND}`; export rootme; \
srcdir=`cd ${srcdir}; ${PWD_COMMAND}` ; export srcdir ; \
if [ -n $(check_p_subno) ] \
[ -n $$GCC_RUNTEST_PARALLELIZE_DIR ] \
diff --git a/libstdc++-v3/testsuite/Makefile.am
b/libstdc++-v3/testsuite/Makefile.am
index e206aba..b4c9e85 100644
--- a/libstdc++-v3/testsuite/Makefile.am
+++ b/libstdc++-v3/testsuite/Makefile.am
@@ -91,9 +91,9 @@ new-abi-baseline:
${extract_symvers} ../src/.libs/libstdc++.so $${output})
%/site.exp: site.exp
- -test -d $* || mkdir $*
+ -@test -d $* || mkdir $*
@srcdir=`cd $(srcdir); ${PWD_COMMAND}`;
- objdir=`${PWD_COMMAND}`/$*; \
+ @objdir=`${PWD_COMMAND}`/$*; \
sed -e s|^set srcdir .*$$|set srcdir $$srcdir| \
-e s|^set objdir .*$$|set objdir $$objdir| \
site.exp $*/site.exp.tmp
@@ -115,7 +115,7 @@ $(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%:
normal%/site.exp
# Run the testsuite in normal mode.
check-DEJAGNU $(check_DEJAGNU_normal_targets): check-DEJAGNU%: site.exp
- AR=$(AR); export AR; \
+ $(if $*,@)AR=$(AR); export AR; \
RANLIB=$(RANLIB); export RANLIB; \
if [ -z $* ] [ $(filter -j, $(MFLAGS)) = -j ]; then \
rm -rf normal-parallel || true; \
diff --git a/libstdc++-v3/testsuite/Makefile.in
b/libstdc++-v3/testsuite/Makefile.in
index 59060b8..0fc26f4 100644
--- a/libstdc++-v3/testsuite/Makefile.in
+++ b/libstdc++-v3/testsuite/Makefile.in
@@ -553,9 +553,9 @@ new-abi-baseline:
${extract_symvers} ../src/.libs/libstdc++.so $${output})
%/site.exp: site.exp
- -test -d $* || mkdir $*
+ -@test -d $* || mkdir $*
@srcdir=`cd $(srcdir); ${PWD_COMMAND}`;
- objdir=`${PWD_COMMAND}`/$*; \
+ @objdir=`${PWD_COMMAND}`/$*; \
sed -e s|^set srcdir .*$$|set srcdir $$srcdir| \
-e s|^set objdir .*$$|set objdir $$objdir| \
site.exp $*/site.exp.tmp
@@ -566,7 +566,7 @@ $(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%:
normal%/site.exp
# Run the testsuite in normal mode.
check-DEJAGNU $(check_DEJAGNU_normal_targets): check-DEJAGNU%: site.exp
- AR=$(AR); export AR; \
+ $(if $*,@)AR=$(AR); export AR; \
RANLIB=$(RANLIB); export RANLIB; \
if [ -z $* ] [ $(filter -j, $(MFLAGS)) = -j ]; then \
rm -rf normal-parallel || true; \
Jakub
Re: [PATCH] gcc parallel make check
On 09/12/2014 08:04 PM, Jakub Jelinek wrote:
I've been worried about the quick cases where
parallelization is not beneficial, like make check-gcc \
RUNTESTFLAGS=dg.exp=pr60123.c or similar, but one doesn't usually pass -jN
in that case.
I have -jN in my $MAKEFLAGS, so I've been running into this with my rgt
shell function:
rgt ()
{
( cd ~/m/$CANON/gcc/gcc;
make check-c++ ${1:+RUNTESTFLAGS=$*} )
}
If I say 'rgt dg.exp=var-templ1.C' the actual test results are lost in
the explosion of shell verbosity. Could we add some '@'s to more of the
rules, perhaps?
Jason
Re: [PATCH] gcc parallel make check
On Mon, Sep 22, 2014 at 11:21:14AM -0400, Jason Merrill wrote:
On 09/12/2014 08:04 PM, Jakub Jelinek wrote:
I've been worried about the quick cases where
parallelization is not beneficial, like make check-gcc \
RUNTESTFLAGS=dg.exp=pr60123.c or similar, but one doesn't usually pass -jN
in that case.
I have -jN in my $MAKEFLAGS, so I've been running into this with my rgt
shell function:
rgt ()
{
( cd ~/m/$CANON/gcc/gcc;
make check-c++ ${1:+RUNTESTFLAGS=$*} )
}
If I say 'rgt dg.exp=var-templ1.C' the actual test results are lost in the
explosion of shell verbosity. Could we add some '@'s to more of the rules,
perhaps?
I've been considering that too, but not sure what info people find valuable
and what they don't.
Jakub
Re: [PATCH] gcc parallel make check
On 09/22/2014 11:26 AM, Jakub Jelinek wrote:
On Mon, Sep 22, 2014 at 11:21:14AM -0400, Jason Merrill wrote:
If I say 'rgt dg.exp=var-templ1.C' the actual test results are lost in the
explosion of shell verbosity. Could we add some '@'s to more of the rules,
perhaps?
I've been considering that too, but not sure what info people find valuable
and what they don't.
I don't see much information in the ~128 repetitions of the
check-parallel rules with different numbers; the actual runtest command
is the same in all of them. Adding @ to all of the commands of the
check-parallel-% rule makes things much better for me:
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 6f251a5..be4c840 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -3674,10 +3674,10 @@ $(lang_checks_parallelized): check-% : site.exp
fi
check-parallel-% : site.exp
- -test -d plugin || mkdir plugin
- -test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
- test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir $(TESTSUITEDIR)/$(check_p_subdir)
- -(rootme=`${PWD_COMMAND}`; export rootme; \
+ -@test -d plugin || mkdir plugin
+ -@test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
+ @test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir $(TESTSUITEDIR)/$(check_p_subdir)
+ -@(rootme=`${PWD_COMMAND}`; export rootme; \
srcdir=`cd ${srcdir}; ${PWD_COMMAND}` ; export srcdir ; \
if [ -n $(check_p_subno) ] \
[ -n $$GCC_RUNTEST_PARALLELIZE_DIR ] \
Re: [PATCH] gcc parallel make check
On Mon, Sep 22, 2014 at 05:26:04PM +0200, Jakub Jelinek wrote: I've been considering that too, but not sure what info people find valuable and what they don't. The ten million Running blablablalba.exp ... messages on a very parallel run aren't helpful in my opinion. There might be more but that drowns out everything else :-) Segher
Re: [PATCH] gcc parallel make check
On Mon, Sep 22, 2014 at 10:44:06AM -0500, Segher Boessenkool wrote: On Mon, Sep 22, 2014 at 05:26:04PM +0200, Jakub Jelinek wrote: I've been considering that too, but not sure what info people find valuable and what they don't. The ten million Running blablablalba.exp ... messages on a very parallel run aren't helpful in my opinion. There might be more but that drowns out everything else :-) It has some value, it shows the actual progress. Sure, you can just watch the *.log files as they are populated and get better picture. I think the Running *.exp messages go from dejagnu, not from gcc testsuite changes. Jakub
Re: [PATCH] gcc parallel make check
On Mon, Sep 22, 2014 at 11:43:35AM -0400, Jason Merrill wrote:
On 09/22/2014 11:26 AM, Jakub Jelinek wrote:
On Mon, Sep 22, 2014 at 11:21:14AM -0400, Jason Merrill wrote:
If I say 'rgt dg.exp=var-templ1.C' the actual test results are lost in the
explosion of shell verbosity. Could we add some '@'s to more of the rules,
perhaps?
I've been considering that too, but not sure what info people find valuable
and what they don't.
I don't see much information in the ~128 repetitions of the check-parallel
rules with different numbers; the actual runtest command is the same in all
of them. Adding @ to all of the commands of the check-parallel-% rule makes
things much better for me:
LGTM (though, supposedly we want similar change in
libstdc++-v3/testsuite/Makefile.am).
Or, if people would really like to see the commands, we could print them
just once, using e.g.
-$(if $(check_p_subno),@)(rootme= ...
(then e.g. check-parallel-gcc goal would print the command, but
check-parallel-gcc-1 or check-parallel-gcc-112 would not).
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -3674,10 +3674,10 @@ $(lang_checks_parallelized): check-% : site.exp
fi
check-parallel-% : site.exp
- -test -d plugin || mkdir plugin
- -test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
- test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir
$(TESTSUITEDIR)/$(check_p_subdir)
- -(rootme=`${PWD_COMMAND}`; export rootme; \
+ -@test -d plugin || mkdir plugin
+ -@test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
+ @test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir
$(TESTSUITEDIR)/$(check_p_subdir)
+ -@(rootme=`${PWD_COMMAND}`; export rootme; \
srcdir=`cd ${srcdir}; ${PWD_COMMAND}` ; export srcdir ; \
if [ -n $(check_p_subno) ] \
[ -n $$GCC_RUNTEST_PARALLELIZE_DIR ] \
Jakub
Re: [PATCH] gcc parallel make check
On Mon, Sep 22, 2014 at 05:49:12PM +0200, Jakub Jelinek wrote: On Mon, Sep 22, 2014 at 10:44:06AM -0500, Segher Boessenkool wrote: On Mon, Sep 22, 2014 at 05:26:04PM +0200, Jakub Jelinek wrote: I've been considering that too, but not sure what info people find valuable and what they don't. The ten million Running blablablalba.exp ... messages on a very parallel run aren't helpful in my opinion. There might be more but that drowns out everything else :-) It has some value, it shows the actual progress. Sure, you can just watch the *.log files as they are populated and get better picture. I think the Running *.exp messages go from dejagnu, not from gcc testsuite changes. Hrm. Looking at the log files it seems there are not more of those messages at all since the changes. Maybe it just all got too fast! :-) Segher
Re: [PATCH] gcc parallel make check
On 09/22/2014 11:58 AM, Jakub Jelinek wrote:
LGTM (though, supposedly we want similar change in
libstdc++-v3/testsuite/Makefile.am).
Or, if people would really like to see the commands, we could print them
just once, using e.g.
-$(if $(check_p_subno),@)(rootme= ...
(then e.g. check-parallel-gcc goal would print the command, but
check-parallel-gcc-1 or check-parallel-gcc-112 would not).
So, like this?
commit c750897381a3f936e27cabd825cfa85ce936a6a9
Author: Jason Merrill ja...@redhat.com
Date: Mon Sep 22 11:44:00 2014 -0400
gcc/
* Makefile.in (check-parallel-%): Add @.
libstdc++-v3/
* testsuite/Makefile.am (%/site.exp): Add @.
(check-DEJAGNU): Likewise.
* testsuite/Makefile.in: Regenerate.
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 6f251a5..97b439a 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -3674,10 +3674,10 @@ $(lang_checks_parallelized): check-% : site.exp
fi
check-parallel-% : site.exp
- -test -d plugin || mkdir plugin
- -test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
- test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir $(TESTSUITEDIR)/$(check_p_subdir)
- -(rootme=`${PWD_COMMAND}`; export rootme; \
+ -@test -d plugin || mkdir plugin
+ -@test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
+ @test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir $(TESTSUITEDIR)/$(check_p_subdir)
+ -$(if $(check_p_subno),@)(rootme=`${PWD_COMMAND}`; export rootme; \
srcdir=`cd ${srcdir}; ${PWD_COMMAND}` ; export srcdir ; \
if [ -n $(check_p_subno) ] \
[ -n $$GCC_RUNTEST_PARALLELIZE_DIR ] \
diff --git a/libstdc++-v3/testsuite/Makefile.am b/libstdc++-v3/testsuite/Makefile.am
index e206aba..b4c9e85 100644
--- a/libstdc++-v3/testsuite/Makefile.am
+++ b/libstdc++-v3/testsuite/Makefile.am
@@ -91,9 +91,9 @@ new-abi-baseline:
${extract_symvers} ../src/.libs/libstdc++.so $${output})
%/site.exp: site.exp
- -test -d $* || mkdir $*
+ -@test -d $* || mkdir $*
@srcdir=`cd $(srcdir); ${PWD_COMMAND}`;
- objdir=`${PWD_COMMAND}`/$*; \
+ @objdir=`${PWD_COMMAND}`/$*; \
sed -e s|^set srcdir .*$$|set srcdir $$srcdir| \
-e s|^set objdir .*$$|set objdir $$objdir| \
site.exp $*/site.exp.tmp
@@ -115,7 +115,7 @@ $(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%: normal%/site.exp
# Run the testsuite in normal mode.
check-DEJAGNU $(check_DEJAGNU_normal_targets): check-DEJAGNU%: site.exp
- AR=$(AR); export AR; \
+ $(if $*,@)AR=$(AR); export AR; \
RANLIB=$(RANLIB); export RANLIB; \
if [ -z $* ] [ $(filter -j, $(MFLAGS)) = -j ]; then \
rm -rf normal-parallel || true; \
diff --git a/libstdc++-v3/testsuite/Makefile.in b/libstdc++-v3/testsuite/Makefile.in
index 59060b8..0fc26f4 100644
--- a/libstdc++-v3/testsuite/Makefile.in
+++ b/libstdc++-v3/testsuite/Makefile.in
@@ -553,9 +553,9 @@ new-abi-baseline:
${extract_symvers} ../src/.libs/libstdc++.so $${output})
%/site.exp: site.exp
- -test -d $* || mkdir $*
+ -@test -d $* || mkdir $*
@srcdir=`cd $(srcdir); ${PWD_COMMAND}`;
- objdir=`${PWD_COMMAND}`/$*; \
+ @objdir=`${PWD_COMMAND}`/$*; \
sed -e s|^set srcdir .*$$|set srcdir $$srcdir| \
-e s|^set objdir .*$$|set objdir $$objdir| \
site.exp $*/site.exp.tmp
@@ -566,7 +566,7 @@ $(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%: normal%/site.exp
# Run the testsuite in normal mode.
check-DEJAGNU $(check_DEJAGNU_normal_targets): check-DEJAGNU%: site.exp
- AR=$(AR); export AR; \
+ $(if $*,@)AR=$(AR); export AR; \
RANLIB=$(RANLIB); export RANLIB; \
if [ -z $* ] [ $(filter -j, $(MFLAGS)) = -j ]; then \
rm -rf normal-parallel || true; \
Re: [PATCH] gcc parallel make check
On Mon, Sep 22, 2014 at 12:21:08PM -0400, Jason Merrill wrote:
On 09/22/2014 11:58 AM, Jakub Jelinek wrote:
LGTM (though, supposedly we want similar change in
libstdc++-v3/testsuite/Makefile.am).
Or, if people would really like to see the commands, we could print them
just once, using e.g.
-$(if $(check_p_subno),@)(rootme= ...
(then e.g. check-parallel-gcc goal would print the command, but
check-parallel-gcc-1 or check-parallel-gcc-112 would not).
So, like this?
Ok, thanks.
commit c750897381a3f936e27cabd825cfa85ce936a6a9
Author: Jason Merrill ja...@redhat.com
Date: Mon Sep 22 11:44:00 2014 -0400
gcc/
* Makefile.in (check-parallel-%): Add @.
libstdc++-v3/
* testsuite/Makefile.am (%/site.exp): Add @.
(check-DEJAGNU): Likewise.
* testsuite/Makefile.in: Regenerate.
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 6f251a5..97b439a 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -3674,10 +3674,10 @@ $(lang_checks_parallelized): check-% : site.exp
fi
check-parallel-% : site.exp
- -test -d plugin || mkdir plugin
- -test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
- test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir
$(TESTSUITEDIR)/$(check_p_subdir)
- -(rootme=`${PWD_COMMAND}`; export rootme; \
+ -@test -d plugin || mkdir plugin
+ -@test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
+ @test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir
$(TESTSUITEDIR)/$(check_p_subdir)
+ -$(if $(check_p_subno),@)(rootme=`${PWD_COMMAND}`; export rootme; \
srcdir=`cd ${srcdir}; ${PWD_COMMAND}` ; export srcdir ; \
if [ -n $(check_p_subno) ] \
[ -n $$GCC_RUNTEST_PARALLELIZE_DIR ] \
diff --git a/libstdc++-v3/testsuite/Makefile.am
b/libstdc++-v3/testsuite/Makefile.am
index e206aba..b4c9e85 100644
--- a/libstdc++-v3/testsuite/Makefile.am
+++ b/libstdc++-v3/testsuite/Makefile.am
@@ -91,9 +91,9 @@ new-abi-baseline:
${extract_symvers} ../src/.libs/libstdc++.so $${output})
%/site.exp: site.exp
- -test -d $* || mkdir $*
+ -@test -d $* || mkdir $*
@srcdir=`cd $(srcdir); ${PWD_COMMAND}`;
- objdir=`${PWD_COMMAND}`/$*; \
+ @objdir=`${PWD_COMMAND}`/$*; \
sed -e s|^set srcdir .*$$|set srcdir $$srcdir| \
-e s|^set objdir .*$$|set objdir $$objdir| \
site.exp $*/site.exp.tmp
@@ -115,7 +115,7 @@ $(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%:
normal%/site.exp
# Run the testsuite in normal mode.
check-DEJAGNU $(check_DEJAGNU_normal_targets): check-DEJAGNU%: site.exp
- AR=$(AR); export AR; \
+ $(if $*,@)AR=$(AR); export AR; \
RANLIB=$(RANLIB); export RANLIB; \
if [ -z $* ] [ $(filter -j, $(MFLAGS)) = -j ]; then \
rm -rf normal-parallel || true; \
diff --git a/libstdc++-v3/testsuite/Makefile.in
b/libstdc++-v3/testsuite/Makefile.in
index 59060b8..0fc26f4 100644
--- a/libstdc++-v3/testsuite/Makefile.in
+++ b/libstdc++-v3/testsuite/Makefile.in
@@ -553,9 +553,9 @@ new-abi-baseline:
${extract_symvers} ../src/.libs/libstdc++.so $${output})
%/site.exp: site.exp
- -test -d $* || mkdir $*
+ -@test -d $* || mkdir $*
@srcdir=`cd $(srcdir); ${PWD_COMMAND}`;
- objdir=`${PWD_COMMAND}`/$*; \
+ @objdir=`${PWD_COMMAND}`/$*; \
sed -e s|^set srcdir .*$$|set srcdir $$srcdir| \
-e s|^set objdir .*$$|set objdir $$objdir| \
site.exp $*/site.exp.tmp
@@ -566,7 +566,7 @@ $(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%:
normal%/site.exp
# Run the testsuite in normal mode.
check-DEJAGNU $(check_DEJAGNU_normal_targets): check-DEJAGNU%: site.exp
- AR=$(AR); export AR; \
+ $(if $*,@)AR=$(AR); export AR; \
RANLIB=$(RANLIB); export RANLIB; \
if [ -z $* ] [ $(filter -j, $(MFLAGS)) = -j ]; then \
rm -rf normal-parallel || true; \
Jakub
Re: [PATCH] gcc parallel make check
On Mon, Sep 15, 2014 at 7:44 PM, Mike Stump mikest...@comcast.net wrote: On Sep 15, 2014, at 9:05 AM, Jakub Jelinek ja...@redhat.com wrote: All the updates sound good. Regtested on x86_64-linux, without the patch toplevel make -k check took 8hrs3minutes (don't have time data for that run), This confuses me, but, no matter. Isn’t 8hrs time data? :-) patch toplevel make -j48 -k check took: real40m21.984s user341m51.675s sys 112m46.993s and with the patch make -j48 -k check took: real32m22.066s user355m1.788s sys 117m5.809s These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks. A nice improvement indeed. The patched result is 15 times faster than the serial unpatched run. So there is room for improvement (I wouldn't say the scaling factor is reasonable - with accounting for overhead I'd expect it should be possible to arrive at a factor of 32 here at least). So - where's the serial parts of the testing run? Thanks for the improvements btw! Richard. Is this version ok for trunk? Ok. Thanks for all your work.
Re: [PATCH] gcc parallel make check
On Tue, Sep 16, 2014 at 11:20:37AM +0200, Richard Biener wrote: This confuses me, but, no matter. Isn’t 8hrs time data? :-) It is, but not time(1) data, just wall clock computed from subtracting mtimes of my make check output log and make -j48 bootstrap log. patch toplevel make -j48 -k check took: real40m21.984s user341m51.675s sys 112m46.993s and with the patch make -j48 -k check took: real32m22.066s user355m1.788s sys 117m5.809s These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks. A nice improvement indeed. The patched result is 15 times faster than the serial unpatched run. So there is room for improvement Note, the box used was oldish AMD 16-core, no ht, box, haven't tried it on anything more parallel, also it was normal hard disk, etc. No idea whether anything from this is relevant to that though. Some CPU time goes into the expect processes, I can retry the build tonight and grab also time(1) info from make -k check to see the user/sys times for serial testing. Jakub
Re: [PATCH] gcc parallel make check
On Tue, Sep 16, 2014 at 11:28 AM, Jakub Jelinek ja...@redhat.com wrote: On Tue, Sep 16, 2014 at 11:20:37AM +0200, Richard Biener wrote: This confuses me, but, no matter. Isn’t 8hrs time data? :-) It is, but not time(1) data, just wall clock computed from subtracting mtimes of my make check output log and make -j48 bootstrap log. patch toplevel make -j48 -k check took: real40m21.984s user341m51.675s sys 112m46.993s and with the patch make -j48 -k check took: real32m22.066s user355m1.788s sys 117m5.809s These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks. A nice improvement indeed. The patched result is 15 times faster than the serial unpatched run. So there is room for improvement Note, the box used was oldish AMD 16-core, no ht, box, haven't tried it on anything Ah, I assumed -j48 testing means you have 48 cores. I usually test with -j12 on my 6-core HT-enabled box. A factor 15 scaling for 16 CPUs is of course close to the best we can achieve. Richard. more parallel, also it was normal hard disk, etc. No idea whether anything from this is relevant to that though. Some CPU time goes into the expect processes, I can retry the build tonight and grab also time(1) info from make -k check to see the user/sys times for serial testing. Jakub
RE: [PATCH] gcc parallel make check
These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks. A nice improvement indeed. The patched result is 15 times faster than the serial unpatched run. So there is room for improvement Note, the box used was oldish AMD 16-core, no ht, box, haven't tried it on anything on a 32 core box, no ht, I see these timings: time make -j32 -k check log.check32 ; time make -j8 -k check log.check8 real18m14.562s user260m21.578s sys 264m26.042s real41m33.210s user233m4.563s sys 72m11.429s so it is not quite reaching the ideal 4x speedup. Counting the number of 'expect' processes they are nicely at around 32 and 8 for the full test, with only a very short tail near the end. So, there might be some overhead somewhere. Total user time is similar, but time in sys goes up.
Re: [PATCH] gcc parallel make check
On Mon, Sep 15, 2014 at 7:44 PM, Mike Stump mikest...@comcast.net wrote: On Sep 15, 2014, at 9:05 AM, Jakub Jelinek ja...@redhat.com wrote: All the updates sound good. Regtested on x86_64-linux, without the patch toplevel make -k check took 8hrs3minutes (don't have time data for that run), This confuses me, but, no matter. Isn’t 8hrs time data? :-) patch toplevel make -j48 -k check took: real40m21.984s user341m51.675s sys 112m46.993s and with the patch make -j48 -k check took: real32m22.066s user355m1.788s sys 117m5.809s These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks. A nice improvement indeed. The patched result is 15 times faster than the serial unpatched run. So there is room for improvement (I wouldn't say the scaling factor is reasonable - with accounting for overhead I'd expect it should be possible to arrive at a factor of 32 here at least). So - where's the serial parts of the testing run? Thanks for the improvements btw! Richard. Is this version ok for trunk? Ok. Thanks for all your work.
Re: [PATCH] gcc parallel make check
On Tue, Sep 16, 2014 at 11:20:37AM +0200, Richard Biener wrote: This confuses me, but, no matter. Isn’t 8hrs time data? :-) It is, but not time(1) data, just wall clock computed from subtracting mtimes of my make check output log and make -j48 bootstrap log. patch toplevel make -j48 -k check took: real40m21.984s user341m51.675s sys 112m46.993s and with the patch make -j48 -k check took: real32m22.066s user355m1.788s sys 117m5.809s These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks. A nice improvement indeed. The patched result is 15 times faster than the serial unpatched run. So there is room for improvement Note, the box used was oldish AMD 16-core, no ht, box, haven't tried it on anything more parallel, also it was normal hard disk, etc. No idea whether anything from this is relevant to that though. Some CPU time goes into the expect processes, I can retry the build tonight and grab also time(1) info from make -k check to see the user/sys times for serial testing. Jakub
Re: [PATCH] gcc parallel make check
On Tue, Sep 16, 2014 at 11:28 AM, Jakub Jelinek ja...@redhat.com wrote: On Tue, Sep 16, 2014 at 11:20:37AM +0200, Richard Biener wrote: This confuses me, but, no matter. Isn’t 8hrs time data? :-) It is, but not time(1) data, just wall clock computed from subtracting mtimes of my make check output log and make -j48 bootstrap log. patch toplevel make -j48 -k check took: real40m21.984s user341m51.675s sys 112m46.993s and with the patch make -j48 -k check took: real32m22.066s user355m1.788s sys 117m5.809s These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks. A nice improvement indeed. The patched result is 15 times faster than the serial unpatched run. So there is room for improvement Note, the box used was oldish AMD 16-core, no ht, box, haven't tried it on anything Ah, I assumed -j48 testing means you have 48 cores. I usually test with -j12 on my 6-core HT-enabled box. A factor 15 scaling for 16 CPUs is of course close to the best we can achieve. Richard. more parallel, also it was normal hard disk, etc. No idea whether anything from this is relevant to that though. Some CPU time goes into the expect processes, I can retry the build tonight and grab also time(1) info from make -k check to see the user/sys times for serial testing. Jakub
RE: [PATCH] gcc parallel make check
These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks. A nice improvement indeed. The patched result is 15 times faster than the serial unpatched run. So there is room for improvement Note, the box used was oldish AMD 16-core, no ht, box, haven't tried it on anything on a 32 core box, no ht, I see these timings: time make -j32 -k check log.check32 ; time make -j8 -k check log.check8 real18m14.562s user260m21.578s sys 264m26.042s real41m33.210s user233m4.563s sys 72m11.429s so it is not quite reaching the ideal 4x speedup. Counting the number of 'expect' processes they are nicely at around 32 and 8 for the full test, with only a very short tail near the end. So, there might be some overhead somewhere. Total user time is similar, but time in sys goes up.
Re: [PATCH] gcc parallel make check
On Fri, Sep 12, 2014 at 04:42:25PM -0700, Mike Stump wrote:
On Sep 12, 2014, at 9:32 AM, Jakub Jelinek ja...@redhat.com wrote:
Here is my latest version of the patch.
I did a timing test:
Here is an updated version.
Changes since last version:
1) acats parallelized the same way (just, because it is in shell,
using mkdir instead of open with O_EXCL|O_CREAT);
also, as acats has pretty significant initial setup time
(up to a minute or so on not really fast box), I'm now
creating the /support stuff before spawning the parallel
jobs and let the parallel jobs use the same shared support
directory
2) I'm now using addprefix instead of patsubst where appropriate
3) I'm using $(if ...) instead of $(or ...) to make it usable
with make 3.80 (3.81 already supports or)
4) parallelization is performed for any kinds of RUNTESTFLAGS arguments now
5) struct-layout-1.exp apparently doesn't have to be performed serially,
and for gnu-encoding.exp I've used similar change as for go-test.exp
6) in libstdc++, abi.exp, pretty-printers.exp and xmethods.exp are performed
together with conformance.exp, so again parallelization for any
RUNTESTFLAGS flags; abi.exp and xmethods.exp are serially tested
by the first runtest instance to encounter them
Regtested on x86_64-linux, without the patch toplevel make -k check
took 8hrs3minutes (don't have time data for that run), without the
patch toplevel make -j48 -k check took:
real40m21.984s
user341m51.675s
sys 112m46.993s
and with the patch make -j48 -k check took:
real32m22.066s
user355m1.788s
sys 117m5.809s
I saw over 45 jobs running pretty much as the point where all the
testing was done, and test_summary run from the non-parallel testing
is the same as test_summary from the -j48 testing with the patch.
Is this version ok for trunk?
2014-09-14 Jakub Jelinek ja...@redhat.com
gcc/
* Makefile.in (dg_target_exps): Remove.
(check_gcc_parallelize): Change to just an upper bound number.
(check-%-subtargets): Always print the non-parallelized goals.
(check_p_vars, check_p_comma, check_p_subwork): Remove.
(check_p_count, check_p_numbers0, check_p_numbers1, check_p_numbers2,
check_p_numbers3, check_p_numbers4, check_p_numbers5,
check_p_numbers6): New variables.
(check_p_numbers): Set to sequence from 1 to .
(check_p_subdirs): Set to sequence from 1 to minimum of
$(check_p_count) and either GCC_TEST_PARALLEL_SLOTS env var if set,
or 128.
(check-%, check-parallel-%): Rewritten so that for parallelized
testing each job runs all the *.exp files, with
GCC_RUNTEST_PARALLELIZE_DIR set in environment.
gcc/go/
* Make-lang.in (check_go_parallelize): Change to just an upper bound
number.
gcc/fortran/
* Make-lang.in (check_gfortran_parallelize): Change to just an upper
bound number.
gcc/cp/
* Make-lang.in (check_g++_parallelize): Change to just an upper bound
number.
gcc/objc/
* Make-lang.in (check_objc_parallelize): Change to just an upper
bound number.
gcc/ada/
* gcc-interface/Make-lang.in (check_acats_numbers0,
check_acats_numbers1, check_acats_numbers2, check_acats_numbers3,
check_acats_numbers4, check_acats_numbers5, check_acats_numbers6,
check_acats_numbers, check_acats_subdirs): New variables.
(check_acats_targets): Use $(check_acats_subdirs).
(check-acats, check-acats%): Rewritten so that for parallelized
testing each job runs all the chapters files, with
GCC_RUNTEST_PARALLELIZE_DIR set in environment. Prepare the support
directory sequentially and share it.
(check-acats-subtargets): Always print just check-acats.
gcc/testsuite/
* lib/gcc-defs.exp (gcc_parallel_test_run_p,
gcc_parallel_test_enable): New procedures. If
GCC_RUNTEST_PARALLELIZE_DIR is set in environment, override
runtest_file_p to invoke also gcc_parallel_test_run_p.
* g++.dg/guality/guality.exp (check_guality): Save/restore
test_counts array around the body of the procedure.
* gcc.dg/guality/guality.exp (check_guality): Likewise.
* g++.dg/plugin/plugin.exp: Run all the tests serially
by the first parallel runtest encountering it.
* gcc.dg/plugin/plugin.exp: Likewise.
* gcc.misc-tests/matrix1.exp: Likewise.
* gcc.misc-tests/dhry.exp: Likewise.
* gcc.misc-tests/acker1.exp: Likewise.
* gcc.misc-tests/linkage.exp: Likewise.
* gcc.misc-tests/mg.exp: Likewise.
* gcc.misc-tests/mg-2.exp: Likewise.
* gcc.misc-tests/sort2.exp: Likewise.
* gcc.misc-tests/sieve.exp: Likewise.
* gcc.misc-tests/options.exp: Likewise.
* gcc.misc-tests/help.exp: Likewise.
* go.test/go-test.exp (go-gc-tests): Use
gcc_parallel_test_enable {0, 1} around all handling of
Re: [PATCH] gcc parallel make check
On Sep 15, 2014, at 9:05 AM, Jakub Jelinek ja...@redhat.com wrote: All the updates sound good. Regtested on x86_64-linux, without the patch toplevel make -k check took 8hrs3minutes (don't have time data for that run), This confuses me, but, no matter. Isn’t 8hrs time data? :-) patch toplevel make -j48 -k check took: real40m21.984s user341m51.675s sys 112m46.993s and with the patch make -j48 -k check took: real32m22.066s user355m1.788s sys 117m5.809s These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks. Is this version ok for trunk? Ok. Thanks for all your work.
Re: [PATCH] gcc parallel make check
On Fri, Sep 12, 2014 at 04:42:25PM -0700, Mike Stump wrote:
On Sep 12, 2014, at 9:32 AM, Jakub Jelinek ja...@redhat.com wrote:
Here is my latest version of the patch.
I did a timing test:
Here is an updated version.
Changes since last version:
1) acats parallelized the same way (just, because it is in shell,
using mkdir instead of open with O_EXCL|O_CREAT);
also, as acats has pretty significant initial setup time
(up to a minute or so on not really fast box), I'm now
creating the /support stuff before spawning the parallel
jobs and let the parallel jobs use the same shared support
directory
2) I'm now using addprefix instead of patsubst where appropriate
3) I'm using $(if ...) instead of $(or ...) to make it usable
with make 3.80 (3.81 already supports or)
4) parallelization is performed for any kinds of RUNTESTFLAGS arguments now
5) struct-layout-1.exp apparently doesn't have to be performed serially,
and for gnu-encoding.exp I've used similar change as for go-test.exp
6) in libstdc++, abi.exp, pretty-printers.exp and xmethods.exp are performed
together with conformance.exp, so again parallelization for any
RUNTESTFLAGS flags; abi.exp and xmethods.exp are serially tested
by the first runtest instance to encounter them
Regtested on x86_64-linux, without the patch toplevel make -k check
took 8hrs3minutes (don't have time data for that run), without the
patch toplevel make -j48 -k check took:
real40m21.984s
user341m51.675s
sys 112m46.993s
and with the patch make -j48 -k check took:
real32m22.066s
user355m1.788s
sys 117m5.809s
I saw over 45 jobs running pretty much as the point where all the
testing was done, and test_summary run from the non-parallel testing
is the same as test_summary from the -j48 testing with the patch.
Is this version ok for trunk?
2014-09-14 Jakub Jelinek ja...@redhat.com
gcc/
* Makefile.in (dg_target_exps): Remove.
(check_gcc_parallelize): Change to just an upper bound number.
(check-%-subtargets): Always print the non-parallelized goals.
(check_p_vars, check_p_comma, check_p_subwork): Remove.
(check_p_count, check_p_numbers0, check_p_numbers1, check_p_numbers2,
check_p_numbers3, check_p_numbers4, check_p_numbers5,
check_p_numbers6): New variables.
(check_p_numbers): Set to sequence from 1 to .
(check_p_subdirs): Set to sequence from 1 to minimum of
$(check_p_count) and either GCC_TEST_PARALLEL_SLOTS env var if set,
or 128.
(check-%, check-parallel-%): Rewritten so that for parallelized
testing each job runs all the *.exp files, with
GCC_RUNTEST_PARALLELIZE_DIR set in environment.
gcc/go/
* Make-lang.in (check_go_parallelize): Change to just an upper bound
number.
gcc/fortran/
* Make-lang.in (check_gfortran_parallelize): Change to just an upper
bound number.
gcc/cp/
* Make-lang.in (check_g++_parallelize): Change to just an upper bound
number.
gcc/objc/
* Make-lang.in (check_objc_parallelize): Change to just an upper
bound number.
gcc/ada/
* gcc-interface/Make-lang.in (check_acats_numbers0,
check_acats_numbers1, check_acats_numbers2, check_acats_numbers3,
check_acats_numbers4, check_acats_numbers5, check_acats_numbers6,
check_acats_numbers, check_acats_subdirs): New variables.
(check_acats_targets): Use $(check_acats_subdirs).
(check-acats, check-acats%): Rewritten so that for parallelized
testing each job runs all the chapters files, with
GCC_RUNTEST_PARALLELIZE_DIR set in environment. Prepare the support
directory sequentially and share it.
(check-acats-subtargets): Always print just check-acats.
gcc/testsuite/
* lib/gcc-defs.exp (gcc_parallel_test_run_p,
gcc_parallel_test_enable): New procedures. If
GCC_RUNTEST_PARALLELIZE_DIR is set in environment, override
runtest_file_p to invoke also gcc_parallel_test_run_p.
* g++.dg/guality/guality.exp (check_guality): Save/restore
test_counts array around the body of the procedure.
* gcc.dg/guality/guality.exp (check_guality): Likewise.
* g++.dg/plugin/plugin.exp: Run all the tests serially
by the first parallel runtest encountering it.
* gcc.dg/plugin/plugin.exp: Likewise.
* gcc.misc-tests/matrix1.exp: Likewise.
* gcc.misc-tests/dhry.exp: Likewise.
* gcc.misc-tests/acker1.exp: Likewise.
* gcc.misc-tests/linkage.exp: Likewise.
* gcc.misc-tests/mg.exp: Likewise.
* gcc.misc-tests/mg-2.exp: Likewise.
* gcc.misc-tests/sort2.exp: Likewise.
* gcc.misc-tests/sieve.exp: Likewise.
* gcc.misc-tests/options.exp: Likewise.
* gcc.misc-tests/help.exp: Likewise.
* go.test/go-test.exp (go-gc-tests): Use
gcc_parallel_test_enable {0, 1} around all handling of
Re: [PATCH] gcc parallel make check
On Sep 15, 2014, at 9:05 AM, Jakub Jelinek ja...@redhat.com wrote: All the updates sound good. Regtested on x86_64-linux, without the patch toplevel make -k check took 8hrs3minutes (don't have time data for that run), This confuses me, but, no matter. Isn’t 8hrs time data? :-) patch toplevel make -j48 -k check took: real40m21.984s user341m51.675s sys 112m46.993s and with the patch make -j48 -k check took: real32m22.066s user355m1.788s sys 117m5.809s These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks. Is this version ok for trunk? Ok. Thanks for all your work.
Re: [PATCH] gcc parallel make check
On 12 September 2014 19:46:33 Mike Stump mikest...@comcast.net wrote: On Sep 12, 2014, at 9:32 AM, Jakub Jelinek ja...@redhat.com wrote: Here is my latest version of the patch. With this patch I get identical test_summary output on make -k check (completely serial testing) and make -j48 -k check from toplevel directory. Major changes since last version: 1) I've changed the granularity, now it does O_EXCL|O_CREAT attempt only every 10th runtest_file_p invocation So, I’d love to see the numbers for 5 and 20 to double check that 10 is the right number to pick. This sort of refinement is trivial post checkin. 3) various other *.exp fails didn't use runtest_file_p, especially the gcc.misc-tests/ ones, tweaked those like struct-layout-1.exp or plugin.exp so that only the first runtest instance to encounter those runs all of the *.exp file serially Regtested on x86_64-linux, ok for trunk? Ok. Please be around after you apply it to try and sort out any major fallout. Usage of $(or) and $(and) will bump GNU make prerequisite version from our current 3.80 to at least 3.82 (IIRC). PS: for the numbers I had used addsuffix rather than patsubst in the hopes that it avoids lots of regexp calls. Very minor not though. Cheers, If someone can check their target post checkin (or help out pre-checkin) and report back, that would be nice. Times before and post checkin with core count -j setting would be nice. I wonder if the libstdc++ problems can be sorted out merely by finding a way to sort them so the expensive ones come early (regexp - 0regexp for example). Or, instead of sorting them by name, sort them by some other key (md5 per line). The idea then would be that the chance of all regexp tests being in one group is 0. Sent with AquaMail for Android http://www.aqua-mail.com
Re: [PATCH] gcc parallel make check
On 13 September 2014 02:04:51 Jakub Jelinek ja...@redhat.com wrote: On Fri, Sep 12, 2014 at 04:42:25PM -0700, Mike Stump wrote: curious, when I run atomic.exp=stdatom\*.c: gcc.dg/atomic/atomic.exp completed in 30 seconds. atomic.exp=c\*.c takes 522 seconds with 3, 2, 5 and 4 being the worst offenders. That's the @if [ -z $(filter-out --target_board=%,$(filter-out --extra_opts%,$(RUNTESTFLAGS))) ] \ [ $(filter -j, $(MFLAGS)) = -j ]; then \ i.e. if you specify anything in RUNTESTFLAGS other than --target_board= or --extra_opts, it is not parallelized. This was done previously because parallelization required setting the flags to something different (manually created *.exp list). The first [] could Yes, this is very inconvenient, especially in the light of -v in the runtestflags which should certainly not prohibit parallel execution. See https://gcc.gnu.org/ml/gcc-patches/2013-11/msg00997.html for how I would fix that.. (findstring empty instead of filter-out). TIA, perhaps be removed now, if one e.g. RUNTESTFLAGS=atomic.exp etc. with sufficiently enough tests, parallelization will be still worth it. I've been worried about the quick cases where parallelization is not beneficial, like make check-gcc \ RUNTESTFLAGS=dg.exp=pr60123.c or similar, but one doesn't usually pass -jN in that case. So yes, the [ -z $(filter-out --target_board=%,$(filter-out --extra_opts%,$(RUNTESTFLAGS))) ] can be dropped (not in libstdc++ though, there are abi.exp and prettyprinters.exp still run serially, though even that could be handled the struct-layout-1.exp way, of running it by the first instance to encounter those with small changes in those *.exp files). Jakub Sent with AquaMail for Android http://www.aqua-mail.com
Re: [PATCH] gcc parallel make check
On 12 September 2014 19:46:33 Mike Stump mikest...@comcast.net wrote: On Sep 12, 2014, at 9:32 AM, Jakub Jelinek ja...@redhat.com wrote: Here is my latest version of the patch. With this patch I get identical test_summary output on make -k check (completely serial testing) and make -j48 -k check from toplevel directory. Major changes since last version: 1) I've changed the granularity, now it does O_EXCL|O_CREAT attempt only every 10th runtest_file_p invocation So, I’d love to see the numbers for 5 and 20 to double check that 10 is the right number to pick. This sort of refinement is trivial post checkin. 3) various other *.exp fails didn't use runtest_file_p, especially the gcc.misc-tests/ ones, tweaked those like struct-layout-1.exp or plugin.exp so that only the first runtest instance to encounter those runs all of the *.exp file serially Regtested on x86_64-linux, ok for trunk? Ok. Please be around after you apply it to try and sort out any major fallout. Usage of $(or) and $(and) will bump GNU make prerequisite version from our current 3.80 to at least 3.82 (IIRC). PS: for the numbers I had used addsuffix rather than patsubst in the hopes that it avoids lots of regexp calls. Very minor not though. Cheers, If someone can check their target post checkin (or help out pre-checkin) and report back, that would be nice. Times before and post checkin with core count -j setting would be nice. I wonder if the libstdc++ problems can be sorted out merely by finding a way to sort them so the expensive ones come early (regexp - 0regexp for example). Or, instead of sorting them by name, sort them by some other key (md5 per line). The idea then would be that the chance of all regexp tests being in one group is 0. Sent with AquaMail for Android http://www.aqua-mail.com
Re: [PATCH] gcc parallel make check
On 13 September 2014 02:04:51 Jakub Jelinek ja...@redhat.com wrote: On Fri, Sep 12, 2014 at 04:42:25PM -0700, Mike Stump wrote: curious, when I run atomic.exp=stdatom\*.c: gcc.dg/atomic/atomic.exp completed in 30 seconds. atomic.exp=c\*.c takes 522 seconds with 3, 2, 5 and 4 being the worst offenders. That's the @if [ -z $(filter-out --target_board=%,$(filter-out --extra_opts%,$(RUNTESTFLAGS))) ] \ [ $(filter -j, $(MFLAGS)) = -j ]; then \ i.e. if you specify anything in RUNTESTFLAGS other than --target_board= or --extra_opts, it is not parallelized. This was done previously because parallelization required setting the flags to something different (manually created *.exp list). The first [] could Yes, this is very inconvenient, especially in the light of -v in the runtestflags which should certainly not prohibit parallel execution. See https://gcc.gnu.org/ml/gcc-patches/2013-11/msg00997.html for how I would fix that.. (findstring empty instead of filter-out). TIA, perhaps be removed now, if one e.g. RUNTESTFLAGS=atomic.exp etc. with sufficiently enough tests, parallelization will be still worth it. I've been worried about the quick cases where parallelization is not beneficial, like make check-gcc \ RUNTESTFLAGS=dg.exp=pr60123.c or similar, but one doesn't usually pass -jN in that case. So yes, the [ -z $(filter-out --target_board=%,$(filter-out --extra_opts%,$(RUNTESTFLAGS))) ] can be dropped (not in libstdc++ though, there are abi.exp and prettyprinters.exp still run serially, though even that could be handled the struct-layout-1.exp way, of running it by the first instance to encounter those with small changes in those *.exp files). Jakub Sent with AquaMail for Android http://www.aqua-mail.com
RE: [PATCH] gcc parallel make check
a newer patch (v8) I'll send soon attached with updated changelog. Compared to the previously posted v6, only the libstdc++-v3/testsuite/Makefile.am has been refined to split a little more the e*/* pattern, and two quickly running goal have been merged, in addition to fixing the pre-exisiting error in some of the patterns in that file. Checked comparing testsuite results before after. Obviously, if Jakub's patch can be made to work around the testsuite special cases, I believe it should be superior. If not, the attached patch is working as far as I can tell, and provides a significant improvement over current trunk. Joostcontrib/ChangeLog 2014-09-12 Joost VandeVondele vond...@gcc.gnu.org * generate_tcl_patterns.sh: New file. gcc/fortran/ChangeLog 2014-09-12 Joost VandeVondele vond...@gcc.gnu.org * Make-lang.in (check_gfortran_parallelize): Improved parallelism. gcc/Changelog 2014-09-12 Joost VandeVondele vond...@gcc.gnu.org * Makefile.in (check_gcc_parallelize): Improved parallelism. (check_p_numbers): Increase maximum value. (dg_target_exps): Mention targets as separate words only. (null,space,comma,dg_target_exps_p1,dg_target_exps_p2, dg_target_exps_p3,dg_target_exps_p4): New variables. gcc/cp/ChangeLog 2014-09-12 Joost VandeVondele vond...@gcc.gnu.org * Make-lang.in (check_g++_parallelize): Improved parallelism. libstdc++-v3/ChangeLog 2014-09-12 Joost VandeVondele vond...@gcc.gnu.org * testsuite/Makefile.am (check_DEJAGNU_normal_targets): Add check-DEJAGNUnormal[11-15]. (check-DEJAGNU): Split into 15 jobs for parallel testing, correct pattern. * testsuite/Makefile.in: Regenerated. Index: libstdc++-v3/testsuite/Makefile.in === --- libstdc++-v3/testsuite/Makefile.in (revision 215147) +++ libstdc++-v3/testsuite/Makefile.in (working copy) @@ -301,7 +301,7 @@ lists_of_files = \ extract_symvers = $(glibcxx_builddir)/scripts/extract_symvers baseline_subdir := $(shell $(CXX) $(baseline_subdir_switch)) -check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10) +check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15) # Runs the testsuite, but in compile only mode. # Can be used to test sources with non-GNU FE's at various warning @@ -562,7 +562,7 @@ check-DEJAGNU $(check_DEJAGNU_normal_tar if [ -z $*$(filter-out --target_board=%, $(RUNTESTFLAGS)) ] \ [ $(filter -j, $(MFLAGS)) = -j ]; then \ $(MAKE) $(AM_MAKEFLAGS) $(check_DEJAGNU_normal_targets); \ - for idx in 0 1 2 3 4 5 6 7 8 9 10; do \ + for idx in 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15; do \ mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \ mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \ done; \ @@ -589,25 +589,35 @@ check-DEJAGNU $(check_DEJAGNU_normal_tar fi; \ dirs=`cd $$srcdir; echo [013-9][0-9]_*/*`;; \ normal1) \ - dirs=`cd $$srcdir; echo [ab]* de* [ep]*/*`;; \ + dirs=`cd $$srcdir; echo experimental/* ext/[a-m]*`;; \ normal2) \ - dirs=`cd $$srcdir; echo 2[01]_*/*`;; \ + dirs=`cd $$srcdir; echo 28_*/a*`;; \ normal3) \ - dirs=`cd $$srcdir; echo 22_*/*`;; \ + dirs=`cd $$srcdir; echo 23_*/[lu]*`;; \ normal4) \ - dirs=`cd $$srcdir; echo 23_*/[a-km-tw-z]*`;; \ + dirs=`cd $$srcdir; echo 2[459]_*/*`;; \ normal5) \ - dirs=`cd $$srcdir; echo 23_*/[luv]*`;; \ + dirs=`cd $$srcdir; echo 2[01]_*/*`;; \ normal6) \ - dirs=`cd $$srcdir; echo 2[459]_*/*`;; \ + dirs=`cd $$srcdir; echo 23_*/[m-tw-z]*`;; \ normal7) \ - dirs=`cd $$srcdir; echo 26_*/* 28_*/[c-z]*`;; \ + dirs=`cd $$srcdir; echo 26_*/*`;; \ normal8) \ dirs=`cd $$srcdir; echo 27_*/*`;; \ normal9) \ - dirs=`cd $$srcdir; echo 28_*/[ab]*`;; \ + dirs=`cd $$srcdir; echo 22_*/*`;; \ normal10) \ dirs=`cd $$srcdir; echo t*/*`;; \ + normal11) \ + dirs=`cd $$srcdir; echo 28_*/b*`;; \ + normal12) \ + dirs=`cd $$srcdir; echo 28_*/[c-z]*`;; \ + normal13) \ + dirs=`cd $$srcdir; echo ext/[n-z]*`;; \ + normal14) \ + dirs=`cd $$srcdir; echo de*/* p*/* [ab]*/* 23_*/v*`;; \ + normal15) \ + dirs=`cd $$srcdir; echo 23_*/[a-k]*`;; \ esac; \ if [ -n $* ]; then cd $*; fi; \ if $(SHELL) -c $$runtest --version /dev/null 21; then \ Index: libstdc++-v3/testsuite/Makefile.am === --- libstdc++-v3/testsuite/Makefile.am (revision 215147) +++ libstdc++-v3/testsuite/Makefile.am (working copy) @@ -101,7 +101,7 @@ new-abi-baseline: @test ! -f $*/site.exp || mv $*/site.exp $*/site.bak @mv $*/site.exp.tmp $*/site.exp -check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10) +check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2
Re: [PATCH] gcc parallel make check
On 12/09/14 09:47 +, VandeVondele Joost wrote: a newer patch (v8) I'll send soon attached with updated changelog. Compared to the previously posted v6, only the libstdc++-v3/testsuite/Makefile.am has been refined to split a little more the e*/* pattern, and two quickly running goal have been merged, in addition to fixing the pre-exisiting error in some of the patterns in that file. The libstdc++ part is OK, thanks.
Re: [PATCH] gcc parallel make check
On Fri, Sep 12, 2014 at 09:47:00AM +, VandeVondele Joost wrote:
Obviously, if Jakub's patch can be made to work around the testsuite
special cases, I believe it should be superior. If not, the attached
patch is working as far as I can tell, and provides a significant
improvement over current trunk.
Here is my latest version of the patch.
With this patch I get identical test_summary output on make -k check
(completely serial testing) and make -j48 -k check from toplevel directory.
Major changes since last version:
1) I've changed the granularity, now it does O_EXCL|O_CREAT attempt
only every 10th runtest_file_p invocation rather than every iteration,
so that it spends a little bit less time in expect processes (especially
if used on slower networked filesystems etc.)
2) in guality.exp I've discovered that it counts as PASS/FAIL also the
initial check whether any guality tests should be performed; that stuff
isn't runtest_file_p guarded (of course, needs to be performed by every
runtest instance), but made the totals dependent on how many runtest
instances invoked guality.exp; IMHO we don't want the pass/fail counts
to be volatile, so I've fixed it by not counting that test into the
results
3) various other *.exp fails didn't use runtest_file_p, especially the
gcc.misc-tests/ ones, tweaked those like struct-layout-1.exp or
plugin.exp so that only the first runtest instance to encounter those
runs all of the *.exp file serially
4) fixed go-test.exp so that runtest_file_p used for a test that
has been already runtest_file_p checked is not testing for parallel
execution - otherwise the numbers in go-parallel/ directory could be
different between different instances
5) libstdc++-v3/testsuite has been changed too to use the same stuff;
note that the new xmethods.exp apparently isn't invoked for
parallel testing, but it hasn't been before either (and the tree
I've been testing on the 16way box still didn't have it). Either
it should be tested serially like abi.exp or pretty-printers.exp,
or needs to be double-checked for parallel testing (guess it
could have similar issues as guality.exp)
6) changed -a to ] [
Regtested on x86_64-linux, ok for trunk?
2014-09-12 Jakub Jelinek ja...@redhat.com
gcc/
* Makefile.in (dg_target_exps): Remove.
(check_gcc_parallelize): Change to just an upper bound number.
(check-%-subtargets): Always print the non-parallelized goals.
(check_p_vars, check_p_comma, check_p_subwork): Remove.
(check_p_count, check_p_numbers0, check_p_numbers1, check_p_numbers2,
check_p_numbers3, check_p_numbers4, check_p_numbers5,
check_p_numbers6): New variables.
(check_p_numbers): Set to sequence from 1 to .
(check_p_subdirs): Set to sequence from 1 to minimum of
$(check_p_count) and either GCC_TEST_PARALLEL_SLOTS env var if set,
or 128.
(check-%, check-parallel-%): Rewritten so that for parallelized
testing each job runs all the *.exp files, with
GCC_RUNTEST_PARALLELIZE_DIR set in environment.
gcc/go/
* Make-lang.in (check_go_parallelize): Change to just an upper bound
number.
gcc/fortran/
* Make-lang.in (check_gfortran_parallelize): Change to just an upper
bound number.
gcc/cp/
* Make-lang.in (check_g++_parallelize): Change to just an upper bound
number.
gcc/objc/
* Make-lang.in (check_objc_parallelize): Change to just an upper
bound number.
gcc/testsuite/
* lib/gcc-defs.exp (gcc_parallel_test_run_p,
gcc_parallel_test_enable): New procedures. If
GCC_RUNTEST_PARALLELIZE_DIR is set in environment, override
runtest_file_p to invoke also gcc_parallel_test_run_p.
* g++.dg/guality/guality.exp (check_guality): Save/restore
test_counts array around the body of the procedure.
* gcc.dg/guality/guality.exp (check_guality): Likewise.
* g++.dg/compat/struct-layout-1.exp: Run all the tests serially
by the first parallel runtest encountering it.
* g++.dg/plugin/plugin.exp: Likewise.
* gcc.dg/compat/struct-layout-1.exp: Likewise.
* gcc.dg/plugin/plugin.exp: Likewise.
* objc.dg/gnu-encoding/gnu-encoding.exp: Likewise.
* gcc.misc-tests/matrix1.exp: Likewise.
* gcc.misc-tests/dhry.exp: Likewise.
* gcc.misc-tests/acker1.exp: Likewise.
* gcc.misc-tests/linkage.exp: Likewise.
* gcc.misc-tests/mg.exp: Likewise.
* gcc.misc-tests/mg-2.exp: Likewise.
* gcc.misc-tests/sort2.exp: Likewise.
* gcc.misc-tests/sieve.exp: Likewise.
* gcc.misc-tests/options.exp: Likewise.
* gcc.misc-tests/help.exp: Likewise.
* go.test/go-test.exp (go-gc-tests): Use
gcc_parallel_test_enable {0, 1} around all handling of
each test.
libstdc++-v3/
* testsuite/Makefile.am
Re: [PATCH] gcc parallel make check
On Fri, Sep 12, 2014 at 06:32:41PM +0200, Jakub Jelinek wrote: Regtested on x86_64-linux, ok for trunk? Oh, forgot to say, PR56408 isn't fixed by this patch, but given the higher granularity (10 tests instead of 1) we don't happen to trigger it right now. Jakub
RE: [PATCH] gcc parallel make check
Regtested on x86_64-linux, ok for trunk? Oh, forgot to say, PR56408 isn't fixed by this patch, but given the higher granularity (10 tests instead of 1) we don't happen to trigger it right now. which means that any commit to that dir could trigger it, right ?
Re: [PATCH] gcc parallel make check
On Fri, Sep 12, 2014 at 04:36:05PM +, VandeVondele Joost wrote: Regtested on x86_64-linux, ok for trunk? Oh, forgot to say, PR56408 isn't fixed by this patch, but given the higher granularity (10 tests instead of 1) we don't happen to trigger it right now. which means that any commit to that dir could trigger it, right ? Sure, if you are unlucky. I mean, the bug needs to be fixed, just IMHO it doesn't need to be fixed immediately or as precondition of this patch, the bug existed there for a year or two, though with the patch might be more likely to be triggered. Jakub
Re: [PATCH] gcc parallel make check
On Sep 12, 2014, at 9:32 AM, Jakub Jelinek ja...@redhat.com wrote: Here is my latest version of the patch. With this patch I get identical test_summary output on make -k check (completely serial testing) and make -j48 -k check from toplevel directory. Major changes since last version: 1) I've changed the granularity, now it does O_EXCL|O_CREAT attempt only every 10th runtest_file_p invocation So, I’d love to see the numbers for 5 and 20 to double check that 10 is the right number to pick. This sort of refinement is trivial post checkin. 3) various other *.exp fails didn't use runtest_file_p, especially the gcc.misc-tests/ ones, tweaked those like struct-layout-1.exp or plugin.exp so that only the first runtest instance to encounter those runs all of the *.exp file serially Regtested on x86_64-linux, ok for trunk? Ok. Please be around after you apply it to try and sort out any major fallout. If someone can check their target post checkin (or help out pre-checkin) and report back, that would be nice. Times before and post checkin with core count -j setting would be nice. I wonder if the libstdc++ problems can be sorted out merely by finding a way to sort them so the expensive ones come early (regexp - 0regexp for example). Or, instead of sorting them by name, sort them by some other key (md5 per line). The idea then would be that the chance of all regexp tests being in one group is 0.
RE: [PATCH] gcc parallel make check
So, I’d love to see the numbers for 5 and 20 to double check that 10 is the right number to pick. This sort of refinement is trivial post checkin. So, some timings with the patch, I think this is great. Doing the testing you suggest, changing the variable doesn't influence things much (at least for Fortran, and on this system). make -j32 -k check-fortran real3m27.875s - gcc_runtest_parallelize_counter_minor == 02 (several testsuite errors: binding_label_tests_10_main.f03, binding_label_tests_11_main.f03, class_45b.f03, class_4b.f03, class_4c.f03, coarray_29_2.f90, test_common_binding_labels_3_main.f03) real3m26.234s - gcc_runtest_parallelize_counter_minor == 05 (one additional testsuite error: whole_file_31.f90) real3m36.405s - gcc_runtest_parallelize_counter_minor == 10 real3m38.736s - gcc_runtest_parallelize_counter_minor == 20 check-c real8m26.935s check-c++ real7m4.165s check real 17m45.185s
Re: [PATCH] gcc parallel make check
On Sep 12, 2014, at 9:32 AM, Jakub Jelinek ja...@redhat.com wrote: Here is my latest version of the patch. I did a timing test: Before: real0m57.198s user1m24.736s sys 0m19.816s after: real0m28.224s user1m27.823s sys 0m22.374s This is a -j70 run on a 64 core power7 of check-objc, I picked an obscure test case that I had no reason to believe was other than ignored and certainly not engineered for and kinda small to ensure the overhead would penalize it… 50.66% faster. There is still room for improvement: $ vmstat 1 procs ---memory-- ---swap-- -io -system-- -cpu-- r b swpd free buff cache si sobibo in cs us sy id wa st 7 0 0 99046848 8515072 1674867200 0 011 0 0 100 0 0 7 0 0 99050432 8515072 1674873600 0 0 7501 9022 13 3 84 0 0 7 0 0 99029376 8515072 1674924800 0 0 7320 8777 10 2 88 0 0 7 0 0 99070656 8515072 1674944000 0 1524 7162 8156 9 2 88 1 0 7 0 0 99034560 8515072 1674982400 0 0 8096 10363 7 2 91 0 0 7 0 0 99030080 8515072 1675072000 0 0 8798 11673 8 3 90 0 0 9 0 0 99037376 8515072 1675008000 0 0 9151 12598 9 3 87 0 0 7 0 0 99024128 8515136 1675065600 0 0 9078 13168 7 3 90 0 0 10 0 0 99034496 8515136 1675148800 0 1800 8633 11675 8 3 88 1 0 8 0 0 98986304 8515136 1675129600 0 0 10159 14553 7 3 90 0 0 7 0 0 99010112 8515520 1676582400 0 0 8814 12036 10 3 87 0 0 4 0 0 99014016 8515648 1677356800 0 0 8091 10445 8 3 90 0 0 4 0 0 99064832 8515712 1677312000 0 0 5416 5071 9 2 89 0 0 3 0 0 99118976 8515712 1677318400 0 12716 4743 3533 4 1 92 2 0 3 0 0 99077504 8515840 1677324800 0 0 4525 3988 3 1 96 0 0 2 0 0 99121152 8515840 1677382400 0 0 4687 3757 3 1 97 0 0 2 0 0 99117056 8515840 1677363200 0 0 4334 3156 3 1 96 0 0 2 0 0 99105728 8515840 1677433600 0 0 4355 3246 3 1 96 0 0 3 0 0 99069120 8515904 1677363200 0 648 4902 4037 2 1 97 0 0 1 0 0 99153664 8515968 1677459200 0 0 3776 2711 2 1 97 0 0 1 0 0 99151232 8515968 1677440000 0 0 877 205 4 0 96 0 0 1 0 0 99151424 8516032 1677452800 0 236 774 466 2 0 97 0 0 2 0 0 99148032 8516032 1677465600 0 0 853 350 2 0 98 0 0 2 0 0 99146176 8516032 1677465600 0 1208 1630 1363 1 0 99 0 0 1 0 0 99156032 8516352 1677715200 0 0 1919 2104 1 0 99 0 0 0 0 0 99189376 8516416 1677651200 0 0 1181 799 2 0 98 0 0 0 0 0 99189312 8516416 1677651200 0 0 118 18 0 0 100 0 0 0 0 0 99189312 8516416 1677651200 0 0 90 18 0 0 100 0 0 0 0 0 99187968 8516416 1677651200 0 5468 196 42 0 0 100 0 0 0 0 0 99187968 8516416 1677651200 0 0 92 24 0 0 100 0 0 0 0 0 99188032 8516416 1677651200 0 0 146 37 0 0 100 0 0 0 0 0 99188160 8516416 1677651200 0 128 91 36 0 0 100 0 0 1 0 0 99188160 8516416 1677651200 0 0 74 16 0 0 100 0 0 0 0 0 99188160 8516416 1677651200 0 0 72 20 0 0 100 0 0 0 0 0 99188224 8516416 1677651200 0 0 76 22 0 0 100 0 0 0 0 0 99188224 8516416 1677651200 0 0 118 29 0 0 100 0 0 which averages to 95% idle. I changed: check_objc_parallelize = 6 to check_objc_parallelize = 70 to try and get it to go faster: real0m21.252s user3m21.035s sys 1m9.937s :-( 7 seconds (24.6%) faster, but consumes 146% more resources to see the benefit. with the filesystem update to 2 (instead of 10): real0m22.478s user4m38.564s sys 1m25.293s and filesystem update 5: real0m21.665s user3m51.615s sys 1m16.005s and filesystem update 20: real0m22.681s user3m2.746s sys 1m5.576s a -j1 filesystem update 20 for comparison: real1m48.127s user1m17.953s sys 0m17.191s a -j1 check_objc_parallelize 6 filesystem update 10 for comparison: real1m47.552s user1m17.410s sys 0m16.909s a -j70 check_objc_parallelize 1 filesystem update 10 for comparison: real0m21.292s user3m17.368s sys 1m10.106s a -j70 check_objc_parallelize 1 filesystem update 2 for comparison: real0m21.976s user4m37.600s sys 1m26.598s a -j70 check_objc_parallelize 1 filesystem update 200 for comparison: real1m12.319s user
Re: [PATCH] gcc parallel make check
On Fri, Sep 12, 2014 at 04:42:25PM -0700, Mike Stump wrote: curious, when I run atomic.exp=stdatom\*.c: gcc.dg/atomic/atomic.exp completed in 30 seconds. atomic.exp=c\*.c takes 522 seconds with 3, 2, 5 and 4 being the worst offenders. That's the @if [ -z $(filter-out --target_board=%,$(filter-out --extra_opts%,$(RUNTESTFLAGS))) ] \ [ $(filter -j, $(MFLAGS)) = -j ]; then \ i.e. if you specify anything in RUNTESTFLAGS other than --target_board= or --extra_opts, it is not parallelized. This was done previously because parallelization required setting the flags to something different (manually created *.exp list). The first [] could perhaps be removed now, if one e.g. RUNTESTFLAGS=atomic.exp etc. with sufficiently enough tests, parallelization will be still worth it. I've been worried about the quick cases where parallelization is not beneficial, like make check-gcc \ RUNTESTFLAGS=dg.exp=pr60123.c or similar, but one doesn't usually pass -jN in that case. So yes, the [ -z $(filter-out --target_board=%,$(filter-out --extra_opts%,$(RUNTESTFLAGS))) ] can be dropped (not in libstdc++ though, there are abi.exp and prettyprinters.exp still run serially, though even that could be handled the struct-layout-1.exp way, of running it by the first instance to encounter those with small changes in those *.exp files). Jakub
RE: [PATCH] gcc parallel make check
a newer patch (v8) I'll send soon attached with updated changelog. Compared to the previously posted v6, only the libstdc++-v3/testsuite/Makefile.am has been refined to split a little more the e*/* pattern, and two quickly running goal have been merged, in addition to fixing the pre-exisiting error in some of the patterns in that file. Checked comparing testsuite results before after. Obviously, if Jakub's patch can be made to work around the testsuite special cases, I believe it should be superior. If not, the attached patch is working as far as I can tell, and provides a significant improvement over current trunk. Joostcontrib/ChangeLog 2014-09-12 Joost VandeVondele vond...@gcc.gnu.org * generate_tcl_patterns.sh: New file. gcc/fortran/ChangeLog 2014-09-12 Joost VandeVondele vond...@gcc.gnu.org * Make-lang.in (check_gfortran_parallelize): Improved parallelism. gcc/Changelog 2014-09-12 Joost VandeVondele vond...@gcc.gnu.org * Makefile.in (check_gcc_parallelize): Improved parallelism. (check_p_numbers): Increase maximum value. (dg_target_exps): Mention targets as separate words only. (null,space,comma,dg_target_exps_p1,dg_target_exps_p2, dg_target_exps_p3,dg_target_exps_p4): New variables. gcc/cp/ChangeLog 2014-09-12 Joost VandeVondele vond...@gcc.gnu.org * Make-lang.in (check_g++_parallelize): Improved parallelism. libstdc++-v3/ChangeLog 2014-09-12 Joost VandeVondele vond...@gcc.gnu.org * testsuite/Makefile.am (check_DEJAGNU_normal_targets): Add check-DEJAGNUnormal[11-15]. (check-DEJAGNU): Split into 15 jobs for parallel testing, correct pattern. * testsuite/Makefile.in: Regenerated. Index: libstdc++-v3/testsuite/Makefile.in === --- libstdc++-v3/testsuite/Makefile.in (revision 215147) +++ libstdc++-v3/testsuite/Makefile.in (working copy) @@ -301,7 +301,7 @@ lists_of_files = \ extract_symvers = $(glibcxx_builddir)/scripts/extract_symvers baseline_subdir := $(shell $(CXX) $(baseline_subdir_switch)) -check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10) +check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15) # Runs the testsuite, but in compile only mode. # Can be used to test sources with non-GNU FE's at various warning @@ -562,7 +562,7 @@ check-DEJAGNU $(check_DEJAGNU_normal_tar if [ -z $*$(filter-out --target_board=%, $(RUNTESTFLAGS)) ] \ [ $(filter -j, $(MFLAGS)) = -j ]; then \ $(MAKE) $(AM_MAKEFLAGS) $(check_DEJAGNU_normal_targets); \ - for idx in 0 1 2 3 4 5 6 7 8 9 10; do \ + for idx in 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15; do \ mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \ mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \ done; \ @@ -589,25 +589,35 @@ check-DEJAGNU $(check_DEJAGNU_normal_tar fi; \ dirs=`cd $$srcdir; echo [013-9][0-9]_*/*`;; \ normal1) \ - dirs=`cd $$srcdir; echo [ab]* de* [ep]*/*`;; \ + dirs=`cd $$srcdir; echo experimental/* ext/[a-m]*`;; \ normal2) \ - dirs=`cd $$srcdir; echo 2[01]_*/*`;; \ + dirs=`cd $$srcdir; echo 28_*/a*`;; \ normal3) \ - dirs=`cd $$srcdir; echo 22_*/*`;; \ + dirs=`cd $$srcdir; echo 23_*/[lu]*`;; \ normal4) \ - dirs=`cd $$srcdir; echo 23_*/[a-km-tw-z]*`;; \ + dirs=`cd $$srcdir; echo 2[459]_*/*`;; \ normal5) \ - dirs=`cd $$srcdir; echo 23_*/[luv]*`;; \ + dirs=`cd $$srcdir; echo 2[01]_*/*`;; \ normal6) \ - dirs=`cd $$srcdir; echo 2[459]_*/*`;; \ + dirs=`cd $$srcdir; echo 23_*/[m-tw-z]*`;; \ normal7) \ - dirs=`cd $$srcdir; echo 26_*/* 28_*/[c-z]*`;; \ + dirs=`cd $$srcdir; echo 26_*/*`;; \ normal8) \ dirs=`cd $$srcdir; echo 27_*/*`;; \ normal9) \ - dirs=`cd $$srcdir; echo 28_*/[ab]*`;; \ + dirs=`cd $$srcdir; echo 22_*/*`;; \ normal10) \ dirs=`cd $$srcdir; echo t*/*`;; \ + normal11) \ + dirs=`cd $$srcdir; echo 28_*/b*`;; \ + normal12) \ + dirs=`cd $$srcdir; echo 28_*/[c-z]*`;; \ + normal13) \ + dirs=`cd $$srcdir; echo ext/[n-z]*`;; \ + normal14) \ + dirs=`cd $$srcdir; echo de*/* p*/* [ab]*/* 23_*/v*`;; \ + normal15) \ + dirs=`cd $$srcdir; echo 23_*/[a-k]*`;; \ esac; \ if [ -n $* ]; then cd $*; fi; \ if $(SHELL) -c $$runtest --version /dev/null 21; then \ Index: libstdc++-v3/testsuite/Makefile.am === --- libstdc++-v3/testsuite/Makefile.am (revision 215147) +++ libstdc++-v3/testsuite/Makefile.am (working copy) @@ -101,7 +101,7 @@ new-abi-baseline: @test ! -f $*/site.exp || mv $*/site.exp $*/site.bak @mv $*/site.exp.tmp $*/site.exp -check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10) +check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2
Re: [PATCH] gcc parallel make check
On 12/09/14 09:47 +, VandeVondele Joost wrote: a newer patch (v8) I'll send soon attached with updated changelog. Compared to the previously posted v6, only the libstdc++-v3/testsuite/Makefile.am has been refined to split a little more the e*/* pattern, and two quickly running goal have been merged, in addition to fixing the pre-exisiting error in some of the patterns in that file. The libstdc++ part is OK, thanks.
Re: [PATCH] gcc parallel make check
On Fri, Sep 12, 2014 at 09:47:00AM +, VandeVondele Joost wrote:
Obviously, if Jakub's patch can be made to work around the testsuite
special cases, I believe it should be superior. If not, the attached
patch is working as far as I can tell, and provides a significant
improvement over current trunk.
Here is my latest version of the patch.
With this patch I get identical test_summary output on make -k check
(completely serial testing) and make -j48 -k check from toplevel directory.
Major changes since last version:
1) I've changed the granularity, now it does O_EXCL|O_CREAT attempt
only every 10th runtest_file_p invocation rather than every iteration,
so that it spends a little bit less time in expect processes (especially
if used on slower networked filesystems etc.)
2) in guality.exp I've discovered that it counts as PASS/FAIL also the
initial check whether any guality tests should be performed; that stuff
isn't runtest_file_p guarded (of course, needs to be performed by every
runtest instance), but made the totals dependent on how many runtest
instances invoked guality.exp; IMHO we don't want the pass/fail counts
to be volatile, so I've fixed it by not counting that test into the
results
3) various other *.exp fails didn't use runtest_file_p, especially the
gcc.misc-tests/ ones, tweaked those like struct-layout-1.exp or
plugin.exp so that only the first runtest instance to encounter those
runs all of the *.exp file serially
4) fixed go-test.exp so that runtest_file_p used for a test that
has been already runtest_file_p checked is not testing for parallel
execution - otherwise the numbers in go-parallel/ directory could be
different between different instances
5) libstdc++-v3/testsuite has been changed too to use the same stuff;
note that the new xmethods.exp apparently isn't invoked for
parallel testing, but it hasn't been before either (and the tree
I've been testing on the 16way box still didn't have it). Either
it should be tested serially like abi.exp or pretty-printers.exp,
or needs to be double-checked for parallel testing (guess it
could have similar issues as guality.exp)
6) changed -a to ] [
Regtested on x86_64-linux, ok for trunk?
2014-09-12 Jakub Jelinek ja...@redhat.com
gcc/
* Makefile.in (dg_target_exps): Remove.
(check_gcc_parallelize): Change to just an upper bound number.
(check-%-subtargets): Always print the non-parallelized goals.
(check_p_vars, check_p_comma, check_p_subwork): Remove.
(check_p_count, check_p_numbers0, check_p_numbers1, check_p_numbers2,
check_p_numbers3, check_p_numbers4, check_p_numbers5,
check_p_numbers6): New variables.
(check_p_numbers): Set to sequence from 1 to .
(check_p_subdirs): Set to sequence from 1 to minimum of
$(check_p_count) and either GCC_TEST_PARALLEL_SLOTS env var if set,
or 128.
(check-%, check-parallel-%): Rewritten so that for parallelized
testing each job runs all the *.exp files, with
GCC_RUNTEST_PARALLELIZE_DIR set in environment.
gcc/go/
* Make-lang.in (check_go_parallelize): Change to just an upper bound
number.
gcc/fortran/
* Make-lang.in (check_gfortran_parallelize): Change to just an upper
bound number.
gcc/cp/
* Make-lang.in (check_g++_parallelize): Change to just an upper bound
number.
gcc/objc/
* Make-lang.in (check_objc_parallelize): Change to just an upper
bound number.
gcc/testsuite/
* lib/gcc-defs.exp (gcc_parallel_test_run_p,
gcc_parallel_test_enable): New procedures. If
GCC_RUNTEST_PARALLELIZE_DIR is set in environment, override
runtest_file_p to invoke also gcc_parallel_test_run_p.
* g++.dg/guality/guality.exp (check_guality): Save/restore
test_counts array around the body of the procedure.
* gcc.dg/guality/guality.exp (check_guality): Likewise.
* g++.dg/compat/struct-layout-1.exp: Run all the tests serially
by the first parallel runtest encountering it.
* g++.dg/plugin/plugin.exp: Likewise.
* gcc.dg/compat/struct-layout-1.exp: Likewise.
* gcc.dg/plugin/plugin.exp: Likewise.
* objc.dg/gnu-encoding/gnu-encoding.exp: Likewise.
* gcc.misc-tests/matrix1.exp: Likewise.
* gcc.misc-tests/dhry.exp: Likewise.
* gcc.misc-tests/acker1.exp: Likewise.
* gcc.misc-tests/linkage.exp: Likewise.
* gcc.misc-tests/mg.exp: Likewise.
* gcc.misc-tests/mg-2.exp: Likewise.
* gcc.misc-tests/sort2.exp: Likewise.
* gcc.misc-tests/sieve.exp: Likewise.
* gcc.misc-tests/options.exp: Likewise.
* gcc.misc-tests/help.exp: Likewise.
* go.test/go-test.exp (go-gc-tests): Use
gcc_parallel_test_enable {0, 1} around all handling of
each test.
libstdc++-v3/
* testsuite/Makefile.am
Re: [PATCH] gcc parallel make check
On Fri, Sep 12, 2014 at 06:32:41PM +0200, Jakub Jelinek wrote: Regtested on x86_64-linux, ok for trunk? Oh, forgot to say, PR56408 isn't fixed by this patch, but given the higher granularity (10 tests instead of 1) we don't happen to trigger it right now. Jakub
RE: [PATCH] gcc parallel make check
Regtested on x86_64-linux, ok for trunk? Oh, forgot to say, PR56408 isn't fixed by this patch, but given the higher granularity (10 tests instead of 1) we don't happen to trigger it right now. which means that any commit to that dir could trigger it, right ?
Re: [PATCH] gcc parallel make check
On Fri, Sep 12, 2014 at 04:36:05PM +, VandeVondele Joost wrote: Regtested on x86_64-linux, ok for trunk? Oh, forgot to say, PR56408 isn't fixed by this patch, but given the higher granularity (10 tests instead of 1) we don't happen to trigger it right now. which means that any commit to that dir could trigger it, right ? Sure, if you are unlucky. I mean, the bug needs to be fixed, just IMHO it doesn't need to be fixed immediately or as precondition of this patch, the bug existed there for a year or two, though with the patch might be more likely to be triggered. Jakub
Re: [PATCH] gcc parallel make check
On Sep 12, 2014, at 9:32 AM, Jakub Jelinek ja...@redhat.com wrote: Here is my latest version of the patch. With this patch I get identical test_summary output on make -k check (completely serial testing) and make -j48 -k check from toplevel directory. Major changes since last version: 1) I've changed the granularity, now it does O_EXCL|O_CREAT attempt only every 10th runtest_file_p invocation So, I’d love to see the numbers for 5 and 20 to double check that 10 is the right number to pick. This sort of refinement is trivial post checkin. 3) various other *.exp fails didn't use runtest_file_p, especially the gcc.misc-tests/ ones, tweaked those like struct-layout-1.exp or plugin.exp so that only the first runtest instance to encounter those runs all of the *.exp file serially Regtested on x86_64-linux, ok for trunk? Ok. Please be around after you apply it to try and sort out any major fallout. If someone can check their target post checkin (or help out pre-checkin) and report back, that would be nice. Times before and post checkin with core count -j setting would be nice. I wonder if the libstdc++ problems can be sorted out merely by finding a way to sort them so the expensive ones come early (regexp - 0regexp for example). Or, instead of sorting them by name, sort them by some other key (md5 per line). The idea then would be that the chance of all regexp tests being in one group is 0.
RE: [PATCH] gcc parallel make check
So, I’d love to see the numbers for 5 and 20 to double check that 10 is the right number to pick. This sort of refinement is trivial post checkin. So, some timings with the patch, I think this is great. Doing the testing you suggest, changing the variable doesn't influence things much (at least for Fortran, and on this system). make -j32 -k check-fortran real3m27.875s - gcc_runtest_parallelize_counter_minor == 02 (several testsuite errors: binding_label_tests_10_main.f03, binding_label_tests_11_main.f03, class_45b.f03, class_4b.f03, class_4c.f03, coarray_29_2.f90, test_common_binding_labels_3_main.f03) real3m26.234s - gcc_runtest_parallelize_counter_minor == 05 (one additional testsuite error: whole_file_31.f90) real3m36.405s - gcc_runtest_parallelize_counter_minor == 10 real3m38.736s - gcc_runtest_parallelize_counter_minor == 20 check-c real8m26.935s check-c++ real7m4.165s check real 17m45.185s
Re: [PATCH] gcc parallel make check
On Sep 12, 2014, at 9:32 AM, Jakub Jelinek ja...@redhat.com wrote: Here is my latest version of the patch. I did a timing test: Before: real0m57.198s user1m24.736s sys 0m19.816s after: real0m28.224s user1m27.823s sys 0m22.374s This is a -j70 run on a 64 core power7 of check-objc, I picked an obscure test case that I had no reason to believe was other than ignored and certainly not engineered for and kinda small to ensure the overhead would penalize it… 50.66% faster. There is still room for improvement: $ vmstat 1 procs ---memory-- ---swap-- -io -system-- -cpu-- r b swpd free buff cache si sobibo in cs us sy id wa st 7 0 0 99046848 8515072 1674867200 0 011 0 0 100 0 0 7 0 0 99050432 8515072 1674873600 0 0 7501 9022 13 3 84 0 0 7 0 0 99029376 8515072 1674924800 0 0 7320 8777 10 2 88 0 0 7 0 0 99070656 8515072 1674944000 0 1524 7162 8156 9 2 88 1 0 7 0 0 99034560 8515072 1674982400 0 0 8096 10363 7 2 91 0 0 7 0 0 99030080 8515072 1675072000 0 0 8798 11673 8 3 90 0 0 9 0 0 99037376 8515072 1675008000 0 0 9151 12598 9 3 87 0 0 7 0 0 99024128 8515136 1675065600 0 0 9078 13168 7 3 90 0 0 10 0 0 99034496 8515136 1675148800 0 1800 8633 11675 8 3 88 1 0 8 0 0 98986304 8515136 1675129600 0 0 10159 14553 7 3 90 0 0 7 0 0 99010112 8515520 1676582400 0 0 8814 12036 10 3 87 0 0 4 0 0 99014016 8515648 1677356800 0 0 8091 10445 8 3 90 0 0 4 0 0 99064832 8515712 1677312000 0 0 5416 5071 9 2 89 0 0 3 0 0 99118976 8515712 1677318400 0 12716 4743 3533 4 1 92 2 0 3 0 0 99077504 8515840 1677324800 0 0 4525 3988 3 1 96 0 0 2 0 0 99121152 8515840 1677382400 0 0 4687 3757 3 1 97 0 0 2 0 0 99117056 8515840 1677363200 0 0 4334 3156 3 1 96 0 0 2 0 0 99105728 8515840 1677433600 0 0 4355 3246 3 1 96 0 0 3 0 0 99069120 8515904 1677363200 0 648 4902 4037 2 1 97 0 0 1 0 0 99153664 8515968 1677459200 0 0 3776 2711 2 1 97 0 0 1 0 0 99151232 8515968 1677440000 0 0 877 205 4 0 96 0 0 1 0 0 99151424 8516032 1677452800 0 236 774 466 2 0 97 0 0 2 0 0 99148032 8516032 1677465600 0 0 853 350 2 0 98 0 0 2 0 0 99146176 8516032 1677465600 0 1208 1630 1363 1 0 99 0 0 1 0 0 99156032 8516352 1677715200 0 0 1919 2104 1 0 99 0 0 0 0 0 99189376 8516416 1677651200 0 0 1181 799 2 0 98 0 0 0 0 0 99189312 8516416 1677651200 0 0 118 18 0 0 100 0 0 0 0 0 99189312 8516416 1677651200 0 0 90 18 0 0 100 0 0 0 0 0 99187968 8516416 1677651200 0 5468 196 42 0 0 100 0 0 0 0 0 99187968 8516416 1677651200 0 0 92 24 0 0 100 0 0 0 0 0 99188032 8516416 1677651200 0 0 146 37 0 0 100 0 0 0 0 0 99188160 8516416 1677651200 0 128 91 36 0 0 100 0 0 1 0 0 99188160 8516416 1677651200 0 0 74 16 0 0 100 0 0 0 0 0 99188160 8516416 1677651200 0 0 72 20 0 0 100 0 0 0 0 0 99188224 8516416 1677651200 0 0 76 22 0 0 100 0 0 0 0 0 99188224 8516416 1677651200 0 0 118 29 0 0 100 0 0 which averages to 95% idle. I changed: check_objc_parallelize = 6 to check_objc_parallelize = 70 to try and get it to go faster: real0m21.252s user3m21.035s sys 1m9.937s :-( 7 seconds (24.6%) faster, but consumes 146% more resources to see the benefit. with the filesystem update to 2 (instead of 10): real0m22.478s user4m38.564s sys 1m25.293s and filesystem update 5: real0m21.665s user3m51.615s sys 1m16.005s and filesystem update 20: real0m22.681s user3m2.746s sys 1m5.576s a -j1 filesystem update 20 for comparison: real1m48.127s user1m17.953s sys 0m17.191s a -j1 check_objc_parallelize 6 filesystem update 10 for comparison: real1m47.552s user1m17.410s sys 0m16.909s a -j70 check_objc_parallelize 1 filesystem update 10 for comparison: real0m21.292s user3m17.368s sys 1m10.106s a -j70 check_objc_parallelize 1 filesystem update 2 for comparison: real0m21.976s user4m37.600s sys 1m26.598s a -j70 check_objc_parallelize 1 filesystem update 200 for comparison: real1m12.319s user
Re: [PATCH] gcc parallel make check
On Fri, Sep 12, 2014 at 04:42:25PM -0700, Mike Stump wrote: curious, when I run atomic.exp=stdatom\*.c: gcc.dg/atomic/atomic.exp completed in 30 seconds. atomic.exp=c\*.c takes 522 seconds with 3, 2, 5 and 4 being the worst offenders. That's the @if [ -z $(filter-out --target_board=%,$(filter-out --extra_opts%,$(RUNTESTFLAGS))) ] \ [ $(filter -j, $(MFLAGS)) = -j ]; then \ i.e. if you specify anything in RUNTESTFLAGS other than --target_board= or --extra_opts, it is not parallelized. This was done previously because parallelization required setting the flags to something different (manually created *.exp list). The first [] could perhaps be removed now, if one e.g. RUNTESTFLAGS=atomic.exp etc. with sufficiently enough tests, parallelization will be still worth it. I've been worried about the quick cases where parallelization is not beneficial, like make check-gcc \ RUNTESTFLAGS=dg.exp=pr60123.c or similar, but one doesn't usually pass -jN in that case. So yes, the [ -z $(filter-out --target_board=%,$(filter-out --extra_opts%,$(RUNTESTFLAGS))) ] can be dropped (not in libstdc++ though, there are abi.exp and prettyprinters.exp still run serially, though even that could be handled the struct-layout-1.exp way, of running it by the first instance to encounter those with small changes in those *.exp files). Jakub
RE: [PATCH] RE: gcc parallel make check
Jakub, First of all, the -j2 testing shows more tests tested in gcc and libstdc++: -# of expected passes 10133 +# of expected passes 10152 +PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors) [...] Not sure where the bug is, could be e.g. in i386.exp for gcc, but for libstdc++ less likely to be there rather than in the split. I looked into this, and believe this problem is already in current trunk, and not due to my patch. I.e. unmodified trunk also has these tests executed several times: libstdc++-v3/testsuite/normal4/libstdc++.log.sep:PASS: 23_containers/map/modifiers/erase/abi_tag.cc libstdc++-v3/testsuite/normal1/libstdc++.log.sep:PASS: 23_containers/map/modifiers/erase/abi_tag.cc I believe the current trunk pattern could indeed match those twice (Makefile.in in trunk): normal1) \ dirs=`cd $$srcdir; echo [ab]* de* [ep]*/*`;; \ normal4) \ dirs=`cd $$srcdir; echo 23_*/[a-km-tw-z]*`;; \ could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? Joost
Re: [PATCH] gcc parallel make check
On Wed, Sep 10, 2014 at 11:23:34PM +0200, Jakub Jelinek wrote:
On Wed, Sep 10, 2014 at 11:08:22PM +0200, Jakub Jelinek wrote:
Perhaps better approach might be if we have some way how to synchronize
among
multiple expect processes and spawn only as many expects (of course, per
check target) as there are CPUs. E.g. if mkdir is atomic on all
hosts/filesystems we care about, we could have some shared directory that
make would clear before spawning all the expects, and after checking
runtest_file_p we could attempt to mkdir something (e.g. testcase filename
with $(srcdir) part removed, or *.exp filename / counter what test are we
considering or something similar) in the shared directory, if that would
succeed, it would tell us that we are the process that should run the test,
if that failed, we'd know some other runtest did that.
Or perhaps not for every single test, but every 10 or 100 tests or
something.
E.g. we could just override runtest_file_p itself, so that it would first
call the original dejagnu version, and then do this check.
Seems file mkdir in tcl doesn't error on pre-existing directory, so perhaps
[open $path {WRONLY EXCL CREAT}] ?
Now, does this work properly on all hosts we care about?
Here is a proof of concept on the tcl side.
To get a large seq of numbers in the Makefile, I guess we can use something
like
check_p_numbers0:=1 2 3 4 5 6 7 8 9
check_p_numbers1:=0 $(check_p_numbers0)
check_p_numbers2:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers1)))
check_p_numbers3:=$(patsubst %,0%,$(check_p_numbers1)) $(check_p_numbers2)
check_p_numbers4:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers3)))
check_p_numbers5:=$(patsubst %,0%,$(check_p_numbers3)) $(check_p_numbers4)
check_p_numbers6:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers5)))
check_p_numbers:=$(check_p_numbers0) $(check_p_numbers2) $(check_p_numbers4)
$(check_p_numbers6)
(and then what
check_p_subdirs=$(wordlist 1,$(words
$(check_$*_parallelize)),$(check_p_numbers))
uses, just with $(check_$*_parallelize) replaced with something to match the
number of desired goals.
Looking at some of the *.exp tests, it seems only some of them (though, the
majority of the time consuming ones) actually use runtest_file_p, e.g.
compat.exp or struct-layout-1.exp and several others don't.
So, IMHO what we should do in the Makefile is, right inside
@if [ -z $(filter-out --target_board=%,$(filter-out
--extra_opts%,$(RUNTESTFLAGS))) ] \
[ $(filter -j, $(MFLAGS)) = -j ]; then \
first rm -rf $(TESTSUITEDIR)/$*-parallel; mkdir $(TESTSUITEDIR)/$*-parallel
so that we start with empty dir, compute check_p_subdirs from actual -jN
number, then in check-parallel-gcc_1 etc. goals (but not in
check-parallel-gcc) set
GCC_RUNTEST_PARALLELIZE_DIR=$(TESTSUITEDIR)/$(check_p_tool)-parallel
in the environment and use RUNTESTFLAGS with selected known to be
parallelizable *.exp files (dg.exp execute.exp compile.exp and the like),
and use all the other *.exp files for check-parallel-gcc.
Thoughts on this?
Unfortunately, not sure how would that work with the
check-subtargets stuff if people are used to parallelize testing across
multiple machines (but it is unclear to me how they are merging the log/sum
files from the multiple machines anyway). Not sure if this works over
NFS/AFS and other networked filesystems, if it does, supposedly they could
arrange for the *-parallel directories to be shared.
I can't find how to query the -jN value passed to make check by the user
though, both $(MFLAGS) and $(MAKEFLAGS) only contain something like
--jobserver-fds=3,5 -j from which it is not possible to find out how many
goals would be the upper reasonable limit. Running too many goals would
waste time (once scheduled, the goal would only wildcard all the test, and
for all of them find in the *-parallel directory the test has been run
already), running too few could prevent good parallelization.
--- gcc/testsuite/lib/gcc-defs.exp.jj 2014-09-01 09:43:28.0 +0200
+++ gcc/testsuite/lib/gcc-defs.exp 2014-09-11 08:37:43.871943270 +0200
@@ -188,6 +188,30 @@ if { [info procs runtest_file_p] == }
}
}
+if { [info exists env(GCC_RUNTEST_PARALLELIZE_DIR)] \
+ [info procs runtest_file_p] != [list] \
+ [info procs gcc_parallelize_saved_runtest_file_p] == [list] } then {
+rename runtest_file_p gcc_parallelize_saved_runtest_file_p
+global gcc_runtest_parallelize_counter
+
+set gcc_runtest_parallelize_counter 0
+proc runtest_file_p { runtests testcase } {
+ global gcc_runtest_parallelize_counter
+ if ![gcc_parallelize_saved_runtest_file_p $runtests $testcase] {
+ return 0
+ }
+
+ set dir [getenv GCC_RUNTEST_PARALLELIZE_DIR]
+ set path $dir/$gcc_runtest_parallelize_counter
+ set gcc_runtest_parallelize_counter [expr
{$gcc_runtest_parallelize_counter + 1}]
+ if {![catch
Re: [PATCH] gcc parallel make check
On Thu, Sep 11, 2014 at 09:51:23AM +0200, Jakub Jelinek wrote:
I can't find how to query the -jN value passed to make check by the user
though, both $(MFLAGS) and $(MAKEFLAGS) only contain something like
--jobserver-fds=3,5 -j from which it is not possible to find out how many
goals would be the upper reasonable limit. Running too many goals would
waste time (once scheduled, the goal would only wildcard all the test, and
for all of them find in the *-parallel directory the test has been run
already), running too few could prevent good parallelization.
After a little googling, it seems there is no way to do that :(, unless
one e.g. attempts to find the command line of the topmost parent make
and scan it through ps or something.
There is an option to touch say *-parallel/finished file once any of the
check-parallel-gcc-{1,2,...} goals is done (because when it finishes, it
means all the tests for the particular check-$lang that are parallelizable
have either finished, or at least touched their file) and not start runtest
at all if finished already exists, but guess it would be still undesirable to
have
tens of thousands of goals by default, so perhaps we could go with say
128 subgoals by default and have some env var to override it, so on the
really highly parallel boxes you'd specify
make -j512 -k check GCC_TEST_PARALLEL_SLOTS=512
or similar.
Jakub
RE: [PATCH] RE: gcc parallel make check
could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? I've checked that this fixes the bug in the current trunk split. I.e. files are stil tested, but now only once. Consider this change added to the previously submitted patch.
Re: [PATCH] RE: gcc parallel make check
On 11 September 2014 07:22, VandeVondele Joost wrote: Jakub, First of all, the -j2 testing shows more tests tested in gcc and libstdc++: -# of expected passes 10133 +# of expected passes 10152 +PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors) [...] Not sure where the bug is, could be e.g. in i386.exp for gcc, but for libstdc++ less likely to be there rather than in the split. I looked into this, and believe this problem is already in current trunk, and not due to my patch. I.e. unmodified trunk also has these tests executed several times: libstdc++-v3/testsuite/normal4/libstdc++.log.sep:PASS: 23_containers/map/modifiers/erase/abi_tag.cc libstdc++-v3/testsuite/normal1/libstdc++.log.sep:PASS: 23_containers/map/modifiers/erase/abi_tag.cc I believe the current trunk pattern could indeed match those twice (Makefile.in in trunk): normal1) \ dirs=`cd $$srcdir; echo [ab]* de* [ep]*/*`;; \ normal4) \ dirs=`cd $$srcdir; echo 23_*/[a-km-tw-z]*`;; \ could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? Yes, we are running these tests multiple times: PASS: 23_containers/map/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multimap/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multiset/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 26_numerics/complex/abi_tag.cc (test for excess errors) I'll fix that.
RE: [PATCH] RE: gcc parallel make check
could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? Yes, we are running these tests multiple times: PASS: 23_containers/map/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multimap/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multiset/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 26_numerics/complex/abi_tag.cc (test for excess errors) I'll fix that. Actually, the proper pattern should presumably be '[ab]*/* de*/* [ep]*/*' even though it seems to make no difference in testing. I'll have this included in yet another version of the parallel make check patch (plus some further reschuffling as requested by Jakub), so I think there is no need for you to fix this now.
Re: [PATCH] RE: gcc parallel make check
On 11 September 2014 15:45, VandeVondele Joost joost.vandevond...@mat.ethz.ch wrote: could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? Yes, we are running these tests multiple times: PASS: 23_containers/map/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multimap/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multiset/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 26_numerics/complex/abi_tag.cc (test for excess errors) I'll fix that. Actually, the proper pattern should presumably be '[ab]*/* de*/* [ep]*/*' even though it seems to make no difference in testing. Yes, that's what I'm testing. I'll have this included in yet another version of the parallel make check patch (plus some further reschuffling as requested by Jakub), so I think there is no need for you to fix this now. This can (and should) be fixed now, without waiting for some other change.
Re: [PATCH] gcc parallel make check
On Thu, Sep 11, 2014 at 10:06:40AM +0200, Jakub Jelinek wrote:
There is an option to touch say *-parallel/finished file once any of the
check-parallel-gcc-{1,2,...} goals is done (because when it finishes, it
means all the tests for the particular check-$lang that are parallelizable
have either finished, or at least touched their file) and not start runtest
at all if finished already exists, but guess it would be still undesirable to
have
tens of thousands of goals by default, so perhaps we could go with say
128 subgoals by default and have some env var to override it, so on the
really highly parallel boxes you'd specify
make -j512 -k check GCC_TEST_PARALLEL_SLOTS=512
or similar.
Here is a patch I'm testing now:
--- gcc/Makefile.in.jj 2014-09-08 22:12:56.0 +0200
+++ gcc/Makefile.in 2014-09-11 16:06:36.641219430 +0200
@@ -513,34 +513,10 @@ xm_include_list=@xm_include_list@
xm_defines=@xm_defines@
lang_checks=
lang_checks_parallelized=
-dg_target_exps:=aarch64.exp,alpha.exp,arm.exp,avr.exp,bfin.exp,cris.exp
-dg_target_exps:=$(dg_target_exps),epiphany.exp,frv.exp,i386.exp,ia64.exp
-dg_target_exps:=$(dg_target_exps),m68k.exp,microblaze.exp,mips.exp,powerpc.exp
-dg_target_exps:=$(dg_target_exps),rx.exp,s390.exp,sh.exp,sparc.exp,spu.exp
-dg_target_exps:=$(dg_target_exps),tic6x.exp,xstormy16.exp
-# This lists a couple of test files that take most time during check-gcc.
-# When doing parallelized check-gcc, these can run in parallel with the
-# remaining tests. Each word in this variable stands for work for one
-# make goal and one extra make goal is added to handle all the *.exp
-# files not handled explicitly already. If multiple *.exp files
-# should be run in the same runtest invocation (usually if they aren't
-# very long running, but still should be split of from the check-parallel-$lang
-# remaining tests runtest invocation), they should be concatenated with commas.
-# Note that [a-zA-Z] wildcards need to have []s prefixed with \ (needed
-# by tcl) and as the *.exp arguments are mached both as is and with
-# */ prefixed to it in runtest_file_p, it is usually desirable to include
-# a subdirectory name.
-check_gcc_parallelize=execute.exp=execute/2* \
- execute.exp=execute/\[013-9a-fA-F\]* \
- execute.exp=execute/\[pP\]*,dg.exp \
-
execute.exp=execute/\[g-oq-zG-OQ-Z\]*,compile.exp=compile/2* \
- compile.exp=compile/\[9pP\]*,builtins.exp \
- compile.exp=compile/\[013-8a-oq-zA-OQ-Z\]* \
- dg-torture.exp,ieee.exp \
- vect.exp,unsorted.exp \
- guality.exp \
- struct-layout-1.exp,stackalign.exp \
- $(dg_target_exps)
+# Upper limit to which it is useful to parallelize this lang target.
+# It doesn't make sense to try e.g. 128 goals for small testsuites
+# like objc or go.
+check_gcc_parallelize=1
lang_opt_files=@lang_opt_files@ $(srcdir)/c-family/c.opt $(srcdir)/common.opt
lang_specs_files=@lang_specs_files@
lang_tree_files=@lang_tree_files@
@@ -3631,27 +3607,32 @@ $(filter-out $(lang_checks_parallelized)
export TCL_LIBRARY ; fi ; \
$(RUNTEST) --tool $* $(RUNTESTFLAGS))
-$(patsubst %,%-subtargets,$(filter-out
$(lang_checks_parallelized),$(lang_checks))): check-%-subtargets:
+$(patsubst %,%-subtargets,$(lang_checks)): check-%-subtargets:
@echo check-$*
check_p_tool=$(firstword $(subst _, ,$*))
-check_p_vars=$(check_$(check_p_tool)_parallelize)
+check_p_count=$(check_$(check_p_tool)_parallelize)
check_p_subno=$(word 2,$(subst _, ,$*))
-check_p_comma=,
-check_p_subwork=$(subst $(check_p_comma), ,$(if $(check_p_subno),$(word
$(check_p_subno),$(check_p_vars
-check_p_numbers=1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
+check_p_numbers0:=1 2 3 4 5 6 7 8 9
+check_p_numbers1:=0 $(check_p_numbers0)
+check_p_numbers2:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers1)))
+check_p_numbers3:=$(patsubst %,0%,$(check_p_numbers1)) $(check_p_numbers2)
+check_p_numbers4:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers3)))
+check_p_numbers5:=$(patsubst %,0%,$(check_p_numbers3)) $(check_p_numbers4)
+check_p_numbers6:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers5)))
+check_p_numbers:=$(check_p_numbers0) $(check_p_numbers2) $(check_p_numbers4)
$(check_p_numbers6)
check_p_subdir=$(subst _,,$*)
-check_p_subdirs=$(wordlist 1,$(words
$(check_$*_parallelize)),$(check_p_numbers))
+check_p_subdirs=$(wordlist 1,$(check_p_count),$(wordlist 1,$(or
$(GCC_TEST_PARALLEL_SLOTS),128),$(check_p_numbers)))
# For parallelized check-% targets, this decides whether parallelization
# is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
# but optional --target_board or --extra_opts arguments). If desirable,
# recursive make is run with check-parallel-$lang{,1,2,3,4,5} etc. goals,
Re: [PATCH] gcc parallel make check
Jakub == Jakub Jelinek ja...@redhat.com writes: Jakub I fear that is going to be too expensive, because e.g. all the Jakub caching that dejagnu and our tcl stuff does would be gone, all Jakub the tests for lp64 etc. would need to be repeated for each test. In gdb I arranged to have this stuff saved in a special cache directory. See gdb/testsuite/lib/cache.exp for the mechanism. Tom
RE: [PATCH] gcc parallel make check
Here is a patch I'm testing now:
Hi Jakub,
I also tested your patch to compare timings vs a newer patch (v8) I'll send soon
== patch v8 == make -j32 -k ==
check-fortran 4m58.178s
check-c++ ~10m
check-c ~10m
check 15m29.873s
== patch Jakub
check-c++ ~20m
check-fortran 3m31.237s
check-c 8m8
on the positive side, your patch provides a further speedup e.g. fortran and c
testing (where it splits things nicely). The libstdc++ bottleneck is not
solved, but I guess that is expected.
As you have presumably found as well, your patch introduces a number failures,
because some tests seem to have additional dependencies, either explicit or
implicit:
e.g. in gfortran.dg/binding_label_tests_10_main.f03
! { dg-do compile }
! This file must be compiled AFTER binding_label_tests_10.f03, which it
! should be because dejagnu will sort the files.
module binding_label_tests_10_main
in gfortran.dg/class_45b.f03
! { dg-do link }
! { dg-additional-sources class_45a.f03 }
This could clearly trigger as well in the current scheme of splitting, only we
have been lucky that dependencies seem to be 'well behaved' in having the same
initial letter in the filename.
Joost
Re: [PATCH] gcc parallel make check
On Thu, Sep 11, 2014 at 05:04:56PM +, VandeVondele Joost wrote: Here is a patch I'm testing now: I also tested your patch to compare timings vs a newer patch (v8) I'll send soon == patch v8 == make -j32 -k == check-fortran 4m58.178s check-c++ ~10m check-c ~10m check 15m29.873s == patch Jakub check-c++ ~20m check-fortran 3m31.237s check-c 8m8 on the positive side, your patch provides a further speedup e.g. fortran and c testing (where it splits things nicely). The libstdc++ bottleneck is not solved, but I guess that is expected. The same technique can be of course used for libstdc++, I just didn't want to do that until the -C gcc testing is changed. As you have presumably found as well, your patch introduces a number failures, because some tests seem to have additional dependencies, either explicit or implicit: I found more issues, in particular it seemed that struct-layout-1.exp, gnu-encoding.exp, plugin.exp and some go*.exp don't call runtest_file_p in the same amounts and same arguments in all invocations. And these Fortran inter-test dependencies, which Tobias told me is PR56408. Unfortunately my remote testing box is unreachable now and I'm still waiting for DDR4 modules to finish building better workstation, so can't test this right now. The patch below intends to serialize the content of the problematic *.exp tests (the first runtest to reach one of those will simply run all the tests from that *.exp file, others will skip it). For go I currently have no idea why does that happen, quick hack would be just disable parallelization of go temporarily and let Ian investigate. For PR56408 we need some fix. Jakub
Re: [PATCH] gcc parallel make check
On Thu, Sep 11, 2014 at 07:26:37PM +0200, Jakub Jelinek wrote:
right now. The patch below intends to serialize the content of the
problematic *.exp tests (the first runtest to reach one of those will simply
run all the tests from that *.exp file, others will skip it).
Forgotten patch below. BTW, something will probably need to be done about
acats too, either similar approach or just splitting the chapters into
little more jobs, because otherwise in make -C check -j48 acats dominated
the testing time for me.
--- gcc/Makefile.in.jj 2014-09-08 22:12:56.0 +0200
+++ gcc/Makefile.in 2014-09-11 16:58:01.076371437 +0200
@@ -513,34 +513,10 @@ xm_include_list=@xm_include_list@
xm_defines=@xm_defines@
lang_checks=
lang_checks_parallelized=
-dg_target_exps:=aarch64.exp,alpha.exp,arm.exp,avr.exp,bfin.exp,cris.exp
-dg_target_exps:=$(dg_target_exps),epiphany.exp,frv.exp,i386.exp,ia64.exp
-dg_target_exps:=$(dg_target_exps),m68k.exp,microblaze.exp,mips.exp,powerpc.exp
-dg_target_exps:=$(dg_target_exps),rx.exp,s390.exp,sh.exp,sparc.exp,spu.exp
-dg_target_exps:=$(dg_target_exps),tic6x.exp,xstormy16.exp
-# This lists a couple of test files that take most time during check-gcc.
-# When doing parallelized check-gcc, these can run in parallel with the
-# remaining tests. Each word in this variable stands for work for one
-# make goal and one extra make goal is added to handle all the *.exp
-# files not handled explicitly already. If multiple *.exp files
-# should be run in the same runtest invocation (usually if they aren't
-# very long running, but still should be split of from the check-parallel-$lang
-# remaining tests runtest invocation), they should be concatenated with commas.
-# Note that [a-zA-Z] wildcards need to have []s prefixed with \ (needed
-# by tcl) and as the *.exp arguments are mached both as is and with
-# */ prefixed to it in runtest_file_p, it is usually desirable to include
-# a subdirectory name.
-check_gcc_parallelize=execute.exp=execute/2* \
- execute.exp=execute/\[013-9a-fA-F\]* \
- execute.exp=execute/\[pP\]*,dg.exp \
-
execute.exp=execute/\[g-oq-zG-OQ-Z\]*,compile.exp=compile/2* \
- compile.exp=compile/\[9pP\]*,builtins.exp \
- compile.exp=compile/\[013-8a-oq-zA-OQ-Z\]* \
- dg-torture.exp,ieee.exp \
- vect.exp,unsorted.exp \
- guality.exp \
- struct-layout-1.exp,stackalign.exp \
- $(dg_target_exps)
+# Upper limit to which it is useful to parallelize this lang target.
+# It doesn't make sense to try e.g. 128 goals for small testsuites
+# like objc or go.
+check_gcc_parallelize=1
lang_opt_files=@lang_opt_files@ $(srcdir)/c-family/c.opt $(srcdir)/common.opt
lang_specs_files=@lang_specs_files@
lang_tree_files=@lang_tree_files@
@@ -3631,27 +3607,32 @@ $(filter-out $(lang_checks_parallelized)
export TCL_LIBRARY ; fi ; \
$(RUNTEST) --tool $* $(RUNTESTFLAGS))
-$(patsubst %,%-subtargets,$(filter-out
$(lang_checks_parallelized),$(lang_checks))): check-%-subtargets:
+$(patsubst %,%-subtargets,$(lang_checks)): check-%-subtargets:
@echo check-$*
check_p_tool=$(firstword $(subst _, ,$*))
-check_p_vars=$(check_$(check_p_tool)_parallelize)
+check_p_count=$(check_$(check_p_tool)_parallelize)
check_p_subno=$(word 2,$(subst _, ,$*))
-check_p_comma=,
-check_p_subwork=$(subst $(check_p_comma), ,$(if $(check_p_subno),$(word
$(check_p_subno),$(check_p_vars
-check_p_numbers=1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
+check_p_numbers0:=1 2 3 4 5 6 7 8 9
+check_p_numbers1:=0 $(check_p_numbers0)
+check_p_numbers2:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers1)))
+check_p_numbers3:=$(patsubst %,0%,$(check_p_numbers1)) $(check_p_numbers2)
+check_p_numbers4:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers3)))
+check_p_numbers5:=$(patsubst %,0%,$(check_p_numbers3)) $(check_p_numbers4)
+check_p_numbers6:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers5)))
+check_p_numbers:=$(check_p_numbers0) $(check_p_numbers2) $(check_p_numbers4)
$(check_p_numbers6)
check_p_subdir=$(subst _,,$*)
-check_p_subdirs=$(wordlist 1,$(words
$(check_$*_parallelize)),$(check_p_numbers))
+check_p_subdirs=$(wordlist 1,$(check_p_count),$(wordlist 1,$(or
$(GCC_TEST_PARALLEL_SLOTS),128),$(check_p_numbers)))
# For parallelized check-% targets, this decides whether parallelization
# is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
# but optional --target_board or --extra_opts arguments). If desirable,
# recursive make is run with check-parallel-$lang{,1,2,3,4,5} etc. goals,
# which can be executed in parallel, as they are run in separate directories.
-# check-parallel-$lang{1,2,3,4,5} etc. goals invoke runtest with the longest
-# running *.exp files from the testsuite, as determined by
RE: [PATCH] gcc parallel make check
And these Fortran inter-test dependencies, which Tobias told me is
PR56408.
For PR56408 we need some fix.
BTW, is there anything special about Fortran ? There are at least 180 test
files that contain 'dg-additional-sources' some in a very non-local way:
./objc.dg/foreach-2.m: /* { dg-additional-sources
../objc-obj-c++-shared/nsconstantstring-class-impl.m } */
Joost
Re: [PATCH] gcc parallel make check
On Thu, Sep 11, 2014 at 06:33:27PM +, VandeVondele Joost wrote:
And these Fortran inter-test dependencies, which Tobias told me is
PR56408.
For PR56408 we need some fix.
BTW, is there anything special about Fortran ? There are at least 180 test
files that contain 'dg-additional-sources' some in a very non-local way:
./objc.dg/foreach-2.m: /* { dg-additional-sources
../objc-obj-c++-shared/nsconstantstring-class-impl.m } */
gc-additional-sources is not a problem, that is the solution if you need
Fortran modules inter-TU and can do a link test, see e.g.
libgomp/testsuite/libgomp.fortran/declare-simd-{2,3}.f90 for one way how to
do that. With gc-additional-sources one command line of the compiler driver
compiles both source files, it is a single test from dejagnu POV, so
necessarily run together.
But there are some tests that want to have other TU modules and want to be
dg-do compile only, currently this uses a keep-modules hack which creates
inter-test dependencies. So we need something different.
Jakub
Re: [PATCH] gcc parallel make check
On 11.09.2014 20:33, VandeVondele Joost wrote: For PR56408 we need some fix. BTW, is there anything special about Fortran ? There are at least 180 test files that contain 'dg-additional-sources' some in a very non-local way: Well, the question is what you want to do with the different files. If you just want to compile them, e.g. for linking or executing, you are fine. However, with Fortran module's there is a .mod file produced – which gives an ordering constraint: The file with the module has to be compiled first before the other file can be compiled. (If one puts the module into the same file, one has effectively one translation unit and some bugs do not pop up in this case.) The current scheme comes at its limits in that case. Mainly because the file specified in dg-additional-sources is compiled after the one in which this line is written. That can be fine for linking/run-time tests, where one disables the by-itself compilation of the second file - and puts the module into the first file. However, as soon as one wants to do more, e.g. dg-error/dg-warning output, checking the dump/assembler etc., one is in trouble. See the files listed in the PR for issues. By contrast, for C/C++, one has a header file which is included by the preprocessor (hence before the compiler), thus, there is no ordering issue as no compiler input is generated. I don't know whether one could run into issues with precompiled header files - but there on has at least the different name *.h/*.hpp and *.c/.cc. I don't know about Ada or C++'s upcoming ISO-version of precompiled headers (modules), maybe there one runs into similar issues? See the PR for some attempts to fix it. Tobias
Re: [PATCH] gcc parallel make check
On 11 September 2014 20:19:31 Jakub Jelinek ja...@redhat.com wrote: On Thu, Sep 11, 2014 at 07:26:37PM +0200, Jakub Jelinek wrote: right now. The patch below intends to serialize the content of the problematic *.exp tests (the first runtest to reach one of those will simply run all the tests from that *.exp file, others will skip it). Forgotten patch below. BTW, something will probably need to be done about acats too, either similar approach or just splitting the chapters into little more jobs, because otherwise in make -C check -j48 acats dominated the testing time for me. + if [ -n $(check_p_subno) \ +-a -n $$GCC_RUNTEST_PARALLELIZE_DIR \ +-a -f $(TESTSUITEDIR)/$(check_p_tool)-parallel/finished ]; then \ test(1) -a and -o are obsolescent, please chain [] [] instead. Thanks -a cheers ;) Sent with AquaMail for Android http://www.aqua-mail.com
Re: [PATCH] gcc parallel make check
On Thu, Sep 11, 2014 at 11:24:08PM +0200, Bernhard Reutner-Fischer wrote: On 11 September 2014 20:19:31 Jakub Jelinek ja...@redhat.com wrote: On Thu, Sep 11, 2014 at 07:26:37PM +0200, Jakub Jelinek wrote: right now. The patch below intends to serialize the content of the problematic *.exp tests (the first runtest to reach one of those will simply run all the tests from that *.exp file, others will skip it). Forgotten patch below. BTW, something will probably need to be done about acats too, either similar approach or just splitting the chapters into little more jobs, because otherwise in make -C check -j48 acats dominated the testing time for me. + if [ -n $(check_p_subno) \ + -a -n $$GCC_RUNTEST_PARALLELIZE_DIR \ + -a -f $(TESTSUITEDIR)/$(check_p_tool)-parallel/finished ]; then \ test(1) -a and -o are obsolescent, please chain [] [] instead. That is news to me, but given the amount of test -a/-o uses e.g. in gcc/configure and hundreds of places, I'd say what we care is what is more portable to old shells. Jakub
Re: [PATCH] gcc parallel make check
On Sep 11, 2014, at 3:15 PM, Jakub Jelinek ja...@redhat.com wrote: That is news to me, but given the amount of test -a/-o uses e.g. in gcc/configure and hundreds of places, I'd say what we care is what is more portable to old shells. No, we can’t care about that. If that were true, the _ _ in the compiler source would have been fixed. Since it has not, then trivially it is portable enough. One day someone will come along and fixup the -a and -o instances for us. should be preferred.
RE: [PATCH] gcc parallel make check
For PR56408 we need some fix. BTW, is there anything special about Fortran ? There are at least 180 test files that contain 'dg-additional-sources' some in a very non-local way: The current scheme comes at its limits in that case. . See the files listed in the PR for issues. So, what about a pragmatic solution, and move the tests that rely on being serialized to a subdirectory serialized/ where, like now, we rely on the implicit ordering we have now ? At least it makes this assumption somewhat explicit. Joost
RE: [PATCH] RE: gcc parallel make check
Jakub, First of all, the -j2 testing shows more tests tested in gcc and libstdc++: -# of expected passes 10133 +# of expected passes 10152 +PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors) [...] Not sure where the bug is, could be e.g. in i386.exp for gcc, but for libstdc++ less likely to be there rather than in the split. I looked into this, and believe this problem is already in current trunk, and not due to my patch. I.e. unmodified trunk also has these tests executed several times: libstdc++-v3/testsuite/normal4/libstdc++.log.sep:PASS: 23_containers/map/modifiers/erase/abi_tag.cc libstdc++-v3/testsuite/normal1/libstdc++.log.sep:PASS: 23_containers/map/modifiers/erase/abi_tag.cc I believe the current trunk pattern could indeed match those twice (Makefile.in in trunk): normal1) \ dirs=`cd $$srcdir; echo [ab]* de* [ep]*/*`;; \ normal4) \ dirs=`cd $$srcdir; echo 23_*/[a-km-tw-z]*`;; \ could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? Joost
Re: [PATCH] gcc parallel make check
On Wed, Sep 10, 2014 at 11:23:34PM +0200, Jakub Jelinek wrote:
On Wed, Sep 10, 2014 at 11:08:22PM +0200, Jakub Jelinek wrote:
Perhaps better approach might be if we have some way how to synchronize
among
multiple expect processes and spawn only as many expects (of course, per
check target) as there are CPUs. E.g. if mkdir is atomic on all
hosts/filesystems we care about, we could have some shared directory that
make would clear before spawning all the expects, and after checking
runtest_file_p we could attempt to mkdir something (e.g. testcase filename
with $(srcdir) part removed, or *.exp filename / counter what test are we
considering or something similar) in the shared directory, if that would
succeed, it would tell us that we are the process that should run the test,
if that failed, we'd know some other runtest did that.
Or perhaps not for every single test, but every 10 or 100 tests or
something.
E.g. we could just override runtest_file_p itself, so that it would first
call the original dejagnu version, and then do this check.
Seems file mkdir in tcl doesn't error on pre-existing directory, so perhaps
[open $path {WRONLY EXCL CREAT}] ?
Now, does this work properly on all hosts we care about?
Here is a proof of concept on the tcl side.
To get a large seq of numbers in the Makefile, I guess we can use something
like
check_p_numbers0:=1 2 3 4 5 6 7 8 9
check_p_numbers1:=0 $(check_p_numbers0)
check_p_numbers2:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers1)))
check_p_numbers3:=$(patsubst %,0%,$(check_p_numbers1)) $(check_p_numbers2)
check_p_numbers4:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers3)))
check_p_numbers5:=$(patsubst %,0%,$(check_p_numbers3)) $(check_p_numbers4)
check_p_numbers6:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers5)))
check_p_numbers:=$(check_p_numbers0) $(check_p_numbers2) $(check_p_numbers4)
$(check_p_numbers6)
(and then what
check_p_subdirs=$(wordlist 1,$(words
$(check_$*_parallelize)),$(check_p_numbers))
uses, just with $(check_$*_parallelize) replaced with something to match the
number of desired goals.
Looking at some of the *.exp tests, it seems only some of them (though, the
majority of the time consuming ones) actually use runtest_file_p, e.g.
compat.exp or struct-layout-1.exp and several others don't.
So, IMHO what we should do in the Makefile is, right inside
@if [ -z $(filter-out --target_board=%,$(filter-out
--extra_opts%,$(RUNTESTFLAGS))) ] \
[ $(filter -j, $(MFLAGS)) = -j ]; then \
first rm -rf $(TESTSUITEDIR)/$*-parallel; mkdir $(TESTSUITEDIR)/$*-parallel
so that we start with empty dir, compute check_p_subdirs from actual -jN
number, then in check-parallel-gcc_1 etc. goals (but not in
check-parallel-gcc) set
GCC_RUNTEST_PARALLELIZE_DIR=$(TESTSUITEDIR)/$(check_p_tool)-parallel
in the environment and use RUNTESTFLAGS with selected known to be
parallelizable *.exp files (dg.exp execute.exp compile.exp and the like),
and use all the other *.exp files for check-parallel-gcc.
Thoughts on this?
Unfortunately, not sure how would that work with the
check-subtargets stuff if people are used to parallelize testing across
multiple machines (but it is unclear to me how they are merging the log/sum
files from the multiple machines anyway). Not sure if this works over
NFS/AFS and other networked filesystems, if it does, supposedly they could
arrange for the *-parallel directories to be shared.
I can't find how to query the -jN value passed to make check by the user
though, both $(MFLAGS) and $(MAKEFLAGS) only contain something like
--jobserver-fds=3,5 -j from which it is not possible to find out how many
goals would be the upper reasonable limit. Running too many goals would
waste time (once scheduled, the goal would only wildcard all the test, and
for all of them find in the *-parallel directory the test has been run
already), running too few could prevent good parallelization.
--- gcc/testsuite/lib/gcc-defs.exp.jj 2014-09-01 09:43:28.0 +0200
+++ gcc/testsuite/lib/gcc-defs.exp 2014-09-11 08:37:43.871943270 +0200
@@ -188,6 +188,30 @@ if { [info procs runtest_file_p] == }
}
}
+if { [info exists env(GCC_RUNTEST_PARALLELIZE_DIR)] \
+ [info procs runtest_file_p] != [list] \
+ [info procs gcc_parallelize_saved_runtest_file_p] == [list] } then {
+rename runtest_file_p gcc_parallelize_saved_runtest_file_p
+global gcc_runtest_parallelize_counter
+
+set gcc_runtest_parallelize_counter 0
+proc runtest_file_p { runtests testcase } {
+ global gcc_runtest_parallelize_counter
+ if ![gcc_parallelize_saved_runtest_file_p $runtests $testcase] {
+ return 0
+ }
+
+ set dir [getenv GCC_RUNTEST_PARALLELIZE_DIR]
+ set path $dir/$gcc_runtest_parallelize_counter
+ set gcc_runtest_parallelize_counter [expr
{$gcc_runtest_parallelize_counter + 1}]
+ if {![catch
Re: [PATCH] gcc parallel make check
On Thu, Sep 11, 2014 at 09:51:23AM +0200, Jakub Jelinek wrote:
I can't find how to query the -jN value passed to make check by the user
though, both $(MFLAGS) and $(MAKEFLAGS) only contain something like
--jobserver-fds=3,5 -j from which it is not possible to find out how many
goals would be the upper reasonable limit. Running too many goals would
waste time (once scheduled, the goal would only wildcard all the test, and
for all of them find in the *-parallel directory the test has been run
already), running too few could prevent good parallelization.
After a little googling, it seems there is no way to do that :(, unless
one e.g. attempts to find the command line of the topmost parent make
and scan it through ps or something.
There is an option to touch say *-parallel/finished file once any of the
check-parallel-gcc-{1,2,...} goals is done (because when it finishes, it
means all the tests for the particular check-$lang that are parallelizable
have either finished, or at least touched their file) and not start runtest
at all if finished already exists, but guess it would be still undesirable to
have
tens of thousands of goals by default, so perhaps we could go with say
128 subgoals by default and have some env var to override it, so on the
really highly parallel boxes you'd specify
make -j512 -k check GCC_TEST_PARALLEL_SLOTS=512
or similar.
Jakub
RE: [PATCH] RE: gcc parallel make check
could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? I've checked that this fixes the bug in the current trunk split. I.e. files are stil tested, but now only once. Consider this change added to the previously submitted patch.
Re: [PATCH] RE: gcc parallel make check
On 11 September 2014 07:22, VandeVondele Joost wrote: Jakub, First of all, the -j2 testing shows more tests tested in gcc and libstdc++: -# of expected passes 10133 +# of expected passes 10152 +PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors) [...] Not sure where the bug is, could be e.g. in i386.exp for gcc, but for libstdc++ less likely to be there rather than in the split. I looked into this, and believe this problem is already in current trunk, and not due to my patch. I.e. unmodified trunk also has these tests executed several times: libstdc++-v3/testsuite/normal4/libstdc++.log.sep:PASS: 23_containers/map/modifiers/erase/abi_tag.cc libstdc++-v3/testsuite/normal1/libstdc++.log.sep:PASS: 23_containers/map/modifiers/erase/abi_tag.cc I believe the current trunk pattern could indeed match those twice (Makefile.in in trunk): normal1) \ dirs=`cd $$srcdir; echo [ab]* de* [ep]*/*`;; \ normal4) \ dirs=`cd $$srcdir; echo 23_*/[a-km-tw-z]*`;; \ could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? Yes, we are running these tests multiple times: PASS: 23_containers/map/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multimap/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multiset/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 26_numerics/complex/abi_tag.cc (test for excess errors) I'll fix that.
RE: [PATCH] RE: gcc parallel make check
could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? Yes, we are running these tests multiple times: PASS: 23_containers/map/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multimap/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multiset/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 26_numerics/complex/abi_tag.cc (test for excess errors) I'll fix that. Actually, the proper pattern should presumably be '[ab]*/* de*/* [ep]*/*' even though it seems to make no difference in testing. I'll have this included in yet another version of the parallel make check patch (plus some further reschuffling as requested by Jakub), so I think there is no need for you to fix this now.
Re: [PATCH] RE: gcc parallel make check
On 11 September 2014 15:45, VandeVondele Joost joost.vandevond...@mat.ethz.ch wrote: could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? Yes, we are running these tests multiple times: PASS: 23_containers/map/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multimap/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multiset/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 26_numerics/complex/abi_tag.cc (test for excess errors) I'll fix that. Actually, the proper pattern should presumably be '[ab]*/* de*/* [ep]*/*' even though it seems to make no difference in testing. Yes, that's what I'm testing. I'll have this included in yet another version of the parallel make check patch (plus some further reschuffling as requested by Jakub), so I think there is no need for you to fix this now. This can (and should) be fixed now, without waiting for some other change.
Re: [PATCH] gcc parallel make check
On Thu, Sep 11, 2014 at 10:06:40AM +0200, Jakub Jelinek wrote:
There is an option to touch say *-parallel/finished file once any of the
check-parallel-gcc-{1,2,...} goals is done (because when it finishes, it
means all the tests for the particular check-$lang that are parallelizable
have either finished, or at least touched their file) and not start runtest
at all if finished already exists, but guess it would be still undesirable to
have
tens of thousands of goals by default, so perhaps we could go with say
128 subgoals by default and have some env var to override it, so on the
really highly parallel boxes you'd specify
make -j512 -k check GCC_TEST_PARALLEL_SLOTS=512
or similar.
Here is a patch I'm testing now:
--- gcc/Makefile.in.jj 2014-09-08 22:12:56.0 +0200
+++ gcc/Makefile.in 2014-09-11 16:06:36.641219430 +0200
@@ -513,34 +513,10 @@ xm_include_list=@xm_include_list@
xm_defines=@xm_defines@
lang_checks=
lang_checks_parallelized=
-dg_target_exps:=aarch64.exp,alpha.exp,arm.exp,avr.exp,bfin.exp,cris.exp
-dg_target_exps:=$(dg_target_exps),epiphany.exp,frv.exp,i386.exp,ia64.exp
-dg_target_exps:=$(dg_target_exps),m68k.exp,microblaze.exp,mips.exp,powerpc.exp
-dg_target_exps:=$(dg_target_exps),rx.exp,s390.exp,sh.exp,sparc.exp,spu.exp
-dg_target_exps:=$(dg_target_exps),tic6x.exp,xstormy16.exp
-# This lists a couple of test files that take most time during check-gcc.
-# When doing parallelized check-gcc, these can run in parallel with the
-# remaining tests. Each word in this variable stands for work for one
-# make goal and one extra make goal is added to handle all the *.exp
-# files not handled explicitly already. If multiple *.exp files
-# should be run in the same runtest invocation (usually if they aren't
-# very long running, but still should be split of from the check-parallel-$lang
-# remaining tests runtest invocation), they should be concatenated with commas.
-# Note that [a-zA-Z] wildcards need to have []s prefixed with \ (needed
-# by tcl) and as the *.exp arguments are mached both as is and with
-# */ prefixed to it in runtest_file_p, it is usually desirable to include
-# a subdirectory name.
-check_gcc_parallelize=execute.exp=execute/2* \
- execute.exp=execute/\[013-9a-fA-F\]* \
- execute.exp=execute/\[pP\]*,dg.exp \
-
execute.exp=execute/\[g-oq-zG-OQ-Z\]*,compile.exp=compile/2* \
- compile.exp=compile/\[9pP\]*,builtins.exp \
- compile.exp=compile/\[013-8a-oq-zA-OQ-Z\]* \
- dg-torture.exp,ieee.exp \
- vect.exp,unsorted.exp \
- guality.exp \
- struct-layout-1.exp,stackalign.exp \
- $(dg_target_exps)
+# Upper limit to which it is useful to parallelize this lang target.
+# It doesn't make sense to try e.g. 128 goals for small testsuites
+# like objc or go.
+check_gcc_parallelize=1
lang_opt_files=@lang_opt_files@ $(srcdir)/c-family/c.opt $(srcdir)/common.opt
lang_specs_files=@lang_specs_files@
lang_tree_files=@lang_tree_files@
@@ -3631,27 +3607,32 @@ $(filter-out $(lang_checks_parallelized)
export TCL_LIBRARY ; fi ; \
$(RUNTEST) --tool $* $(RUNTESTFLAGS))
-$(patsubst %,%-subtargets,$(filter-out
$(lang_checks_parallelized),$(lang_checks))): check-%-subtargets:
+$(patsubst %,%-subtargets,$(lang_checks)): check-%-subtargets:
@echo check-$*
check_p_tool=$(firstword $(subst _, ,$*))
-check_p_vars=$(check_$(check_p_tool)_parallelize)
+check_p_count=$(check_$(check_p_tool)_parallelize)
check_p_subno=$(word 2,$(subst _, ,$*))
-check_p_comma=,
-check_p_subwork=$(subst $(check_p_comma), ,$(if $(check_p_subno),$(word
$(check_p_subno),$(check_p_vars
-check_p_numbers=1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
+check_p_numbers0:=1 2 3 4 5 6 7 8 9
+check_p_numbers1:=0 $(check_p_numbers0)
+check_p_numbers2:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers1)))
+check_p_numbers3:=$(patsubst %,0%,$(check_p_numbers1)) $(check_p_numbers2)
+check_p_numbers4:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers3)))
+check_p_numbers5:=$(patsubst %,0%,$(check_p_numbers3)) $(check_p_numbers4)
+check_p_numbers6:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers5)))
+check_p_numbers:=$(check_p_numbers0) $(check_p_numbers2) $(check_p_numbers4)
$(check_p_numbers6)
check_p_subdir=$(subst _,,$*)
-check_p_subdirs=$(wordlist 1,$(words
$(check_$*_parallelize)),$(check_p_numbers))
+check_p_subdirs=$(wordlist 1,$(check_p_count),$(wordlist 1,$(or
$(GCC_TEST_PARALLEL_SLOTS),128),$(check_p_numbers)))
# For parallelized check-% targets, this decides whether parallelization
# is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
# but optional --target_board or --extra_opts arguments). If desirable,
# recursive make is run with check-parallel-$lang{,1,2,3,4,5} etc. goals,
Re: [PATCH] gcc parallel make check
Jakub == Jakub Jelinek ja...@redhat.com writes: Jakub I fear that is going to be too expensive, because e.g. all the Jakub caching that dejagnu and our tcl stuff does would be gone, all Jakub the tests for lp64 etc. would need to be repeated for each test. In gdb I arranged to have this stuff saved in a special cache directory. See gdb/testsuite/lib/cache.exp for the mechanism. Tom
RE: [PATCH] gcc parallel make check
Here is a patch I'm testing now:
Hi Jakub,
I also tested your patch to compare timings vs a newer patch (v8) I'll send soon
== patch v8 == make -j32 -k ==
check-fortran 4m58.178s
check-c++ ~10m
check-c ~10m
check 15m29.873s
== patch Jakub
check-c++ ~20m
check-fortran 3m31.237s
check-c 8m8
on the positive side, your patch provides a further speedup e.g. fortran and c
testing (where it splits things nicely). The libstdc++ bottleneck is not
solved, but I guess that is expected.
As you have presumably found as well, your patch introduces a number failures,
because some tests seem to have additional dependencies, either explicit or
implicit:
e.g. in gfortran.dg/binding_label_tests_10_main.f03
! { dg-do compile }
! This file must be compiled AFTER binding_label_tests_10.f03, which it
! should be because dejagnu will sort the files.
module binding_label_tests_10_main
in gfortran.dg/class_45b.f03
! { dg-do link }
! { dg-additional-sources class_45a.f03 }
This could clearly trigger as well in the current scheme of splitting, only we
have been lucky that dependencies seem to be 'well behaved' in having the same
initial letter in the filename.
Joost
Re: [PATCH] gcc parallel make check
On Thu, Sep 11, 2014 at 05:04:56PM +, VandeVondele Joost wrote: Here is a patch I'm testing now: I also tested your patch to compare timings vs a newer patch (v8) I'll send soon == patch v8 == make -j32 -k == check-fortran 4m58.178s check-c++ ~10m check-c ~10m check 15m29.873s == patch Jakub check-c++ ~20m check-fortran 3m31.237s check-c 8m8 on the positive side, your patch provides a further speedup e.g. fortran and c testing (where it splits things nicely). The libstdc++ bottleneck is not solved, but I guess that is expected. The same technique can be of course used for libstdc++, I just didn't want to do that until the -C gcc testing is changed. As you have presumably found as well, your patch introduces a number failures, because some tests seem to have additional dependencies, either explicit or implicit: I found more issues, in particular it seemed that struct-layout-1.exp, gnu-encoding.exp, plugin.exp and some go*.exp don't call runtest_file_p in the same amounts and same arguments in all invocations. And these Fortran inter-test dependencies, which Tobias told me is PR56408. Unfortunately my remote testing box is unreachable now and I'm still waiting for DDR4 modules to finish building better workstation, so can't test this right now. The patch below intends to serialize the content of the problematic *.exp tests (the first runtest to reach one of those will simply run all the tests from that *.exp file, others will skip it). For go I currently have no idea why does that happen, quick hack would be just disable parallelization of go temporarily and let Ian investigate. For PR56408 we need some fix. Jakub
Re: [PATCH] gcc parallel make check
On Thu, Sep 11, 2014 at 07:26:37PM +0200, Jakub Jelinek wrote:
right now. The patch below intends to serialize the content of the
problematic *.exp tests (the first runtest to reach one of those will simply
run all the tests from that *.exp file, others will skip it).
Forgotten patch below. BTW, something will probably need to be done about
acats too, either similar approach or just splitting the chapters into
little more jobs, because otherwise in make -C check -j48 acats dominated
the testing time for me.
--- gcc/Makefile.in.jj 2014-09-08 22:12:56.0 +0200
+++ gcc/Makefile.in 2014-09-11 16:58:01.076371437 +0200
@@ -513,34 +513,10 @@ xm_include_list=@xm_include_list@
xm_defines=@xm_defines@
lang_checks=
lang_checks_parallelized=
-dg_target_exps:=aarch64.exp,alpha.exp,arm.exp,avr.exp,bfin.exp,cris.exp
-dg_target_exps:=$(dg_target_exps),epiphany.exp,frv.exp,i386.exp,ia64.exp
-dg_target_exps:=$(dg_target_exps),m68k.exp,microblaze.exp,mips.exp,powerpc.exp
-dg_target_exps:=$(dg_target_exps),rx.exp,s390.exp,sh.exp,sparc.exp,spu.exp
-dg_target_exps:=$(dg_target_exps),tic6x.exp,xstormy16.exp
-# This lists a couple of test files that take most time during check-gcc.
-# When doing parallelized check-gcc, these can run in parallel with the
-# remaining tests. Each word in this variable stands for work for one
-# make goal and one extra make goal is added to handle all the *.exp
-# files not handled explicitly already. If multiple *.exp files
-# should be run in the same runtest invocation (usually if they aren't
-# very long running, but still should be split of from the check-parallel-$lang
-# remaining tests runtest invocation), they should be concatenated with commas.
-# Note that [a-zA-Z] wildcards need to have []s prefixed with \ (needed
-# by tcl) and as the *.exp arguments are mached both as is and with
-# */ prefixed to it in runtest_file_p, it is usually desirable to include
-# a subdirectory name.
-check_gcc_parallelize=execute.exp=execute/2* \
- execute.exp=execute/\[013-9a-fA-F\]* \
- execute.exp=execute/\[pP\]*,dg.exp \
-
execute.exp=execute/\[g-oq-zG-OQ-Z\]*,compile.exp=compile/2* \
- compile.exp=compile/\[9pP\]*,builtins.exp \
- compile.exp=compile/\[013-8a-oq-zA-OQ-Z\]* \
- dg-torture.exp,ieee.exp \
- vect.exp,unsorted.exp \
- guality.exp \
- struct-layout-1.exp,stackalign.exp \
- $(dg_target_exps)
+# Upper limit to which it is useful to parallelize this lang target.
+# It doesn't make sense to try e.g. 128 goals for small testsuites
+# like objc or go.
+check_gcc_parallelize=1
lang_opt_files=@lang_opt_files@ $(srcdir)/c-family/c.opt $(srcdir)/common.opt
lang_specs_files=@lang_specs_files@
lang_tree_files=@lang_tree_files@
@@ -3631,27 +3607,32 @@ $(filter-out $(lang_checks_parallelized)
export TCL_LIBRARY ; fi ; \
$(RUNTEST) --tool $* $(RUNTESTFLAGS))
-$(patsubst %,%-subtargets,$(filter-out
$(lang_checks_parallelized),$(lang_checks))): check-%-subtargets:
+$(patsubst %,%-subtargets,$(lang_checks)): check-%-subtargets:
@echo check-$*
check_p_tool=$(firstword $(subst _, ,$*))
-check_p_vars=$(check_$(check_p_tool)_parallelize)
+check_p_count=$(check_$(check_p_tool)_parallelize)
check_p_subno=$(word 2,$(subst _, ,$*))
-check_p_comma=,
-check_p_subwork=$(subst $(check_p_comma), ,$(if $(check_p_subno),$(word
$(check_p_subno),$(check_p_vars
-check_p_numbers=1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
+check_p_numbers0:=1 2 3 4 5 6 7 8 9
+check_p_numbers1:=0 $(check_p_numbers0)
+check_p_numbers2:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers1)))
+check_p_numbers3:=$(patsubst %,0%,$(check_p_numbers1)) $(check_p_numbers2)
+check_p_numbers4:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers3)))
+check_p_numbers5:=$(patsubst %,0%,$(check_p_numbers3)) $(check_p_numbers4)
+check_p_numbers6:=$(foreach i,$(check_p_numbers0),$(patsubst
%,$(i)%,$(check_p_numbers5)))
+check_p_numbers:=$(check_p_numbers0) $(check_p_numbers2) $(check_p_numbers4)
$(check_p_numbers6)
check_p_subdir=$(subst _,,$*)
-check_p_subdirs=$(wordlist 1,$(words
$(check_$*_parallelize)),$(check_p_numbers))
+check_p_subdirs=$(wordlist 1,$(check_p_count),$(wordlist 1,$(or
$(GCC_TEST_PARALLEL_SLOTS),128),$(check_p_numbers)))
# For parallelized check-% targets, this decides whether parallelization
# is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
# but optional --target_board or --extra_opts arguments). If desirable,
# recursive make is run with check-parallel-$lang{,1,2,3,4,5} etc. goals,
# which can be executed in parallel, as they are run in separate directories.
-# check-parallel-$lang{1,2,3,4,5} etc. goals invoke runtest with the longest
-# running *.exp files from the testsuite, as determined by
RE: [PATCH] gcc parallel make check
And these Fortran inter-test dependencies, which Tobias told me is
PR56408.
For PR56408 we need some fix.
BTW, is there anything special about Fortran ? There are at least 180 test
files that contain 'dg-additional-sources' some in a very non-local way:
./objc.dg/foreach-2.m: /* { dg-additional-sources
../objc-obj-c++-shared/nsconstantstring-class-impl.m } */
Joost
