RE: [PATCH] RE: gcc parallel make check

2014-09-11 Thread VandeVondele Joost
Jakub, First of all, the -j2 testing shows more tests tested in gcc and libstdc++: -# of expected passes 10133 +# of expected passes 10152 +PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors) [...] Not sure where the bug is, could be e.g. in

RE: [PATCH] RE: gcc parallel make check

2014-09-11 Thread VandeVondele Joost
could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? I've checked that this fixes the bug in the current trunk split. I.e. files are stil tested, but now only once. Consider this change added to the previously submitted patch.

Re: [PATCH] RE: gcc parallel make check

2014-09-11 Thread Jonathan Wakely
On 11 September 2014 07:22, VandeVondele Joost wrote: Jakub, First of all, the -j2 testing shows more tests tested in gcc and libstdc++: -# of expected passes 10133 +# of expected passes 10152 +PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors)

RE: [PATCH] RE: gcc parallel make check

2014-09-11 Thread VandeVondele Joost
could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? Yes, we are running these tests multiple times: PASS: 23_containers/map/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multimap/modifiers/erase/abi_tag.cc (test for excess errors) PASS:

Re: [PATCH] RE: gcc parallel make check

2014-09-11 Thread Jonathan Wakely
On 11 September 2014 15:45, VandeVondele Joost joost.vandevond...@mat.ethz.ch wrote: could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? Yes, we are running these tests multiple times: PASS: 23_containers/map/modifiers/erase/abi_tag.cc (test for excess errors)

RE: [PATCH] RE: gcc parallel make check

2014-09-11 Thread VandeVondele Joost
Jakub, First of all, the -j2 testing shows more tests tested in gcc and libstdc++: -# of expected passes 10133 +# of expected passes 10152 +PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors) [...] Not sure where the bug is, could be e.g. in

RE: [PATCH] RE: gcc parallel make check

2014-09-11 Thread VandeVondele Joost
could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? I've checked that this fixes the bug in the current trunk split. I.e. files are stil tested, but now only once. Consider this change added to the previously submitted patch.

Re: [PATCH] RE: gcc parallel make check

2014-09-11 Thread Jonathan Wakely
On 11 September 2014 07:22, VandeVondele Joost wrote: Jakub, First of all, the -j2 testing shows more tests tested in gcc and libstdc++: -# of expected passes 10133 +# of expected passes 10152 +PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors)

RE: [PATCH] RE: gcc parallel make check

2014-09-11 Thread VandeVondele Joost
could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? Yes, we are running these tests multiple times: PASS: 23_containers/map/modifiers/erase/abi_tag.cc (test for excess errors) PASS: 23_containers/multimap/modifiers/erase/abi_tag.cc (test for excess errors) PASS:

Re: [PATCH] RE: gcc parallel make check

2014-09-11 Thread Jonathan Wakely
On 11 September 2014 15:45, VandeVondele Joost joost.vandevond...@mat.ethz.ch wrote: could it be that the pattern in normal1 should have been '[ab]*/ de*/ [ep]*/*' ? Yes, we are running these tests multiple times: PASS: 23_containers/map/modifiers/erase/abi_tag.cc (test for excess errors)

Re: [PATCH] RE: gcc parallel make check

2014-09-10 Thread Jakub Jelinek
On Tue, Sep 09, 2014 at 03:14:08PM +, VandeVondele Joost wrote: Attached is a further revision of the patch, now dealing with check-c++. Roughly 50% speedup here at '-j32' (18m vs 12m). For my setup (--enable-languages=c,c++,fortran) I have now improved all targets called in 'make -j32

RE: [PATCH] RE: gcc parallel make check

2014-09-10 Thread VandeVondele Joost
Thanks for testing. The vect-args.c I explained earlier, and is indeed due to i386.exp hardcoding those. The libstdc++ double counts didn't appear in my testing, but I'll have a look. Note that these patterns are handwritten, so error prone. The long tests in libstdc++ come from (in timing

RE: [PATCH] RE: gcc parallel make check

2014-09-10 Thread VandeVondele Joost
You mean enhancing the script to split across arbitrarily long prefixes? That would be great. I've now a script that does something like that: ~/test$ find /data/vjoost/gnu/gcc_trunk/gcc/gcc/testsuite/gfortran.dg/ -maxdepth 1 -type f -printf %f\n | ./generate_patterns.py 500 foo All 3947

Re: [PATCH] RE: gcc parallel make check

2014-09-10 Thread Jakub Jelinek
On Tue, Sep 09, 2014 at 03:14:08PM +, VandeVondele Joost wrote: Attached is a further revision of the patch, now dealing with check-c++. Roughly 50% speedup here at '-j32' (18m vs 12m). For my setup (--enable-languages=c,c++,fortran) I have now improved all targets called in 'make -j32

Re: [PATCH] RE: gcc parallel make check

2014-09-10 Thread Jakub Jelinek
On Wed, Sep 10, 2014 at 01:57:01PM +, VandeVondele Joost wrote: Thanks for testing. The vect-args.c I explained earlier, and is indeed due to i386.exp hardcoding those. IMHO the best fix for that is following, use the same predicate whether to run the vect-args.c tests or not as is

RE: [PATCH] RE: gcc parallel make check

2014-09-10 Thread VandeVondele Joost
You mean enhancing the script to split across arbitrarily long prefixes? That would be great. I've now a script that does something like that: ~/test$ find /data/vjoost/gnu/gcc_trunk/gcc/gcc/testsuite/gfortran.dg/ -maxdepth 1 -type f -printf %f\n | ./generate_patterns.py 500 foo All 3947

RE: [PATCH] RE: gcc parallel make check

2014-09-09 Thread VandeVondele Joost
Attached is an extended version of the patch, it brings a 100% improvement in make -j32 -k check-gcc (down from 20min to 10min) by modification of check_gcc_parallelize. It includes one non-trivial part, namely a split of the target exps. They are now all split using a common choice (based on

Re: [PATCH] RE: gcc parallel make check

2014-09-09 Thread Yury Gribov
On 09/09/2014 10:51 AM, VandeVondele Joost wrote: Attached is an extended version of the patch, it brings a 100% improvement in make -j32 -k check-gcc First of all, many thanks for working on this. +# ls -1 | ../../../contrib/generate_tcl_patterns.sh 300 dg.exp=gfortran.dg/ How does this

Re: [PATCH] RE: gcc parallel make check

2014-09-09 Thread Jakub Jelinek
On Tue, Sep 09, 2014 at 02:02:18PM +0400, Yury Gribov wrote: On 09/09/2014 10:51 AM, VandeVondele Joost wrote: Attached is an extended version of the patch, it brings a 100% improvement in make -j32 -k check-gcc First of all, many thanks for working on this. +# ls -1 |

RE: [PATCH] RE: gcc parallel make check

2014-09-09 Thread VandeVondele Joost
+# ls -1 | ../../../contrib/generate_tcl_patterns.sh 300 dg.exp=gfortran.dg/ How does this work with subdirectories? Can we replace ls with find? The input to the script is general, you can use this to your advantage. For example, I've been using: ls -1 g++.*/* | cut -c5- |

RE: [PATCH] RE: gcc parallel make check

2014-09-09 Thread VandeVondele Joost
No. As I wrote earlier, splitting on filenames and test counts only is only very rough split, all the splits really need to be backed out by real timing data from popular targets. I'm actually doing quite some testing trying to get a reasonable balance, checking 'completed in' in all

Re: [PATCH] RE: gcc parallel make check

2014-09-09 Thread Jakub Jelinek
On Tue, Sep 09, 2014 at 10:57:09AM +, VandeVondele Joost wrote: No. As I wrote earlier, splitting on filenames and test counts only is only very rough split, all the splits really need to be backed out by real timing data from popular targets. Furthermore, for parallel

RE: [PATCH] RE: gcc parallel make check

2014-09-09 Thread VandeVondele Joost
If you get whitespace right, one can provide multiple different wildcards to a single *.exp file, e.g. make check-gcc RUNTESTFLAGS=dg.exp='p[0-9A-Za-qs-z]* pr[9A-Za-z]*' should cover all tests starting with p other than pr[0-8]*.c (where you could split say pr[0-2]* into another job, pr[3-5]*

Re: [PATCH] RE: gcc parallel make check

2014-09-09 Thread Yury Gribov
On 09/09/2014 06:14 PM, VandeVondele Joost wrote: I certainly don't want to claim that the patch I have now is perfect, it is rather an incremental improvement on the current setup. I'd second this. Writing patterns manually seems rather inefficient and error-prone (not undoable of course but

Re: [PATCH] RE: gcc parallel make check

2014-09-09 Thread Jakub Jelinek
On Tue, Sep 09, 2014 at 06:27:10PM +0400, Yury Gribov wrote: On 09/09/2014 06:14 PM, VandeVondele Joost wrote: I certainly don't want to claim that the patch I have now is perfect, it is rather an incremental improvement on the current setup. I'd second this. Writing patterns manually seems

Re: [PATCH] RE: gcc parallel make check

2014-09-09 Thread Yury Gribov
On 09/09/2014 06:33 PM, Jakub Jelinek wrote: On Tue, Sep 09, 2014 at 06:27:10PM +0400, Yury Gribov wrote: On 09/09/2014 06:14 PM, VandeVondele Joost wrote: I certainly don't want to claim that the patch I have now is perfect, it is rather an incremental improvement on the current setup. I'd

RE: [PATCH] RE: gcc parallel make check

2014-09-09 Thread VandeVondele Joost
Now with gzipped figure.. why do these bounce ? But if there are jobs that just take 1s to complete, then clearly it doesn't make sense to split them off as separate job. I think we don't need 100% even split, but at least roughly is highly desirable. Let me add some data, attached is a

RE: [PATCH] RE: gcc parallel make check

2014-09-09 Thread VandeVondele Joost
Attached is a further revision of the patch, now dealing with check-c++. Roughly 50% speedup here at '-j32' (18m vs 12m). For my setup (--enable-languages=c,c++,fortran) I have now improved all targets called in 'make -j32 -k check'. The latter is now 30% faster (15m vs 20m). Note that there

RE: [PATCH] RE: gcc parallel make check

2014-09-09 Thread VandeVondele Joost
Attached is an extended version of the patch, it brings a 100% improvement in make -j32 -k check-gcc (down from 20min to 10min) by modification of check_gcc_parallelize. It includes one non-trivial part, namely a split of the target exps. They are now all split using a common choice (based on

Re: [PATCH] RE: gcc parallel make check

2014-09-09 Thread Yury Gribov
On 09/09/2014 10:51 AM, VandeVondele Joost wrote: Attached is an extended version of the patch, it brings a 100% improvement in make -j32 -k check-gcc First of all, many thanks for working on this. +# ls -1 | ../../../contrib/generate_tcl_patterns.sh 300 dg.exp=gfortran.dg/ How does this

Re: [PATCH] RE: gcc parallel make check

2014-09-09 Thread Jakub Jelinek
On Tue, Sep 09, 2014 at 02:02:18PM +0400, Yury Gribov wrote: On 09/09/2014 10:51 AM, VandeVondele Joost wrote: Attached is an extended version of the patch, it brings a 100% improvement in make -j32 -k check-gcc First of all, many thanks for working on this. +# ls -1 |

RE: [PATCH] RE: gcc parallel make check

2014-09-09 Thread VandeVondele Joost
+# ls -1 | ../../../contrib/generate_tcl_patterns.sh 300 dg.exp=gfortran.dg/ How does this work with subdirectories? Can we replace ls with find? The input to the script is general, you can use this to your advantage. For example, I've been using: ls -1 g++.*/* | cut -c5- |

RE: [PATCH] RE: gcc parallel make check

2014-09-09 Thread VandeVondele Joost
No. As I wrote earlier, splitting on filenames and test counts only is only very rough split, all the splits really need to be backed out by real timing data from popular targets. I'm actually doing quite some testing trying to get a reasonable balance, checking 'completed in' in all

Re: [PATCH] RE: gcc parallel make check

2014-09-09 Thread Jakub Jelinek
On Tue, Sep 09, 2014 at 10:57:09AM +, VandeVondele Joost wrote: No. As I wrote earlier, splitting on filenames and test counts only is only very rough split, all the splits really need to be backed out by real timing data from popular targets. Furthermore, for parallel

RE: [PATCH] RE: gcc parallel make check

2014-09-09 Thread VandeVondele Joost
If you get whitespace right, one can provide multiple different wildcards to a single *.exp file, e.g. make check-gcc RUNTESTFLAGS=dg.exp='p[0-9A-Za-qs-z]* pr[9A-Za-z]*' should cover all tests starting with p other than pr[0-8]*.c (where you could split say pr[0-2]* into another job, pr[3-5]*

Re: [PATCH] RE: gcc parallel make check

2014-09-09 Thread Yury Gribov
On 09/09/2014 06:14 PM, VandeVondele Joost wrote: I certainly don't want to claim that the patch I have now is perfect, it is rather an incremental improvement on the current setup. I'd second this. Writing patterns manually seems rather inefficient and error-prone (not undoable of course but

Re: [PATCH] RE: gcc parallel make check

2014-09-09 Thread Jakub Jelinek
On Tue, Sep 09, 2014 at 06:27:10PM +0400, Yury Gribov wrote: On 09/09/2014 06:14 PM, VandeVondele Joost wrote: I certainly don't want to claim that the patch I have now is perfect, it is rather an incremental improvement on the current setup. I'd second this. Writing patterns manually seems

Re: [PATCH] RE: gcc parallel make check

2014-09-09 Thread Yury Gribov
On 09/09/2014 06:33 PM, Jakub Jelinek wrote: On Tue, Sep 09, 2014 at 06:27:10PM +0400, Yury Gribov wrote: On 09/09/2014 06:14 PM, VandeVondele Joost wrote: I certainly don't want to claim that the patch I have now is perfect, it is rather an incremental improvement on the current setup. I'd

RE: [PATCH] RE: gcc parallel make check

2014-09-09 Thread VandeVondele Joost
Now with gzipped figure.. why do these bounce ? But if there are jobs that just take 1s to complete, then clearly it doesn't make sense to split them off as separate job. I think we don't need 100% even split, but at least roughly is highly desirable. Let me add some data, attached is a

RE: [PATCH] RE: gcc parallel make check

2014-09-09 Thread VandeVondele Joost
Attached is a further revision of the patch, now dealing with check-c++. Roughly 50% speedup here at '-j32' (18m vs 12m). For my setup (--enable-languages=c,c++,fortran) I have now improved all targets called in 'make -j32 -k check'. The latter is now 30% faster (15m vs 20m). Note that there

Re: [PATCH] RE: gcc parallel make check

2014-09-05 Thread Jakub Jelinek
On Fri, Sep 05, 2014 at 02:48:01PM +, VandeVondele Joost wrote: Please sort the letters (LC_ALL=C sort) and where consecutive, use ranges. Thus \[0-9A-Zhjqvx-z\]* or so. What is - doing in your list? _-9 is quite unexpected range. the '-' is a bug indeed. I added this to the

RE: [PATCH] RE: gcc parallel make check

2014-09-05 Thread VandeVondele Joost
Please sort the letters (LC_ALL=C sort) and where consecutive, use ranges. Thus \[0-9A-Zhjqvx-z\]* OK, works fine with the attached patch, and looks cleaner in Make-lang.in. Now, with the proper email address for gcc-patches... I wonder how many time I'll be punished for typos. unmodified

RE: [PATCH] RE: gcc parallel make check

2014-09-05 Thread VandeVondele Joost
Please sort the letters (LC_ALL=C sort) and where consecutive, use ranges. Thus \[0-9A-Zhjqvx-z\]* OK, works fine with the attached patch, and looks cleaner in Make-lang.in. Now, with the proper email address for gcc-patches... I wonder how many time I'll be punished for typos. unmodified