On Wed, 3 Sep 2014, VandeVondele Joost wrote:
I've noticed that
make -j -k check-fortran
results in a serialized checking, while
make -j32 -k check-fortran
goes parallel. Somehow the explicit 'N' in -jN seems to be needed for the check
target, while the other targets seem to do just fine.
On Wed, Sep 03, 2014 at 09:15:51AM +, VandeVondele Joost wrote:
> I've noticed that
>
> make -j -k check-fortran
>
> results in a serialized checking, while
>
> make -j32 -k check-fortran
>
> goes parallel. Somehow the explicit 'N' in -jN seems to be needed for the
> check target, while th
> It is intentional. With -j it is essentially a fork bomb, just don't use it.
well, silently ignoring it for just this target did cost me a lot of time,
while an eventual fork bomb would have been dealt with much more quickly.
>> Somewhat related is there a rule of thumb on how is the gran
> What did you expect for -j alone? an error?
No, as is standard in gnu make, a new process for any target that can be
processed (i.e. unlimited).
>> ... check-fortran seems to be limited to about ~5 parallel targets ...
>
>Running the make with -j8 gives 7 directories gfortran[1-6]? in gcc/tes
On Wed, Sep 03, 2014 at 09:37:19AM +, VandeVondele Joost wrote:
> > It is intentional. With -j it is essentially a fork bomb, just don't use
> > it.
>
> well, silently ignoring it for just this target did cost me a lot of time,
> while an eventual fork bomb would have been dealt with much
Hi Joost,
VandeVondele Joost wrote:
> I've noticed that
> make -j -k check-fortran
> results in a serialized checking, while
> make -j32 -k check-fortran
> goes parallel.
I have to admit that I don't know why that's the case. However, I can answer
the next question, is presumably related to thi
> I have to admit that I don't know why that's the case.
Actually Marc answered that one (I had the wrong mail address for gcc@ so
repeat here):
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53155
> See: gcc/fortran/Make-lang.in, which has:
I'll have a look and do some testing what the gains/co
>> expect -- /usr/share/dejagnu/runtest.exp --tool gcc lto.exp weak.exp tls.exp
>> ipa.exp tree-ssa.exp debug.exp >dwarf2.exp fixed-point.exp vxworks.exp
>> cilk-plus.exp vmx.exp pch.exp simulate-thread.exp x86_64-costmodel-vect.exp
>> i386-costmodel-vect.exp spu-costmodel-vect.exp ppc-costmodel
On Wed, Sep 03, 2014 at 10:35:41AM +, VandeVondele Joost wrote:
> >> expect -- /usr/share/dejagnu/runtest.exp --tool gcc lto.exp weak.exp
> >> tls.exp ipa.exp tree-ssa.exp debug.exp >dwarf2.exp fixed-point.exp
> >> vxworks.exp cilk-plus.exp vmx.exp pch.exp simulate-thread.exp
> >> x86_64-co
> The splits are in the Makefiles, see check_gcc_parallelize
attached is a patch to improve the parallel performance of 'make -jXX -k
check-fortran'. For XX=16, this yields ~50% speedup, and even with XX=4 we
still have 15%, the measured slowdown at XX=1 (<2%) is in the noise of testing.
The pa
On Fri, Sep 05, 2014 at 02:26:45PM +, VandeVondele Joost wrote:
> Adding label: mu matching files:278
> Adding label: wlgo matching files:284
> Adding label: vhzPkqWx_-9876543210ZYXVUTSRQONMLKJIHGFEDCBAyj matching
> files:94
> patterns:
> dg.exp=gfortran.dg/p* \
> dg.exp=gfortran.dg/c* \
>
On Fri, Sep 05, 2014 at 02:48:01PM +, VandeVondele Joost wrote:
>
> > Please sort the letters (LC_ALL=C sort) and where consecutive, use ranges.
> > Thus \[0-9A-Zhjqvx-z\]* or so. What is - doing in your list? _-9 is quite
> > unexpected range.
>
> the '-' is a bug indeed. I added this t
>> > Please sort the letters (LC_ALL=C sort) and where consecutive, use ranges.
>> > Thus \[0-9A-Zhjqvx-z\]*
OK, works fine with the attached patch, and looks cleaner in Make-lang.in.
Now, with the proper email address for gcc-patches... I wonder how many time
I'll be punished for typos.
unmodi
Attached is an extended version of the patch, it brings a 100% improvement in
make -j32 -k check-gcc (down from 20min to <10min) by modification of
check_gcc_parallelize.
It includes one non-trivial part, namely a split of the target exps. They are
now all split using a common choice (based on
On 09/09/2014 10:51 AM, VandeVondele Joost wrote:
> Attached is an extended version of the patch,
> it brings a 100% improvement in make -j32 -k check-gcc
First of all, many thanks for working on this.
+# ls -1 | ../../../contrib/generate_tcl_patterns.sh 300
"dg.exp=gfortran.dg/"
How does t
On Tue, Sep 09, 2014 at 02:02:18PM +0400, Yury Gribov wrote:
> On 09/09/2014 10:51 AM, VandeVondele Joost wrote:
> > Attached is an extended version of the patch,
> > it brings a 100% improvement in make -j32 -k check-gcc
>
> First of all, many thanks for working on this.
>
> +# ls -1 | ../../.
> +# ls -1 | ../../../contrib/generate_tcl_patterns.sh 300
> "dg.exp=gfortran.dg/"
>
> How does this work with subdirectories? Can we replace ls with find?
The input to the script is general, you can use this to your advantage. For
example, I've been using:
ls -1 g++.*/* | cut -c5- | ../../.
> No. As I wrote earlier, splitting on filenames and test counts only is only
> very rough split, all the splits really need to be backed out by real timing
> data from popular targets.
I'm actually doing quite some testing trying to get a reasonable balance,
checking 'completed in' in all *.l
On Tue, Sep 09, 2014 at 10:57:09AM +, VandeVondele Joost wrote:
> > No. As I wrote earlier, splitting on filenames and test counts only is only
> > very rough split, all the splits really need to be backed out by real timing
> > data from popular targets.
>
> Furthermore, for parallel perf
> If you get whitespace right, one can provide multiple different wildcards to
> a single *.exp file, e.g.
> make check-gcc RUNTESTFLAGS="dg.exp='p[0-9A-Za-qs-z]* pr[9A-Za-z]*'" should
> cover all tests starting with p other than pr[0-8]*.c (where you could split
> say pr[0-2]* into another job, pr
On 09/09/2014 06:14 PM, VandeVondele Joost wrote:
I certainly don't want to claim that the patch I have now is perfect,
it is rather an incremental improvement on the current setup.
I'd second this. Writing patterns manually seems rather inefficient and
error-prone
(not undoable of course but
On Tue, Sep 09, 2014 at 06:27:10PM +0400, Yury Gribov wrote:
> On 09/09/2014 06:14 PM, VandeVondele Joost wrote:
> >I certainly don't want to claim that the patch I have now is perfect,
> >it is rather an incremental improvement on the current setup.
>
> I'd second this. Writing patterns manually
On 09/09/2014 06:33 PM, Jakub Jelinek wrote:
On Tue, Sep 09, 2014 at 06:27:10PM +0400, Yury Gribov wrote:
On 09/09/2014 06:14 PM, VandeVondele Joost wrote:
I certainly don't want to claim that the patch I have now is perfect,
it is rather an incremental improvement on the current setup.
I'd s
Now with gzipped figure.. why do these bounce ?
> But if there are jobs that just take 1s to complete, then clearly it doesn't
> make sense to split them off as separate job. I think we don't need 100%
> even split, but at least roughly is highly desirable.
Let me add some data, attached is a gr
Attached is a further revision of the patch, now dealing with check-c++.
Roughly 50% speedup here at '-j32' (18m vs 12m). For my setup
(--enable-languages=c,c++,fortran) I have now improved all targets called in
'make -j32 -k check'. The latter is now 30% faster (15m vs 20m). Note that
there ar
On Tue, Sep 09, 2014 at 03:14:08PM +, VandeVondele Joost wrote:
> Attached is a further revision of the patch, now dealing with check-c++.
> Roughly 50% speedup here at '-j32' (18m vs 12m). For my setup
> (--enable-languages=c,c++,fortran) I have now improved all targets called in
> 'make -
Thanks for testing.
The vect-args.c I explained earlier, and is indeed due to i386.exp hardcoding
those.
The libstdc++ double counts didn't appear in my testing, but I'll have a look.
Note that these patterns are handwritten, so error prone.
The long tests in libstdc++ come from (in timing o
> You mean enhancing the script to split across arbitrarily long prefixes?
> That would be great.
I've now a script that does something like that:
~/test$ find /data/vjoost/gnu/gcc_trunk/gcc/gcc/testsuite/gfortran.dg/
-maxdepth 1 -type f -printf "%f\n" | ./generate_patterns.py 500 foo
All 3947
Jakub,
> First of all, the -j2 testing shows more tests tested in gcc and libstdc++:
>
>-# of expected passes 10133
>+# of expected passes 10152
>
>+PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for excess errors)
>[...]
>
>Not sure where the bug is, could be e.g. in
> could it be that the pattern in normal1 should have been '[ab]*/ de*/
> [ep]*/*' ?
I've checked that this fixes the bug in the current trunk split. I.e. files are
stil tested, but now only once. Consider this change added to the previously
submitted patch.
On 11 September 2014 07:22, VandeVondele Joost wrote:
> Jakub,
>
>> First of all, the -j2 testing shows more tests tested in gcc and libstdc++:
>>
>>-# of expected passes 10133
>>+# of expected passes 10152
>>
>>+PASS: 23_containers/set/modifiers/erase/abi_tag.cc (test for exce
>> could it be that the pattern in normal1 should have been '[ab]*/ de*/
>> [ep]*/*' ?
>
>Yes, we are running these tests multiple times:
>
>PASS: 23_containers/map/modifiers/erase/abi_tag.cc (test for excess errors)
>PASS: 23_containers/multimap/modifiers/erase/abi_tag.cc (test for excess
>erro
On 11 September 2014 15:45, VandeVondele Joost
wrote:
>
>>> could it be that the pattern in normal1 should have been '[ab]*/ de*/
>>> [ep]*/*' ?
>>
>>Yes, we are running these tests multiple times:
>>
>>PASS: 23_containers/map/modifiers/erase/abi_tag.cc (test for excess errors)
>>PASS: 23_contai
33 matches
Mail list logo
