https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80759
--- Comment #20 from ro at CeBiTec dot Uni-Bielefeld.DE <ro at CeBiTec dot Uni-Bielefeld.DE> --- > --- Comment #17 from Daniel Santos <daniel.santos at pobox dot com> --- > (In reply to Rainer Orth from comment #15) >> Created attachment 41404 [details] >> Switch ms-sysv to more regular dg functions > > You may be surprised to learn how many faulty assumptions you may have about > how the gcc test harness works and manages parallelization. I did try your > patch and it didn't work. I didn't go too deeply into trying to analyze the I didn't even bother to try a parallel run before getting basic functionality right! Should have mentioned that explicitly. > failures, but if you call dg-runtest, you are using gcc's hack-daptation of > parallelization. However, your patch doesn't remove *my* hack-daptation of > parallelization, so we end up with two different parallelization schemes that > step on each other's toes. > > Another problem with the already present parallelization is that it bunches > tests into groups of 10 per job which will perform very poorly for these > tests. > > (https://github.com/gcc-mirror/gcc/blob/master/gcc/testsuite/lib/gcc-defs.exp#L170). > I presume this is to reduce disk I/O and it makes sense from that standpoint > (I don't want to know what it would take to get a ramdisk/tmpfs in a > platform-neutral fashion.) My basic point still stands: running your ms-sysv tests sequentially takes just a few minutes even on an old and (by today's standards) slow CPU, so there's absolutely no point investing lots of effort and complexity to parallelize what already runs adequately fast sequentially! > However, I'm learning a little more about how the test harness works, and it > MAY be possible to call gcc_parallel_test_enable 0 at the start of ms-sysv.exp > and be able to use all of the built-in dg-runtest, et. al. functions! If I > can > get this to work (and not break something else in the process), then we may be > on to a pathway to clean up ms-sysv.exp a little bit -- that is except for a > few outstanding (possibly surmountable) issues: > > 1.) Can the default time-out of 5 minutes be changed? I need 20 minutes for > the slowest processors and a whole HOUR when full tests are enabled. Sure it can: for one there are dg-timeout (and preferably dg-timeout factor) per testcase. I still wonder why you'd need that, though: if all your tests together take no more than a few minutes, why would you need to increase the timeout at all? Which processor would this be that takes 20 minutes or even an hour to run the tests *and complete all other tests well within the five minute timeout*? In fact, every test that takes more than about a minute on a resonably current CPU is frowned upon because under parallel testing/load such tests tend to run into the timeout. If your tests regularly exceed the timeout, there's something wrong with them: you need to split the so individual tests complete within the minute just mentioned. If really necessary in a setup, it is possible to set board_info(unix,gcc,timeout) in a global site.exp file, e.g. to deal with really slow/ancient systems. This would be necessary without your test anyway. > 2.) The test description should include the generator flags and not just the > CFLAGS. Is that possible from dg-runtest, et. al.? I suppose it's always > possible to add them to CFLAGS with -DGEN_FLAGS="-p0-12" as a hack. That's a requirement actually: the summary lines for different runs of a test must differ so you can tell them apart if one of them fails. How this is done in the end is primarily a cosmetic issue, though. > I guess you can see why I said that I was "semi-content" to leave it like it > is. :) But I'm also glad to better understand how the test harness > parallelization works. Maybe it's possible to make a small modification to > dg-defs.exp to get it to divvy out a single test per job instead of 10. As I said: first get the sequential execution right and then, if really really necessary (and I want proof for that) look at parallelization. It may well be that the initial solution is to restrict the number and size of tests run by everyone. After all, this is just for a niche feature of a single architecture/OS and there's no point in everyone testing on x86 having to pay a massive penalty for that. Rainer
