On Thu, 21 Sep 2017, Markus Trippelsdorf wrote: > And it has the basic problem of all automatic testing: that in the long > run everyone simply ignores it.
Hence, see my comments about the value of having someone who monitors the results and files bugs / notifies patch authors / fixes issues. It doesn't need to be the same person who runs the bot. It doesn't need to be the same person for every architecture. But the results need to be monitored and issues raised. That's what I do with my glibc bots. A lot of the time they run cleanly, but sometimes when they show failures it does take a significant amount of work to understand them and fix or inform appropriate people. (A lot of fixes were also involved in getting those bots to a near-clean baseline state.) I once did it, a long time ago, for some GCC bots (on HP-UX, and I think i686-pc-linux-gnu as well, as I recall), but that project ended and I stopped running the HP-UX bots and largely stopped monitoring the i686-pc-linux-gnu one. My experience indicates that GCC bots would require a lot more monitoring than glibc ones, especially if testing lots of unusual configurations. I think it would be straightforward to adapt build-many-glibcs.py to operate as a GCC bot running the compilation parts of the GCC testsuite for all or almost all supported architectures (and a bit more complicated to make it track regressions at the level of individual test failures), but I don't know how long an all-architectures compilation test run would take, and whether there would be people to monitor all-architectures test results is another matter. (build-many-glibcs.py is useful in glibc development even apart from the bots, to allow people to do some all-architectures testing of global changes.) > The same thing is true for the regression mailing list > https://gcc.gnu.org/ml/gcc-regression/current/. > It is obvious that nobody pays any attention to it, e.g. PGO bootstrap > is broken for several months on x86_64 and i686 bootstrap is broken for > a long time, too. I don't know if he currently monitors it, but HJ has certainly filed bugs for regressions found by his bots in the past. -- Joseph S. Myers jos...@codesourcery.com