On Mon, Apr 16, 2007 at 10:09:35PM +0200, Laurent GUERBY wrote: > On Mon, 2007-04-16 at 12:00 -0600, Tom Tromey wrote: > > I wonder whether there is a role for the gcc compile farm in this? > > For instance perhaps "someone" could keep a set of builds there and > > provide folks with a simple way to regression-test ... like a shell > > script that takes a .i file, ssh's to the farm, and does a reghunt... ? > > > > I think this would only be worthwhile if the farm has enough disk, and > > if regression hunting is a fairly common activity. > > > > Tom > > We're a bit "short" on the current CompileFarm machines, > we have 5x16GB + 4x32GB (and as shown below it tends to > be used, I have to ping users from time to time to get GB > back :). > > There is enough cpu power in the farm to build and check a version for > each commit (all languages including Ada) on up to two branches (I sent > a message a while ago about that) with a latency of about 8 hours IIRC. > > We might be able to store only part of the compiler, or if this > proves really useful, I could just add a storage unit to the > farm with cheap & large current generation disks (machines are > unfortunately SCSI based). > > As announced a few weeks ago, all official releases are already > installed on the CompileFarm (/n/b01/guerby/release/X.Y.Z/bin with X.Y.Z > in 3.4.6, 4.0.0-4, 4.1.0-2).
Regression hunts using saved binaries are useful for some cases but not all. They are useful, for example, for narrowing down the range to search for a front end bug. They are not useful for finding bugs for a specific target, other than the ones for which binaries are saved, or for finding wrong-code bugs, or bugs in functionality that is not enabled by default. Unless a compiler is saved for every revision, determining the patch at fault requires either time from an experienced person looking through ChangeLog entries or else an automated hunt for the smaller time period, but since the hunt uses a binary search, it doesn't take that much longer to search for a period of months than it does to search for patches applied in a single day. About one-fourth of the regression hunts that I've run have required some kind of manual intervention because the test wasn't set up correctly, a build failed, or something unexpected happened when the test was run with one of the compilers within the range. No matter how automated the hunts are, they always need at least a small amount of individual attention. A regression hunt setup requires a local copy of the GCC Subversion repository, which takes LOTS of space. If one of the CompileFarm systems already has a copy of the repository, or can get more disk space to handle one, then it would be great to have a regression hunt setup on it as well. The official releases of GCC installed on those machines would be very useful in narrowing down ranges (and for reporting information about regressions in general). It would also help to keep a mainline build from every few months. I've posted my current reghunt tools in separate mail; see http://gcc.gnu.org/ml/gcc/2007-04/msg00635.html. Janis
