On Thu, Sep 22, 2011 at 20:06, Hans-Peter Nilsson <h...@bitrange.com> wrote: > On Thu, 8 Sep 2011, Diego Novillo wrote: > >> On Thu, Sep 8, 2011 at 04:31, Richard Guenther >> <richard.guent...@gmail.com> wrote: >> >> > I think it would be more useful to have a script parse gcc-testresults@ >> > postings from the various autotesters and produce a nice webpage >> > with revisions and known FAIL/XPASSes for the target triplets that >> > are tested. >> >> Sure, though that describes a different tool. I'm after a tool that >> will 'exit 0' if the testsuite finished with nominal results. > > Not to stop you from (partly) reinventing the wheel, but that's > pretty much what contrib/regression/btest-gcc.sh already does, > though you have to feed it a baseline a set of processed .sum > files which could (for a calling script or a modified > btest-gcc.sh) live in, say, contrib/target-results/<target>. > It handles "duplicate" test names by marking it as failing if > any of them has failed. Works good enough.
Yeah, I actually considered using it by extracting the actual .sum file processing out of it (I was not interested in it running the build nor the tests). However, I also needed to add support for marking flaky tests and putting an expiration date on failures. Additionally, I needed versioned failure manifests, and I could not justify storing in SVN multiple directories with 12Mb worth of .sum files in them. The small manifest file also has the local advantage of serving as release documentation for what we expect to fail and why. Diego.
