Thorkil Naur wrote:
With the tnaur PPC OSX buildbot slave back in business,
yay! thanks for getting it going again.
I have finally spent
the effort to figure out in detail what these reports actually tell me. As I
understand the buildbot-collator code, an element in the lists of unexpected
passes and failures gives the test name, the number of builders that report
the unexpected result, and the name of the "smallest" of these builders. And
"smallest" is defined in Data.hs as the most preferred builder for debugging
the problem according to suitable criteria. So, for example, the line
GMapAssoc 3 x86-64 Linux head
among the "Old unexpected test failures" tells me that 3 builders got this
unexpected failure and the smallest of these were "x86-64 Linux head".
I would suggest extending this scheme by listing, for each test, all the
builders, sorted, rather than just the smallest. Perhaps simply on a single
line, although more elaborate formatting could be imagined. But in such a way
that it was easy to see, for a particular test, which builders actually
reported the unexpected result.
It also doesn't tell you which "ways" failed for this test, so in practice you
probably have to go and look at the actual log. If you're interested in
failures for one particular build (yours), then this is the best place to look
in any case. e.g.
http://darcs.haskell.org/buildbot/head/tnaur%20PPC%20OSX%20head/builds/66/step-runtestsuite/2
It's pretty clear that we have some kind of problem with the "threaded1" way on
PPC/OSX.
FWIW, I think listing all the builds that failed a particular test would help as
long as it didn't overflow 80 columns, since then it would be a lot less readable.
Perhaps the easiest thing to do is attach the test summaries from each build?
Another thing we could do is distinguish between tests that are failing on all
the platform we have seen, and tests that have succeeded on some platforms and
failed on others. Unfortunately it's hard to know which tests were only *run*
on certain platforms, so this is a bit tricky.
Cheers,
Simon
_______________________________________________
Cvs-ghc mailing list
[email protected]
http://www.haskell.org/mailman/listinfo/cvs-ghc
