Simon x 2,
I've just been chatting with Simon about this, and basically we're
willing to try a more rigorous testing regime for changes to the HEAD.
Yesterday Ian and I uncovered a problem with the "known buildable"
repositories idea: the build slave has a partial repository, so it can't
necessarily push to the buildable repository because it doesn't
necessarily have the patches it needs to push. We could solve this by
giving the builders full repositories, but it's yet more fiddling around
with the buildbot setup.
Oh, yes, good point.
Here's our draft game plan. There would be a minimal testing
requirement for patches to HEAD, consisting of
- a 2-stage bootstrap, probably with the "quick" settings, core libs only
- a 'make fast' in testsuite, with zero failures
in addition, if you make changes that might break something that isn't
tested by the above (multiple platforms, older versions of GHC,
profiling, unregisterised, GHCi, binary distributions, HPC etc.) then it
is your responsibility to make a reasonable attempt to test it. We'll
only hand out the pointy hats to people who fail to do minimal testing,
though.
Fair enough.
To make testing easier, we'll add an automated 'validate' script which
does the complete build:
- make distclean
- sh boot
- configure
- put an appropriate build.mk in place
- make
- make -C testsuite fast
and if it gets all the way through and reports zero failures, you're ok
to push.
It may be worthwhile to put at the end of that script an
echo reminding people that, if your changes break something
not covered by the minimal test, you should make a
reasonable attempt to test it. I am sure people *want* to
do the right thing, it's just easy to forget that there may
be problems beyond what minimal testing covers, especially
for new people.
The standard workflow would be to keep two trees: one for developing in,
and one for testing. You record patches in developing, and occasionally
pull into testing and run 'validate'. When it succeeds, you push from
testing. Of course you can have multiple developing trees and a single
testing tree. Your developing trees can lag behind HEAD, but the
testing tree should be up to date with HEAD before running validate.
Good plan!
Right now the fast testsuite doesn't have zero failures (it has about
20), we'll need to get that down to zero first. The fast testsuite
currently runs in about 8 minutes on a fast Linux machine; it was about
5 minutes when I first did it, so we probably need some tuning there. I
have no idea why the build is currently slower than it was (indeed I
wasn't aware that it was), so we should look into that too.
BTW, does the testsuite work with -j2 or similar? It's
something that very obviously would benefit from multiple cores.
Generally, it'd be great if the validate script would by
default or as an option use parallel builds.
Thanks for looking into this!
Manuel
_______________________________________________
Cvs-ghc mailing list
[email protected]
http://www.haskell.org/mailman/listinfo/cvs-ghc
