On 23 Apr 2014, at 19:55, Ben Coman <b...@openinworld.com> wrote: > Sven Van Caekenberghe wrote: >> And we're all GREEN, great work ! >> Let's keep it that way until release. >> Everything that breaks a test should now be reverted. >> >> > Thats great news. However while the CI monkey is great, there are still a > few tests (at least in Windows) that fail in a fresh image since: > * they are bypassed by the CI > * they are not run on a Windows CI slave > > I fixed a few similar failures but ran out of time to finish this. It might > be worthwhile some others doing a pass the old fashioned way via the > TestRunner GUI. Thats what users will end up seeing. It would be "real > nice" if TestRunner showed users ALL GREEN for the Released image. I've just > started a run with the latest image while I head off to bed. I'll report > more on the issue tracker tomorrow.
Agreed, but we know from experience that running the tests in a very controlled environment is the only way to have a common frame of reference. Yes, there are always a couple of guys that will download the release, run all tests and complain about a couple of failures. It would be super if we could avoid that, but I am pretty sure that will not happen (i.e. they will not succeed 100% for everyone). More eyes (especially on, the many different variants of, Windows) will always help, but the community is what it is. Sven
