On Tue, May 25, 2010 at 3:16 AM, Tim Ellison <t.p.elli...@gmail.com> wrote: > On 25/May/2010 01:17, Nathan Beyer wrote: >> For future milestone votes, I'd like to propose that we set the >> expectation of zero test failures. Every time we do a milestone and I >> run the build and tests I find myself looking at the test failures >> (there are ALWAYS failures) and ask 'are these expected'? If the >> failures are expected, they should be excluded and the tests should >> just pass. >> >> If this proposal is workable, immediately after this upcoming release, >> all 'expected' failures will be pulled into the exclusion lists. > > or, preferably the tests are fixed so they pass reliably. > > I see failures in AWT/Swing that are likely caused by me not leaving the > machine alone while I'm running the tests. I've seen other people post > results that I see when I've left my firewall enabled.
Yeah, I see those as well, but that's not what I'm referring to. There are a handful of AWT/Swing tests that have been failing on Windows for as long as I can remember. Not excluding them doesn't seem to be getting them fixed any faster than those that are excluded. > > So, yes, I agree that getting all the tests to pass is the goal; but not > simply by putting them into the exclude list without some further > investigation. > > Regards, > Tim >
