Frank Schoenheit, Sun Microsystems Germany wrote: > Hi Mathias, > >> Really, this part of the work is superfluous. IMHO tests that are known >> to be broken in a particular milestone should be skipped in a CWS based >> on it also. > > Don't think this is a good idea, since a test can be broken in different > ways. For instance, if your test checks 10 aspects, and one of them is > broken in MWS, you still want to know if the other 9 are okay in your > CWS. Otherwise, you'll notice a breakage in those 9 only when the one > failure is fixed, and the whole test re-enabled in MWS.
Then the granularity of test is wrong. As we have limited resources, we must not waste our time. It's better to skip a particular test (hopefully in a granularity so that not too many "aspects" are dropped) for some milestones and fix the root cause for its breakage ASAP then wasting so much time as now. Of course fixing the breakage must get high priority. > An interesting read (IMO), somewhat related to the topic: > http://blog.ulf-wendel.de/?p=259 (sorry, German, but I know you speak it > pretty well :) > My favoite quote, which I'd strongly agree to: > > Deactivating a test, to reach "0 test failures", is wrong, since the > equality of a failure is also a sensible information. Just because someone blogs it doesn't mean that it is correct. There are a lot of blogs available. :-) Or in short words: I don't buy that statement. "Information" comes at a price and it must be taken into account if you don't have unlimited resources. We could keep the test if the status page clearly would show me "test run gave the same result as the master test". The waste in time would be smaller then (a broken test doesn't take a lot of time, the waste comes from manually checking the results as we have to do now). But I take every bet that everybody would do the same with the "sensible information" that the master has the same failure as the CWS: read it and forget it. Ciao, Mathias -- Mathias Bauer (mba) - Project Lead OpenOffice.org Writer OpenOffice.org Engineering at Sun: http://blogs.sun.com/GullFOSS Please don't reply to "nospamfor...@gmx.de". I use it for the OOo lists and only rarely read other mails sent to it. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
