On 05/31/10 10:37, Rene Engelhard wrote:
On Mon, May 31, 2010 at 10:24:17AM +0200, Stephan Bergmann wrote:
With CWS sb120 integrated in DEV300_m80, the framework and tests will
hopefully be reliable enough for actual use, see
[...]
the bots (see the second link above), and there might still be sporadic
failures (see
<http://wiki.services.openoffice.org/w/index.php?title=Test_Cleanup#unoapi_Tests_2>),
potentially causing buildbot builds to go red. I leave it up to Gregor
So it's not reliable enough.
to disable any test steps again that turn out to cause trouble; please
inform him about problems you encounter. (Due to vacation schedules, we
probably won't be able to track down those X11 problems for the next two
weeks.)
How should people getting "accused" of breaking stuff then handle red
tinderboxes where the red status is caused by this?
Nobody gets accused. Erroneous red statuses, while they admittedly
suck, are not too uncommon, for various reasons. People know how to
handle them (by looking at the logs, finding out what caused the
breakage, and taking a note in the CWS EIS data in case the cause is
external to their CWS).
I am all for doing everything to reduce false positives to as low a
level as practically possible, and I am especially determined to do so
for the parts "I own." However, we cannot improve subsequenttests
without trying it out, in the wild. We need to balance the value we get
out of these tests against the annoyances that the false positives cause.
Timing of CWS sb120 hitting the master and me going on two weeks of
vacation might be a little unfortunate. That's why I put it into
Gregor's hands to get that balancing right for now. But be assured that
I will evaluate the usefulness of subsequenttests as soon as I return,
based on any data that has accumulated by then.
-Stephan
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
