I am not suggesting regular emails. But I also doubt that developers,
who are all quite busy, are going to remember to regularly check the
results.
What I'd like to see is somebody who volunteers to monitor the page and
send an email out if things are getting out of hand. I would leave it
to the judgement of the volunteer to identify what "out of hand" means.
Personally, if more than three to five tests are failing in derbyall
then I think we have a problem. Or if we are seeing a general increase
in failures rather than a decrease.
David
Ole Solberg wrote:
David W. Van Couvering wrote:
I got some test failures in derbynetclient mats, so I checked the
tinderbox. We have quite a number of failures on the latest revision
that ran tests, 370061, running on Solaris 10 x86:
http://www.multinet.no/~solberg/public/Apache/TinderBox_Derby/Limited/testSummary-370061.html
derbylang: 1 failure
derbytools: 1 failure
encryptionBlowfish: 1 failure
i18nTest: 1 failure
jdbcapi: 1 failure
derbynetclientmats: 3 failures
derbynetmats: 2 failures
encryptionAll: 2 failures
encryptoin: 1 failure
derbyall: 12 failures
This seems a bit much; it's hard for a developer to know if their own
changes are valid as you have to sift through all the existing
failures. The last Solaris 10 x86 regression test run on revision
369861 had only 2 failures.
Looking at the derbyall history, at least on XP it seems to be getting
worse and worse (from 7 failures on 1/6 to 12 failures on 1/17). On
Solaris 10 it's gone from 1 failure to 5 failures, while on Linux it's
stayed steady.
Can those of us who have checked in/contributed patches lately please
look at the failures and see if you recognize what might be causing them?
Also: is anybody keeping an eye on these and raising a flag if the
tests start failing? I thought we had a pretty strict rule that we
should have 100% pass on derbyall.
Are you suggesting that we should send an e-mail on derby-dev on the
status of the tests? Wouldn't that just drown in the existing volume?
Since these test results are available for anyone to see at
http://www.multinet.no/~solberg/public/Apache/
I was thinking of the 3-three sub-pages as the *flags* !
I was hoping that patch submitters and committers would use these pages
to check results on other platforms than what they do test on themselves.
And, of course, if the results on
http://www.multinet.no/~solberg/public/Apache/ do not match what other
people are getting on the same platforms please tell me, because then I
probably have a problem with my test environment!
Thanks,
David
begin:vcard
fn:David W Van Couvering
n:Van Couvering;David W
org:Sun Microsystems, Inc.;Database Technology Group
email;internet:[EMAIL PROTECTED]
title:Senior Staff Software Engineer
tel;work:510-550-6819
tel;cell:510-684-7281
x-mozilla-html:TRUE
version:2.1
end:vcard
