Also, to add to the idea of creating a JIRA for each test failure, I think this should be done only
when the test fails even after a clean new run on the respective platform. This can avoid unwanted
JIRA issues due to issues in test environment.
My 2 cents.
-Rajesh.
On 1/19/06, Kathey Marsden <[EMAIL PROTECTED]
> wrote:
Daniel John Debrunner wrote:
>>In order to increase awareness, I am proposing that an email is sent to
>>derby-dev after each tinderbox and nightly test run sending out the test
>>results.
>>
>>
>
>
>
The thing about such mails is:
1) They do not have enough historical context.
2) People in general get used to ignoring what they see everyday, a mail
that says "nightly test report"
I like Mike's idea of using Jira and it would conform to our distributed
model and get these issues the attention the deserve.
1) Add component "regression test failure"
2) Whoever is running/checking results will either
1) File a new issue for new failures. Put the name of the test in
the summary.
Include environment information, diff in the bug and put in
component:
"regression test failure". Any probable suspect submissions can
be pointed out.
2) Add a comment to old issues that the test failed that day.
This will get the attention of those who need to really look at the
issue and provide clear tracking of when
the issue first started, how often intermittent issues occur and when
the issue is resolved. Also folks can
run a quick query to see if their failure is a known issue.
Kathey
