On 8/29/2010 8:27 AM, Jonathan Costers wrote:
seems my previous email got truncated :)
in any case, the list of tests would be the same than if you would run the
following (all) categories:
...
If anyone feels adventurous, please go ahead :-)
I will try and let this run for a while and see what happens.
I do know that some tests used to hang on me in the past, so im not
expecting this run to complete normally.
...
Maybe just run it until you have enough failures to keep us busy for a
while?
I think we may have a lot of test failures in our near future, so maybe
we should think about a process for managing and resolving them. Here
are my ideas, but remember my inexperience with River and Apache
development process, and modify as needed:
1. Every test failure gets a JIRA. A group of similar tests can share a
single JIRA until they are found to have different causes. E-mails
discussing a test failure should refer to the JIRA item.
I volunteer to write up the ones that have been discussed recently in
the mailing list. Once that is done, I suggest whoever finds the test
failure write it up.
2. Examine each test to see if it needs a resource or configuration that
is not available to us. If so, edit the .td to skip the test, but keep
the JIRA open until we get the appropriate configuration, or modify the
test to not need it, or decide the test is unnecessary and can be
permanently skipped.
Ideally, we should pick a fixed keyword for each missing resource, and
use the keywords in .td comments, so that we can easily find all tests
that are being skipped for a given reason.
3. Find when the test stopped working. The presumption is that every
test has worked at some time in the past. For recent regressions, narrow
it down to the SVN revision level. For older regressions, the release
level may be sufficient.
4. Analyze and fix the problem.
I see that the JIRA records have an "assigned" field. Who can set that
field? It may be useful to avoid tripping over each other to note who is
working on a given test failure.
I can file JIRA items, and narrow regressions down by binary search. I
don't know enough about the resources to determine which tests need ones
we don't have. At least initially, I'll be slow debugging just because
I'll be learning my way around River as I go.
Does this make any sense? Improvements?
Patricia
