Kelly O'Hair wrote:
On Mar 29, 2010, at 7:04 PM, Martin Buchholz wrote:
On Mon, Mar 29, 2010 at 17:14, Kelly O'Hair <[email protected]> wrote:
Jonathan is working on some changes to jtreg that may allow many of the
tests on the ProblemList to be removed, I am hopeful anyway.
Once we have that new jtreg available, trimming this list down is next.
I admit to being a little quick to add some tests to the list, it was a
somewhat
frustrating experience to isolate these things when looking at all 12
platforms/vms.
I think in this case it was timing out on Solaris sparc and appeared
to be a
stress test.
This test is already marked othervm, so I can only assume that it is
very
close
to the default timeout,
If increasing the timeout will make failures go away, then that is
perfectly fine.
You can reasonably increase the timeout up to 1 hour.
I'm not sure having this one test take an hour is fair to the other
1,000's of tests.
It it can't run in under 5min, I'd question the test, maybe it's
sleeping or blocking
too much if that is the case.
Part of the goal in all this was to provide wide solid test coverage
in a reasonable amount
of time, but if we allow many tests to run for an hour, that's a bit
of a problem.
Regardless, I don't think this test takes an hour, I'll put a 10min
timeout on it
and see how that goes. But I suspect it will run in well under 5min.
-kto
Martin
I think it is worth noting that just because a test is on the
ProblemList does not mean it does not deserve to be run. It just means
it needs more TLC than other tests, for one of any number of reasons.
Since we have (regrettably) historically had trouble running all the
tests and getting all them to pass, the approach is to separate the
set of tests into two groups -- those that are well behaved and run in
reasonable time with reasonable resources, and reliably pass on a good
JDK -- and the rest. The well behaved tests are good candidates for
automated running on systems like JPRT or Hudson, and failures should be
indicative of a fault in what is being tested. The other tests should
still be run, but by their nature, they are harder to run, and may
require manual checking of the results to see if failures are real or
spurious. The real problem, in all of this, is the dwindling resources
to expend on such TLC :-(
-- Jon
