Weldon Washburn wrote:
Folks,
I have spent the last two months committing patches to the VM. While we
have added a ton of much needed functionality, the stability of the system
has been ignored. By chance, I looked at thread synchronization design
problems this week. Its very apparent that we lack the regression testing
to really find threading bugs, test the fixes and test against regression.
No doubt there are similar problems in other VM subsystems. "build test"
is necessary but not sufficient for where we need to go. In a sense,
committing code with only "build test" to prevent regression is the
equivalent to flying in the fog without instrumentation.
So that we can get engineers focused on stability, I am thinking of coding
the JIRAs that involve new features as "later" or even "won't fix". Please
feel free to comment.
We also need to restart the old email threads on regression tests. For
example, we need some sort of automated test script that runs Eclipse and
tomcat, etc. in a deterministic fashion so that we can compare test
results. It does not have to be perfect for starts, just repeatable and
easy to use. Feel free to beat me to starting these threads :)
In my experience on working with drlvm, stability problems are often
discovered on the existing VM acceptance tests. Big applications like
eclipse or tomcat with long workloads usually reveal problems like lack
of class unloading unless they crash on something like threading
problems. The acceptance VM tests that we have already are a good start
to test stability if they are ran nonstop many times.
I don't say that we shouldn't have real application workloads. I just
want to say that acceptance tests already usually reveal threading
problems quite well if they are ran many times, and race conditions
happen in some circumstances.
However at the moment we already have failing tests, some of them like
gc.LOS on WinXP don't need multiple times to make them fail. There's
also java.lang.ThreadTest which fails for me on Windows 2003 server SP1
and now started to fail on Linux as well.
--
Gregory
