Tatiana, I cannot stop myself from thinking about your wonderful scripts.
1. Looking at http://issues.apache.org/jira/browse/HARMONY-2282 I realized that in case of absence of DRLVM crash it would be quite useful to know if the bug is reproducible with J9. Failures on two different VMs convinces test authors to look in their tests once again to decide how valid they are. So if it wasn't difficult to launch your script for J9, this would be quite useful. 2. The second thing which comes into my mind is speeding up test runs by hacking build.xml files: perl -i -e 'undef $/; $_ = <>; s/(<batchtest.*?<\/batchtest>)/$1 x 50/es; print' modules/*/build.xml You can revert build files later using svn revert modules/*/build.xml. I wonder if we can set forkmode to vary a reliability load from running all tests in the same JVM to running each test separately. -- Thank you, Alexei On 11/23/06, tatyana doubtsova <[EMAIL PROTECTED]> wrote:
I created lists of intermittent unit tests failures on Win XP in perTest mode: http://issues.apache.org/jira/browse/HARMONY-2282 - swing module http://issues.apache.org/jira/browse/HARMONY-2284 - security module Could anybody take a look at it? I did not observe intermittent failures in other modules on Win XP so far. I'm going to continue iterative classlib tests runs. Thanks, Tanya On 11/22/06, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > Tatiana, > That's great! Feel free to file JIRA issues about new problems. > > This is quite interesting that you haven't discovered > org.apache.harmony.logging.tests.java.util.logging.SocketHandlerTest > failure using effectively the same revision as I. This makes me think > that my problems can be local to my computer. > > -- > Thank you, > Alexei > > On 11/21/06, tatyana doubtsova <[EMAIL PROTECTED]> wrote: > > Hello, > > > > I've run classlib unit tests on Win XP in Other VM mode 50 times for > each > > module in order to reveal sporadic failures. > > > > > > > > The script is > > > > > > > > #!sh > > > > cat modulesList | while read moduleName; do > > > > A=0 > > > > while [ "$A" -lt "50" ]; do > > > > echo ITERATION $A > > > > ant - > > > Dtest.jre.home=c:/users/tdoubtso/harmony/drlvm/trunk/build/win_ia32_msvc_debug/deploy/jre\ > > > > -Dbuild.module=$moduleName - > > Dhy.test.forkmode=perTest \ > > > > test 2>&1 | tee log_${moduleName}_${A}.txt > > > > cp -rf build/test_report build/test_report.${moduleName}_${A} > > > > A=`expr $A + 1` > > > > done > > > > done > > > > > > > > > > > > $ cat moduleList > > > > > > > > accessibility > > > > archive > > > > auth > > > > awt > > > > beans > > > > crypto > > > > instrument > > > > jndi > > > > lang-management > > > > logging > > > > luni > > > > math > > > > nio > > > > nio_char > > > > prefs > > > > print > > > > regex > > > > rmi > > > > security > > > > sound > > > > sql > > > > swing > > > > text > > > > tools > > > > x-net > > > > > > > > > > > > I've observed the following failures on svn=r476124 > > > > > > > > > > > > *Module* > > > > *Test* > > > > *Number of failures* > > > > *issue* > > > > luni > > > > > org.apache.harmony.tests.internal.net.www.protocol.http.HttpURLConnectionTest > > > > 50 out of 50 > > > > 1879 <http://issues.apache.org/jira/browse/HARMONY-1879>, updated > > > > luni > > > > tests.api.java.net.InetAddressTest > > > > 50 out of 50 > > > > Test was eliminated from repository after Novenber,17 > > > > prefs > > > > org.apache.harmony.prefs.tests.java.util.prefs.PreferencesTest > > > > 50 out of 50 > > > > 1751 <http://issues.apache.org/jira/browse/HARMONY-1751> > > > > swing > > > > javax.swing.text.WrappedPlainView_SimpleTest > > > > 1 time out of 50 > > > > 1) > > > > swing > > > > javax.swing.SpinnerDateModelTest > > > > 35 times out of 50 > > > > 2) > > > > > > > > > > 1). Eror > > > > Timeout occurred > > > > junit.framework.AssertionFailedError: Timeout occurred > > > > > > > > 2). Failure > > > > expected:<0> but was:<7> > > > > junit.framework.AssertionFailedError: expected:<0> but was:<7> at > > javax.swing.SpinnerDateModelTest.testSpinnerDateModel( > > SpinnerDateModelTest.java:59) at > > java.lang.reflect.VMReflection.invokeMethod(Native > > Method) at javax.swing.BasicSwingTestCase.runBareSuper( > > BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase.runBareImpl( > > BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase$1.run( > > BasicSwingTestCase.java) at java.lang.Thread.run(Unknown Source) > > > > > > > > 2070 <http://issues.apache.org/jira/browse/HARMONY-2070> was not > reproduced > > during this test run > > > > > > > > I can't reproduce 1) and 2) today on svn = r477507 > > > > > > > > I've excluded failed tests and restarted multiple test run on Win XP in > > order to make sure there is no other sporadic failures. > > > > Also I'm going to start the multiple classlib tests run on SUSE 9 > > > > > > > > > > > > To extract the results I've used commands: > > > > cd classlib/trunk/buid > > > > # find errors > > > > find . -name *Test.xml -exec grep -l "<testsuite errors=\"[1-9]*\" " {} > \; > > > ERRORS > > > > # find failures > > > > find . -name *Test.xml -exec grep -l "<testsuite errors=\"*\" > > failures=\"[1-9]*\"" {} \; > FAILURES > > > > find . -type f -size 0 | grep *.xml > ZERO_LENGTH > > > > > > > > Also I've compared the number of files in test_report directories: > > > > #!sh > > > > cat modulesList | while read moduleName; do > > > > echo $moduleName > > > > A=1 > > > > while [ "$A" -lt "51" ]; do > > > > L=`ls -1 build/test_report.${moduleName}_${A} | grep xml | wc > > -l` > > > > echo build/test_report.${moduleName}_${A} $L >> > > test_report_length > > > > A=`expr $A + 1` > > > > done > > done > > > > Thanks, > > Tanya > > > > >
