Hello,
I've run classlib unit tests on Win XP in Other VM mode 50 times for each
module in order to reveal sporadic failures.
The script is
#!sh
cat modulesList | while read moduleName; do
A=0
while [ "$A" -lt "50" ]; do
echo ITERATION $A
ant -
Dtest.jre.home=c:/users/tdoubtso/harmony/drlvm/trunk/build/win_ia32_msvc_debug/deploy/jre\
-Dbuild.module=$moduleName -
Dhy.test.forkmode=perTest \
test 2>&1 | tee log_${moduleName}_${A}.txt
cp -rf build/test_report build/test_report.${moduleName}_${A}
A=`expr $A + 1`
done
done
$ cat moduleList
accessibility
archive
auth
awt
beans
crypto
instrument
jndi
lang-management
logging
luni
math
nio
nio_char
prefs
print
regex
rmi
security
sound
sql
swing
text
tools
x-net
I've observed the following failures on svn=r476124
*Module*
*Test*
*Number of failures*
*issue*
luni
org.apache.harmony.tests.internal.net.www.protocol.http.HttpURLConnectionTest
50 out of 50
1879 <http://issues.apache.org/jira/browse/HARMONY-1879>, updated
luni
tests.api.java.net.InetAddressTest
50 out of 50
Test was eliminated from repository after Novenber,17
prefs
org.apache.harmony.prefs.tests.java.util.prefs.PreferencesTest
50 out of 50
1751 <http://issues.apache.org/jira/browse/HARMONY-1751>
swing
javax.swing.text.WrappedPlainView_SimpleTest
1 time out of 50
1)
swing
javax.swing.SpinnerDateModelTest
35 times out of 50
2)
1). Eror
Timeout occurred
junit.framework.AssertionFailedError: Timeout occurred
2). Failure
expected:<0> but was:<7>
junit.framework.AssertionFailedError: expected:<0> but was:<7> at
javax.swing.SpinnerDateModelTest.testSpinnerDateModel(
SpinnerDateModelTest.java:59) at
java.lang.reflect.VMReflection.invokeMethod(Native
Method) at javax.swing.BasicSwingTestCase.runBareSuper(
BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase.runBareImpl(
BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase$1.run(
BasicSwingTestCase.java) at java.lang.Thread.run(Unknown Source)
2070 <http://issues.apache.org/jira/browse/HARMONY-2070> was not reproduced
during this test run
I can't reproduce 1) and 2) today on svn = r477507
I've excluded failed tests and restarted multiple test run on Win XP in
order to make sure there is no other sporadic failures.
Also I'm going to start the multiple classlib tests run on SUSE 9
To extract the results I've used commands:
cd classlib/trunk/buid
# find errors
find . -name *Test.xml -exec grep -l "<testsuite errors=\"[1-9]*\" " {} \; >
ERRORS
# find failures
find . -name *Test.xml -exec grep -l "<testsuite errors=\"*\"
failures=\"[1-9]*\"" {} \; > FAILURES
find . -type f -size 0 | grep *.xml > ZERO_LENGTH
Also I've compared the number of files in test_report directories:
#!sh
cat modulesList | while read moduleName; do
echo $moduleName
A=1
while [ "$A" -lt "51" ]; do
L=`ls -1 build/test_report.${moduleName}_${A} | grep xml | wc
-l`
echo build/test_report.${moduleName}_${A} $L >>
test_report_length
A=`expr $A + 1`
done
done
Thanks,
Tanya
