I'm not sure the iterative runs should be incorporated in CC for 2 reasons: 1. Those runs are time consuming. 2. ant is not acceptable to organize loops.
Thanks, Tanya On 11/24/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:
This is incredibly useful. Thanks. We need to incorporate this as a possible target in CC. geir tatyana doubtsova wrote: > I created lists of intermittent unit tests failures on Win XP in perTest > mode: > http://issues.apache.org/jira/browse/HARMONY-2282 - swing module > http://issues.apache.org/jira/browse/HARMONY-2284 - security module > > Could anybody take a look at it? > > I did not observe intermittent failures in other modules on Win XP so far. > I'm going to continue iterative classlib tests runs. > > Thanks, > Tanya > > On 11/22/06, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > >> Tatiana, >> That's great! Feel free to file JIRA issues about new problems. >> >> This is quite interesting that you haven't discovered >> org.apache.harmony.logging.tests.java.util.logging.SocketHandlerTest >> failure using effectively the same revision as I. This makes me think >> that my problems can be local to my computer. >> >> -- >> Thank you, >> Alexei >> >> On 11/21/06, tatyana doubtsova <[EMAIL PROTECTED]> wrote: >> > Hello, >> > >> > I've run classlib unit tests on Win XP in Other VM mode 50 times for >> each >> > module in order to reveal sporadic failures. >> > >> > >> > >> > The script is >> > >> > >> > >> > #!sh >> > >> > cat modulesList | while read moduleName; do >> > >> > A=0 >> > >> > while [ "$A" -lt "50" ]; do >> > >> > echo ITERATION $A >> > >> > ant - >> > >> Dtest.jre.home=c:/users/tdoubtso/harmony/drlvm/trunk/build/win_ia32_msvc_debug/deploy/jre\ >> >> > >> > -Dbuild.module=$moduleName - >> > Dhy.test.forkmode=perTest \ >> > >> > test 2>&1 | tee log_${moduleName}_${A}.txt >> > >> > cp -rf build/test_report >> build/test_report.${moduleName}_${A} >> > >> > A=`expr $A + 1` >> > >> > done >> > >> > done >> > >> > >> > >> > >> > >> > $ cat moduleList >> > >> > >> > >> > accessibility >> > >> > archive >> > >> > auth >> > >> > awt >> > >> > beans >> > >> > crypto >> > >> > instrument >> > >> > jndi >> > >> > lang-management >> > >> > logging >> > >> > luni >> > >> > math >> > >> > nio >> > >> > nio_char >> > >> > prefs >> > >> > print >> > >> > regex >> > >> > rmi >> > >> > security >> > >> > sound >> > >> > sql >> > >> > swing >> > >> > text >> > >> > tools >> > >> > x-net >> > >> > >> > >> > >> > >> > I've observed the following failures on svn=r476124 >> > >> > >> > >> > >> > >> > *Module* >> > >> > *Test* >> > >> > *Number of failures* >> > >> > *issue* >> > >> > luni >> > >> > >> org.apache.harmony.tests.internal.net.www.protocol.http.HttpURLConnectionTest >> >> > >> > 50 out of 50 >> > >> > 1879 <http://issues.apache.org/jira/browse/HARMONY-1879>, updated >> > >> > luni >> > >> > tests.api.java.net.InetAddressTest >> > >> > 50 out of 50 >> > >> > Test was eliminated from repository after Novenber,17 >> > >> > prefs >> > >> > org.apache.harmony.prefs.tests.java.util.prefs.PreferencesTest >> > >> > 50 out of 50 >> > >> > 1751 <http://issues.apache.org/jira/browse/HARMONY-1751> >> > >> > swing >> > >> > javax.swing.text.WrappedPlainView_SimpleTest >> > >> > 1 time out of 50 >> > >> > 1) >> > >> > swing >> > >> > javax.swing.SpinnerDateModelTest >> > >> > 35 times out of 50 >> > >> > 2) >> > >> > >> > >> > >> > 1). Eror >> > >> > Timeout occurred >> > >> > junit.framework.AssertionFailedError: Timeout occurred >> > >> > >> > >> > 2). Failure >> > >> > expected:<0> but was:<7> >> > >> > junit.framework.AssertionFailedError: expected:<0> but was:<7> at >> > javax.swing.SpinnerDateModelTest.testSpinnerDateModel( >> > SpinnerDateModelTest.java:59) at >> > java.lang.reflect.VMReflection.invokeMethod(Native >> > Method) at javax.swing.BasicSwingTestCase.runBareSuper( >> > BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase.runBareImpl( >> > BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase$1.run( >> > BasicSwingTestCase.java) at java.lang.Thread.run(Unknown Source) >> > >> > >> > >> > 2070 <http://issues.apache.org/jira/browse/HARMONY-2070> was not >> reproduced >> > during this test run >> > >> > >> > >> > I can't reproduce 1) and 2) today on svn = r477507 >> > >> > >> > >> > I've excluded failed tests and restarted multiple test run on Win XP in >> > order to make sure there is no other sporadic failures. >> > >> > Also I'm going to start the multiple classlib tests run on SUSE 9 >> > >> > >> > >> > >> > >> > To extract the results I've used commands: >> > >> > cd classlib/trunk/buid >> > >> > # find errors >> > >> > find . -name *Test.xml -exec grep -l "<testsuite errors=\"[1-9]*\" " {} >> \; > >> > ERRORS >> > >> > # find failures >> > >> > find . -name *Test.xml -exec grep -l "<testsuite errors=\"*\" >> > failures=\"[1-9]*\"" {} \; > FAILURES >> > >> > find . -type f -size 0 | grep *.xml > ZERO_LENGTH >> > >> > >> > >> > Also I've compared the number of files in test_report directories: >> > >> > #!sh >> > >> > cat modulesList | while read moduleName; do >> > >> > echo $moduleName >> > >> > A=1 >> > >> > while [ "$A" -lt "51" ]; do >> > >> > L=`ls -1 build/test_report.${moduleName}_${A} | grep xml >> | wc >> > -l` >> > >> > echo build/test_report.${moduleName}_${A} $L >> >> > test_report_length >> > >> > A=`expr $A + 1` >> > >> > done >> > done >> > >> > Thanks, >> > Tanya >> > >> > >> >
