Thanks Alex for the details, so, from what you say, it should probably be more SWF timeout than step timeout which would explained the difference in the number of tests ran.
My server point the only one mustella/tests folder I have on this machine and I can see the list on http://locahost Unfortunately I didn't send the logs to a file, will re-launch the failures tonight and check the logs. -Fred -----Message d'origine----- De : Alex Harui [mailto:aha...@adobe.com] Envoyé : lundi 9 septembre 2013 17:43 À : dev@flex.apache.org Objet : Re: Mustella question On 9/9/13 6:01 AM, "Frédéric THOMAS" <webdoubl...@hotmail.com> wrote: >> "./mini_run.sh -all" will run all the main Mustella tests (close to >40000) tests, taking about 8 - 10 hours. > >I ran "-all" on purpose and it gave me about 20500 tests passes and 20 >failed included 17 from the Marshall plan > >> If any of these tests fail, those tests will be written to >>'failures.txt'. >If you want to re-run these failed tests, you simply run "./mini_run.sh >-failures". No additional arguments needed. >Mustella will read 'failures.txt' and run only the tests specified in >that file. This should be much sooner ;-) > >I used the "-rerun" to avoid compiling again as I didn't change any >test code so, I just wanted to rerun the failed tests to make the >timeout failed tests passed. There are two common kinds of timeouts: 1) Step timeout: A test step did not receive the waitEvent in time (usually 1 second) 2) SWF timeout: The test engine hasn't seen any output from the SWF in time (IIRC, usually 12 seconds) The output in the console is different. For a step timeout, the output will tell you which test step failed and the next test will run. For a SWF timeout the output will just saw that some SWF timed out. When that happens all subsequent tests in that SWF are not run and your totals will be lower. > >My problems are: >1- Why the MarshallPlan tests don't pass ? Marshall plan tests require a localhost or localhost:8080 server pointed at the mustella tests folder so we can load child SWFs into a sandbox. >2- Why re-running faillures gives me more failed tests than the >previous call to the same command ? If you had SWF timeouts you can get a different/higher number of tests. > >-Fred >
