Hmm ... thought it would be a simple fix such as changing
boolean failFast = systemPropertyAsBoolean(SYSPROP_FAILFAST, false);
to
boolean failFast = systemPropertyAsBoolean(SYSPROP_FAILFAST, true);
But after I do that, tests like TestSystemPropertiesInvariantRule (and few
others) fail, on some unexpected failures.
I think that in general the code that initializes ignoreAfterMaxFailures
should take tests.iters into account?
I.e. what's the point of running a test with maxfailures=5 or failfast=true
without iters?
After I do this, all tests are happy:
Index:
lucene/test-framework/src/java/org/apache/lucene/util/LuceneTestCase.java
===================================================================
---
lucene/test-framework/src/java/org/apache/lucene/util/LuceneTestCase.java
(revision 1481311)
+++
lucene/test-framework/src/java/org/apache/lucene/util/LuceneTestCase.java
(working copy)
@@ -358,12 +358,13 @@
/**
* Ignore tests after hitting a designated number of initial failures.
*/
- final static TestRuleIgnoreAfterMaxFailures ignoreAfterMaxFailures;
+ final static TestRuleIgnoreAfterMaxFailures ignoreAfterMaxFailures;
static {
int maxFailures = systemPropertyAsInt(SYSPROP_MAXFAILURES,
Integer.MAX_VALUE);
- boolean failFast = systemPropertyAsBoolean(SYSPROP_FAILFAST, false);
-
- if (failFast) {
+ boolean failFast = systemPropertyAsBoolean(SYSPROP_FAILFAST, true);
+ int iters = systemPropertyAsInt("tests.iters", 1);
+
+ if (iters > 1 && failFast) {
if (maxFailures == Integer.MAX_VALUE) {
maxFailures = 1;
} else {
What do you think? Maybe we should also print a warning about using these
props without iters?
Shai
On Fri, May 10, 2013 at 10:45 PM, Dawid Weiss
<[email protected]>wrote:
> > Dawid promised me he will one day find the time to do it! :)
>
> I have it pretty high on my list of things to do in my spare time. The
> problem is twofold -- I have very few spare cycles at the moment and the
> problem is actually more complicated than it seems on the surface. I do
> have some ideas though.
>
> Maybe if you have a shellscript that's generic enough, it's worth putting
>> under dev-tools?
>>
>
> The simplest string would be something that just calls ant test
> -Dtests.seed=`...` repeatedly with a different seed and checks for the
> output status. This is doable even in bash, as in:
>
> for i in `seq 1 100`; do
> seed=`printf "%X%X" $RANDOM $RANDOM`
> echo "Iteration $i, seed: $seed"
> # ant -Dtests.seed=$seed
> # check if $? indicates a failure and abort if so.
> done
>
> D.
>
