On Dec 11, 2011, at 2:42 PM, Sean Owen wrote: > On Sun, Dec 11, 2011 at 7:35 PM, Ted Dunning <[email protected]> wrote: > >> The right way to handle this is to have instances get a random number >> generator that works like it should. Magic resets in the middle of >> operation are not a good idea. >> > > Why would the caller care? It's all random numbers, whether "reset" or not. > > >> >> I think we need a better way to inject generators that doesn't involve >> statics. >> >> > There is no way to control and reset Randoms other than what's done in > RandomWrapper (with statics). If you think deterministic tests are a good > goal, there's no argument with this, as it's the only way to do it. (And if > you don't agree that's another conversation -- but we do need this.)
In Lucene, we simply print out what the seed is if the tests fail and then you can rerun that test by saying ant -Dtestseed=XXXX test AFAICT, the issue with some of these tests, LogLikelihoodTest (the frequency comparison test fails when you change the seed) is that they are testing specific values as outcomes based on a specific test seed. That's OK, but it doesn't lend itself to the reset problem and it makes things problematic to debug should we ever change the seed, etc. > > To be clear: the injection is not at all the issue here. I'm happy to write > an essay on this, but, the punchline is: there is virtually no way to have > deterministic tests in one JVM while running all the tests in parallel. We > were pursuing means to execute subsets of the tests in parallel in > different JVMs, which can work and does work. We haven't yet found a way to > do it in Maven. This is where Ant seems to be a lot better.
