I forgot to mention: If a test fails, the message using the seed is printed to stdout. The developer can then change the test temporarily:
LuceneTestCase.newRandom() -> LuceneTestCase.newRandom(long) using the seed from the failed test printout. Uwe ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -----Original Message----- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Wednesday, February 04, 2009 2:22 PM > To: java-dev@lucene.apache.org > Subject: RE: failure in TestTrieRangeQuery > > Hi, > > atached is a patch for LuceneTestCase and a implementation in > TestTrieRangeQuery. It overrides the protected method runTest() and > inserts > a try-catch around the super.runTest() call. Two new methods newRandom() > and > newRandom(long) are available for the test case. As each test case is run > in > an own TestCase object instance (so 5 test methods in a class instantiate > 5 > instances each method working in separate), the random seed is saved on > newRandom() and when the test fails with any Throwable, a message with the > seed (if not null) is printed out. If newRandom was never called no > message > will be printed. > > This patch has only one problem: If a single test method calls newRandom() > more than once, only the last seed is saved and printed out. But each test > method in a Testcase should call newRandom() exactly once for usage during > the execution of this test method. And it is not thread save (no sync, no > volatile), but for tests it's unimportant. > > Maybe we open a JIRA issue? > > Uwe > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > -----Original Message----- > > From: Michael McCandless [mailto:luc...@mikemccandless.com] > > Sent: Wednesday, February 04, 2009 12:02 PM > > To: java-dev@lucene.apache.org > > Subject: Re: failure in TestTrieRangeQuery > > > > > > Sweet! > > > > I was wondering (but didn't dig) whether we could extend > > LuceneTestCase to expose a getRandom() method (which'd record the > > seed), and then override invocation of a test (which I'm not sure > > JUnit allows you to do) to add a try/finally that prints out the seeds. > > > > Though: I thought JUnit invokes tests in the sequential order as they > > are defined in your class? (I'm not sure about this... it's just what > > seems to be the case). And even if it is the case, it's not clear > > that's guaranteed as part of JUnit's "contract". If it isn't, we > > could have getRandom take a String name and then on exception we print > > out the full name -> seed for all getRandom calls for that test? > > > > I'd like to to find a simple common API, if we can, so that we can fix > > all tests that use Random to use it... though really the mods you had > > to make are fairly minimal, so we could simply adopt that per test too. > > > > Mike > > > > Uwe Schindler wrote: > > > > > Hi, > > > > > >>> : By allowing Random to randomly seed itself, we effectively test a > > >>> much > > >>> : much larger space, ie every time we all run the test, it's > > >>> different. We can > > >>> : potentially cast a much larger net than a fixed seed. > > >>> > > >>> i guess i'm just in favor of less randomness and more iterations. > > >>> > > >>> : Fixing the bug is the "easy" part; discovering a bug is present is > > >>> where > > >>> : we need all the help we can get ;) > > >>> > > >>> yes, but knowing a bug is there w/o having any idea what it is or > > >>> how to > > >>> trigger it can be very frustrating. > > >> > > >> I agree, it's frustrating. But I'd prefer to know the bug is there > > >> and then > > >> writhe in frustration at not being able to reproduce it very easily, > > >> then let > > >> the bug go undetected. I guess ignorance is not bliss, for me ;) > > >> > > >>> it would be enough for tests to pick a random number, log it, and > > >>> then use > > >>> it as the seed ... that way if you get a failure you at least know > > >>> what > > >>> seed was used and you can then hardcode it temporarily to reproduce/ > > >>> debug > > >> > > >> +1! I like this approach. We could record the seed up front, and > > >> then in > > >> a try/finally if the test failed, print the seed. > > > > > > I implemented this for TestTrieRangeQuery (see patch). I catch > > > java.lang.Error and print the random seed recorded before (The seed is > > > generated by a static Random instance for each test method in > > > separate: > > > Because you cannot predict the order of tests, each test method > > > should have > > > its own Random instance). As both the Java 1.4 AssertionError and > > > the jUnit > > > AssertionFailedError are subclasses of Error, they can be catched and > > > rethrown easily. > > > > > > Uwe > > > <random-trie- > > > test > > > .patch > > > >--------------------------------------------------------------------- > > > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > > > For additional commands, e-mail: java-dev-h...@lucene.apache.org > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-dev-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org