[
https://issues.apache.org/jira/browse/LUCENE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219016#comment-13219016
]
Dawid Weiss commented on LUCENE-3808:
-------------------------------------
I've hacked a local branch where I substituted LuceneTestCaseRunner for
RandomizedRunner. And boy, hell broke loose ;)
So... the biggest issue I'm facing is indeed with Random sharing across
threads. One may argue that sharing a Random instance across threads (with data
races) is in fact making it even more random, but this doesn't seem to be the
direction consistent with the possibility of consistently re-running the tests
given the same starting seed. I know it isn't possible to coordinate threads
anyway but I would still like to have Random instances to return identical
sequences from within a single thread's perspective.
After this lengthy introduction, here comes an example of Random sharing:
{noformat}
> Caused by: java.lang.RuntimeException: java.lang.IllegalStateException:
This Random was created for/by another thread
(Thread[RandomizedRunner-SuiteThread-org.apache.lucene.search.TestSearcherManager-seed#[3F1A26A6DA253628],5,RandomizedRunner-SuiteThreadGroup-org.apache.lucene.search.TestSearcherManager]).
Random instances must not be shared (acquire per-thread). Current thread:
Thread[Thread-53,5,RandomizedRunner-SuiteThreadGroup-org.apache.lucene.search.TestSearcherManager]
> at __randomizedtesting.SeedInfo.seed([3F1A26A6DA253628]:0)
> at
org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$1.run(ThreadedIndexingAndSearchingTestCase.java:307)
> Caused by: java.lang.IllegalStateException: This Random was created for/by
another thread
(Thread[RandomizedRunner-SuiteThread-org.apache.lucene.search.TestSearcherManager-seed#[3F1A26A6DA253628],5,RandomizedRunner-SuiteThreadGroup-org.apache.lucene.search.TestSearcherManager]).
Random instances must not be shared (acquire per-thread). Current thread:
Thread[Thread-53,5,RandomizedRunner-SuiteThreadGroup-org.apache.lucene.search.TestSearcherManager]
> at
com.carrotsearch.randomizedtesting.RandomNoSetSeed.checkValid(RandomNoSetSeed.java:124)
> at
com.carrotsearch.randomizedtesting.RandomNoSetSeed.nextInt(RandomNoSetSeed.java:72)
> at org.apache.lucene.util.LuceneTestCase.rarely(LuceneTestCase.java:938)
> at
org.apache.lucene.analysis.MockAnalyzer.maybePayload(MockAnalyzer.java:102)
> at
org.apache.lucene.analysis.MockAnalyzer.createComponents(MockAnalyzer.java:95)
> at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:83)
> at org.apache.lucene.document.Field.tokenStream(Field.java:467)
> at
org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:86)
> at
org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:287)
> at
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:241)
> at
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:376)
> at
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1533)
> at
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1505)
> at
org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.updateDocument(ThreadedIndexingAndSearchingTestCase.java:111)
> at
org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$1.run(ThreadedIndexingAndSearchingTestCase.java:259)
> Caused by: com.carrotsearch.randomizedtesting.NotAnException: Original
allocation stack for this Random.
> at java.lang.Thread.getStackTrace(Thread.java:1479)
> at
com.carrotsearch.randomizedtesting.RandomNoSetSeed.<init>(RandomNoSetSeed.java:26)
> at
com.carrotsearch.randomizedtesting.Randomness.<init>(Randomness.java:20)
> at
com.carrotsearch.randomizedtesting.Randomness.<init>(Randomness.java:24)
> at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.run(RandomizedRunner.java:531)
> at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:551)
> at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$3(RandomizedRunner.java:495)
> at
com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:472)
{noformat}
What's happening is that a Random is pushed to MockAnalyzer and then multiple
threads access the same instance. I am investigating possible workarounds for
this -- Random instances are in fact thread local in RandomizedRunner so it
would be possible to just grab your own Random in places like MockAnalyzer,
without passing a ready to use instance.
An ostrich solution is to dodge the issue and simply drop checking if a Random
instance is shared or not, but it wouldn't be so much fun then, would it?
> Switch LuceneTestCaseRunner to RandomizedRunner. Enforce Random sharing
> contracts. Enforce thread leaks.
> --------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-3808
> URL: https://issues.apache.org/jira/browse/LUCENE-3808
> Project: Lucene - Java
> Issue Type: Sub-task
> Components: general/test
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
>
> Switch the runner to RandomizedRunner. Enforce the following:
> - Random sharing will result in a failure/ exception.
> - Thread leaks will result in a failure (add lingering if needed, but no
> ignores).
> - Add a validator for @Test method overrides (no-no).
> - Add a validator for static hook shadowing (no-no).
> - -Add a validator for testXXX without @Test annotation.-
> - Modify custom execution groups in LTC to be real @Groups.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
