[
https://issues.apache.org/jira/browse/LUCENE-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13526633#comment-13526633
]
Shai Erera commented on LUCENE-4596:
------------------------------------
2% isn't critical, but I'd like to get to the bottom of it before I add the
sync back. For instance, I added this line to getParent:
{code}
assert ordinal < parents.length : "requested ordinal (" + ordinal + ");
parents.length (" + parents.length + ") !";
{code}
And the failures are:
{noformat}
7/Dez/2012 18:33:06
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
WARNING: Uncaught exception in thread:
Thread[Thread-10,5,TGRP-TestDirectoryTaxonomyWriter]
Throwable occurred: java.lang.AssertionError: requested ordinal (2078) >
parents.length (2078) !
at __randomizedtesting.SeedInfo.seed([1CE23D1B45DD9EE5]:0)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.getParent(DirectoryTaxonomyWriter.java:837)
at
org.apache.lucene.facet.taxonomy.directory.TestDirectoryTaxonomyWriter$4.run(TestDirectoryTaxonomyWriter.java:267)
7/Dez/2012 18:33:06
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
WARNING: Uncaught exception in thread:
Thread[Thread-6,5,TGRP-TestDirectoryTaxonomyWriter]
Throwable occurred: java.lang.AssertionError: requested ordinal (2338) >
parents.length (2338) !
at __randomizedtesting.SeedInfo.seed([1CE23D1B45DD9EE5]:0)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.getParent(DirectoryTaxonomyWriter.java:837)
at
org.apache.lucene.facet.taxonomy.directory.TestDirectoryTaxonomyWriter$4.run(TestDirectoryTaxonomyWriter.java:267)
Thread-9 growing array: curlen=2962 ord=2962 newlen=3334
Thread-11 growing array: curlen=3334 ord=3334 newlen=3752
7/Dez/2012 18:33:06
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
WARNING: Uncaught exception in thread:
Thread[Thread-9,5,TGRP-TestDirectoryTaxonomyWriter]
Throwable occurred: java.lang.AssertionError: requested ordinal (3334) >
parents.length (3334) !
at __randomizedtesting.SeedInfo.seed([1CE23D1B45DD9EE5]:0)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.getParent(DirectoryTaxonomyWriter.java:837)
at
org.apache.lucene.facet.taxonomy.directory.TestDirectoryTaxonomyWriter$4.run(TestDirectoryTaxonomyWriter.java:267)
7/Dez/2012 18:33:06
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
WARNING: Uncaught exception in thread:
Thread[Thread-8,5,TGRP-TestDirectoryTaxonomyWriter]
Throwable occurred: java.lang.AssertionError: requested ordinal (3334) >
parents.length (3334) !
at __randomizedtesting.SeedInfo.seed([1CE23D1B45DD9EE5]:0)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.getParent(DirectoryTaxonomyWriter.java:837)
at
org.apache.lucene.facet.taxonomy.directory.TestDirectoryTaxonomyWriter$4.run(TestDirectoryTaxonomyWriter.java:267)
7/Dez/2012 18:33:06
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
WARNING: Uncaught exception in thread:
Thread[Thread-7,5,TGRP-TestDirectoryTaxonomyWriter]
Throwable occurred: java.lang.AssertionError: requested ordinal (3334) >
parents.length (3334) !
at __randomizedtesting.SeedInfo.seed([1CE23D1B45DD9EE5]:0)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.getParent(DirectoryTaxonomyWriter.java:837)
at
org.apache.lucene.facet.taxonomy.directory.TestDirectoryTaxonomyWriter$4.run(TestDirectoryTaxonomyWriter.java:267)
{noformat}
Note how the requested ordinal always equals to parents.length !! I need to
figure this out, but perhaps another set of eyes can help. The test fails
pretty easily with the attached patch.
> DirectoryTaxonomyWriter concurrency bug
> ---------------------------------------
>
> Key: LUCENE-4596
> URL: https://issues.apache.org/jira/browse/LUCENE-4596
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/facet
> Reporter: Shai Erera
> Assignee: Shai Erera
> Priority: Blocker
> Fix For: 4.1, 5.0
>
> Attachments: LUCENE-4596.patch
>
>
> Mike tripped this error while running some benchmarks:
> {no format}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 130
> at
> org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.getParent(DirectoryTaxonomyWriter.java:835)
> at
> org.apache.lucene.facet.index.streaming.CategoryParentsStream.incrementToken(CategoryParentsStream.java:106)
> at
> org.apache.lucene.facet.index.streaming.CountingListTokenizer.incrementToken(CountingListTokenizer.java:63)
> at
> org.apache.lucene.facet.index.streaming.CategoryTokenizer.incrementToken(CategoryTokenizer.java:48)
> at
> org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:177)
> at
> org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:272)
> at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:250)
> at
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:376)
> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1455)
> at
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1130)
> at
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1111)
> at perf.IndexThreads$IndexThread.run(IndexThreads.java:335)
> {noformat}
> At first we thought this might be related to LUCENE-4565, but he reverted to
> before that commit and still hit the exception. I modified
> TestDirTaxoWriter.testConcurrency to index hierarchical categories, thinking
> that's the cause, but failed to reproduce.
> Eventually I realized that the test doesn't call getParent(), because it
> tests DirTaxoWriter concurrency, not concurrent indexing. As soon as I added
> a call to getParent, I hit this exception too.
> Adding 'synchronized' to DirTaxoWriter.addCategory seems to avoid that ex.
> I'll upload a patch with the modifications to the test and dig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
