[
https://issues.apache.org/jira/browse/LUCENE-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458607#comment-13458607
]
Shai Erera commented on LUCENE-4402:
------------------------------------
Found the problem. On 3.6, all DirTW methods are synchronized, but addTaxonomy
wasn't. The idea was to let someone call addTaxo in parallel to addCategory
(hence the test), while addTaxo itself calls addCategory internally (which is
synchronized). The problem is that prior to calling addCategory, it calls
findCategory, which is not synchronized and cache implementations are not
thread safe (in 3.6).
So the solution is to call addCategory from addTaxonomy in 3x, 4x and trunk. In
4x and trunk addCategory is not synchronized, checking the cache first, so the
existing code is redundant. In 3x since we cannot trust findCategory in a
multi-threaded env., calling addCategory will solve it.
I'll modify the code and commit the changes - a patch is trivial therefore I
won't post one.
> TestAddTaxonomy.testConcurrency failure
> ---------------------------------------
>
> Key: LUCENE-4402
> URL: https://issues.apache.org/jira/browse/LUCENE-4402
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/facet
> Affects Versions: 3.6.2
> Reporter: Robert Muir
>
> on the 3.x branch:
> {noformat}
> [junit] Testsuite:
> org.apache.lucene.facet.taxonomy.directory.TestAddTaxonomy
> [junit] Testcase:
> testConcurrency(org.apache.lucene.facet.taxonomy.directory.TestAddTaxonomy):
> Caused an ERROR
> [junit] Index: 1, Size: 2
> [junit] java.lang.IndexOutOfBoundsException: Index: 1, Size: 2
> [junit] at java.util.ArrayList.rangeCheck(ArrayList.java:604)
> [junit] at java.util.ArrayList.get(ArrayList.java:382)
> [junit] at
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CharBlockArray.charAt(CharBlockArray.java:148)
> [junit] at
> org.apache.lucene.facet.taxonomy.CategoryPath.equalsToSerialized(CategoryPath.java:888)
> [junit] at
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:323)
> [junit] at
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:163)
> [junit] at
> org.apache.lucene.facet.taxonomy.writercache.cl2o.Cl2oTaxonomyWriterCache.get(Cl2oTaxonomyWriterCache.java:49)
> [junit] at
> org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.findCategory(DirectoryTaxonomyWriter.java:386)
> [junit] at
> org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.addTaxonomy(DirectoryTaxonomyWriter.java:833)
> [junit] at
> org.apache.lucene.facet.taxonomy.directory.TestAddTaxonomy.testConcurrency(TestAddTaxonomy.java:206)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:601)
> [junit] at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> [junit] at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> [junit] at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> [junit] at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> [junit] at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> [junit] at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> [junit] at
> org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:630)
> [junit] at
> org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:536)
> [junit] at
> org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:67)
> [junit] at
> org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:457)
> [junit] at
> org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:74)
> [junit] at
> org.apache.lucene.util.LuceneTestCase$SaveThreadAndTestNameRule$1.evaluate(LuceneTestCase.java:508)
> [junit] at org.junit.rules.RunRules.evaluate(RunRules.java:18)
> [junit] at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> [junit] at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> [junit] at
> org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:146)
> [junit] at
> org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50)
> [junit] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> [junit] at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> [junit] at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> [junit] at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> [junit] at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> [junit] at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> [junit] at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> [junit] at
> org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:74)
> [junit] at
> org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:36)
> [junit] at
> org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:67)
> [junit] at org.junit.rules.RunRules.evaluate(RunRules.java:18)
> [junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> [junit] at
> junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
> [junit] at
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
> [junit] at
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
> [junit] at
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:879)
> [junit]
> [junit]
> [junit] Tests run: 5, Failures: 0, Errors: 1, Time elapsed: 0.342 sec
> [junit]
> [junit] ------------- Standard Error -----------------
> [junit] NOTE: Ignoring nightly-only test method 'testBig'
> [junit] WARNING: test method: 'testConcurrency' left thread running:
> Thread[Thread-1,5,main]
> [junit] RESOURCE LEAK: test method: 'testConcurrency' left 1 thread(s)
> running
> [junit] NOTE: reproduce with: ant test -Dtestcase=TestAddTaxonomy
> -Dtestmethod=testConcurrency
> -Dtests.seed=ad99a0aac3d5bf3:4af538a36d0c94b4:-7b609955992bc1c9
> -Dargs="-Dfile.encoding=UTF-8"
> [junit] NOTE: reproduce with: ant test -Dtestcase=TestAddTaxonomy
> -Dtestmethod=testConcurrency
> -Dtests.seed=ad99a0aac3d5bf3:4af538a36d0c94b4:-7b609955992bc1c9
> -Dargs="-Dfile.encoding=UTF-8"
> [junit] NOTE: test params are: locale=mt_MT, timezone=Etc/GMT+6
> [junit] NOTE: all tests run in this JVM:
> [junit] [TwoEnhancementsTest, TestAdaptiveExample,
> CategoryListPayloadStreamTest, CategoryAttributeImplTest,
> PerDimensionIndexingParamsTest, AdaptiveAccumulatorTest, TestFacetArrays,
> TestFacetsAccumulatorWithComplement, FacetSearchParamsTest,
> SamplingAccumulatorTest, TestTaxonomyCombined, TestAddTaxonomy]
> [junit] NOTE: Linux 3.2.0-24-generic amd64/Oracle Corporation 1.7.0_01
> (64-bit)/cpus=8,threads=1,free=171017984,total=269352960
> [junit] ------------- ---------------- ---------------
> [junit] TEST org.apache.lucene.facet.taxonomy.directory.TestAddTaxonomy
> FAILED
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
