Tricky indeed, kudos Steve. D.
On Wed, Dec 28, 2016 at 10:20 AM, Michael McCandless <luc...@mikemccandless.com> wrote: > Phew, tricky! The patch looks good Steve. Thanks for digging and > beasting and finding a separate crab! > > Mike McCandless > > http://blog.mikemccandless.com > > > On Tue, Dec 27, 2016 at 10:00 PM, Steve Rowe <sar...@gmail.com> wrote: >> Yes, I’ve posted a couple of them on the issue, but they don’t reproduce. >> >> My beasting so far (couple thousand iterations) hasn’t yet reproduced. >> Beasting did find <https://issues.apache.org/jira/browse/LUCENE-7605> though. >> >> I have an idea about what might cause the problem: under multi-threaded use, >> in lookup(), the suggester’s SearcherManager could be re-assigned in the >> middle of an acquire/release cycle as a result of >> <https://issues.apache.org/jira/browse/LUCENE-7564>. I’m also beasting a >> change to take a local reference and use it for the whole cycle.[1] >> >> -- >> Steve >> www.lucidworks.com >> >> [1] Patch to hold SearcherManager constant in >> AnalyzingInfixSuggester.lookup(): >> >> —— >> diff --git >> a/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java >> >> b/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java >> index 1e98361..c63726f 100644 >> --- >> a/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java >> +++ >> b/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java >> @@ -642,7 +642,8 @@ public class AnalyzingInfixSuggester extends Lookup >> implements Closeable { >> // only retrieve the first num hits now: >> Collector c2 = new EarlyTerminatingSortingCollector(c, SORT, num); >> List<LookupResult> results = null; >> - IndexSearcher searcher = searcherMgr.acquire(); >> + SearcherManager mgr = searcherMgr; // use local reference to perform >> acquire & release on same SearcherManager >> + IndexSearcher searcher = mgr.acquire(); >> try { >> //System.out.println("got searcher=" + searcher); >> searcher.search(finalQuery, c2); >> @@ -653,7 +654,7 @@ public class AnalyzingInfixSuggester extends Lookup >> implements Closeable { >> // hits = searcher.search(query, null, num, SORT); >> results = createResults(searcher, hits, num, key, doHighlight, >> matchedTokens, prefixToken); >> } finally { >> - searcherMgr.release(searcher); >> + mgr.release(searcher); >> } >> >> //System.out.println((System.currentTimeMillis() - t0) + " msec for >> infix suggest"); >> —— >> >> >> >>> On Dec 26, 2016, at 5:56 PM, Michael McCandless <luc...@mikemccandless.com> >>> wrote: >>> >>> I think (not sure) these failures starting happening after >>> https://issues.apache.org/jira/browse/LUCENE-7564 ? >>> >>> Mike McCandless >>> >>> http://blog.mikemccandless.com >>> >>> >>> On Mon, Dec 26, 2016 at 4:29 PM, Apache Jenkins Server >>> <jenk...@builds.apache.org> wrote: >>>> Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/628/ >>>> >>>> 1 tests failed. >>>> FAILED: >>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest.testRandomNRT >>>> >>>> Error Message: >>>> Captured an uncaught exception in thread: Thread[id=29, name=Thread-11, >>>> state=RUNNABLE, group=TGRP-AnalyzingInfixSuggesterTest] >>>> >>>> Stack Trace: >>>> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an >>>> uncaught exception in thread: Thread[id=29, name=Thread-11, >>>> state=RUNNABLE, group=TGRP-AnalyzingInfixSuggesterTest] >>>> at >>>> __randomizedtesting.SeedInfo.seed([608DC176393E1CC5:C4A3CFCB61E1C079]:0) >>>> Caused by: org.apache.lucene.store.AlreadyClosedException: this >>>> ReferenceManager is closed >>>> at __randomizedtesting.SeedInfo.seed([608DC176393E1CC5]:0) >>>> at >>>> org.apache.lucene.search.ReferenceManager.acquire(ReferenceManager.java:98) >>>> at >>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:645) >>>> at >>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:457) >>>> at >>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest$LookupThread.run(AnalyzingInfixSuggesterTest.java:533) >>>> >>>> >>>> >>>> >>>> Build Log: >>>> [...truncated 9362 lines...] >>>> [junit4] Suite: >>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest >>>> [junit4] 2> ??? 26, 2016 1:30:30 ?? >>>> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler >>>> uncaughtException >>>> [junit4] 2> WARNING: Uncaught exception in thread: >>>> Thread[Thread-11,5,TGRP-AnalyzingInfixSuggesterTest] >>>> [junit4] 2> org.apache.lucene.store.AlreadyClosedException: this >>>> ReferenceManager is closed >>>> [junit4] 2> at >>>> __randomizedtesting.SeedInfo.seed([608DC176393E1CC5]:0) >>>> [junit4] 2> at >>>> org.apache.lucene.search.ReferenceManager.acquire(ReferenceManager.java:98) >>>> [junit4] 2> at >>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:645) >>>> [junit4] 2> at >>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:457) >>>> [junit4] 2> at >>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest$LookupThread.run(AnalyzingInfixSuggesterTest.java:533) >>>> [junit4] 2> >>>> [junit4] 2> NOTE: reproduce with: ant test >>>> -Dtestcase=AnalyzingInfixSuggesterTest -Dtests.method=testRandomNRT >>>> -Dtests.seed=608DC176393E1CC5 -Dtests.multiplier=2 -Dtests.slow=true >>>> -Dtests.locale=el -Dtests.timezone=SystemV/MST7MDT -Dtests.asserts=true >>>> -Dtests.file.encoding=US-ASCII >>>> [junit4] ERROR 32.8s J1 | AnalyzingInfixSuggesterTest.testRandomNRT <<< >>>> [junit4] > Throwable #1: >>>> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an >>>> uncaught exception in thread: Thread[id=29, name=Thread-11, >>>> state=RUNNABLE, group=TGRP-AnalyzingInfixSuggesterTest] >>>> [junit4] > at >>>> __randomizedtesting.SeedInfo.seed([608DC176393E1CC5:C4A3CFCB61E1C079]:0) >>>> [junit4] > Caused by: org.apache.lucene.store.AlreadyClosedException: >>>> this ReferenceManager is closed >>>> [junit4] > at >>>> __randomizedtesting.SeedInfo.seed([608DC176393E1CC5]:0) >>>> [junit4] > at >>>> org.apache.lucene.search.ReferenceManager.acquire(ReferenceManager.java:98) >>>> [junit4] > at >>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:645) >>>> [junit4] > at >>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:457) >>>> [junit4] > at >>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest$LookupThread.run(AnalyzingInfixSuggesterTest.java:533) >>>> [junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): >>>> {textgrams=PostingsFormat(name=Memory doPackFST= false), >>>> text=PostingsFormat(name=Memory doPackFST= false), >>>> contexts=PostingsFormat(name=LuceneVarGapFixedInterval), >>>> exacttext=PostingsFormat(name=LuceneVarGapFixedInterval)}, >>>> docValues:{payloads=DocValuesFormat(name=Memory), >>>> textgrams=DocValuesFormat(name=Memory), >>>> weight=DocValuesFormat(name=Lucene54), text=DocValuesFormat(name=Memory), >>>> contexts=DocValuesFormat(name=Lucene54), >>>> exacttext=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=846, >>>> maxMBSortInHeap=6.751687589960355, >>>> sim=RandomSimilarity(queryNorm=false,coord=crazy): {}, locale=el, >>>> timezone=SystemV/MST7MDT >>>> [junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation >>>> 1.8.0_102 (64-bit)/cpus=4,threads=1,free=158780648,total=343932928 >>>> [junit4] 2> NOTE: All tests run in this JVM: [TestContextQuery, >>>> TestSuggestStopFilter, TestPrefixCompletionQuery, >>>> AnalyzingInfixSuggesterTest] >>>> [junit4] Completed [28/32 (1!)] on J1 in 37.17s, 22 tests, 1 error <<< >>>> FAILURES! >>>> >>>> [...truncated 65892 lines...] >>>> >>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
