[
https://issues.apache.org/jira/browse/LUCENE-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15693248#comment-15693248
]
Ferenczi Jim commented on LUCENE-7569:
--------------------------------------
{quote}I'm not sure why I didn't realize we must do so as well for the sorted
set case. {quote}
I am not sure it is needed. The iterations are stateless in 6x since it's just
an iteration over the docids and the norm ids and we call setDocument on the
underlying doc values reader each time we need to access it ?
> TestIndexSorting failures
> -------------------------
>
> Key: LUCENE-7569
> URL: https://issues.apache.org/jira/browse/LUCENE-7569
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Steve Rowe
> Assignee: Michael McCandless
> Priority: Blocker
> Fix For: master (7.0), 6.4
>
> Attachments: LUCENE-7569.patch
>
>
> My Jenkins found two reproducing seeds on branch_6x - these look different,
> but the failures happened on consecutive nightly runs:
> {noformat}
> Checking out Revision 535bf59a3b239f5c7bcd8c00f3e452c9b5e9b539
> (refs/remotes/origin/branch_6x)
> [...]
> [junit4] Suite: org.apache.lucene.index.TestIndexSorting
> [junit4] 2> ??? 18, 2016 9:50:39 AM
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> [junit4] 2> WARNING: Uncaught exception in thread: Thread[Lucene Merge
> Thread #0,5,TGRP-TestIndexSorting]
> [junit4] 2> org.apache.lucene.index.MergePolicy$MergeException:
> java.lang.AssertionError: nextValue=4594289799775307848 vs
> previous=4606302611760746829
> [junit4] 2> at
> __randomizedtesting.SeedInfo.seed([5F8898DCABBFD056]:0)
> [junit4] 2> at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
> [junit4] 2> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
> [junit4] 2> Caused by: java.lang.AssertionError:
> nextValue=4594289799775307848 vs previous=4606302611760746829
> [junit4] 2> at
> org.apache.lucene.codecs.asserting.AssertingDocValuesFormat$AssertingDocValuesConsumer.addSortedNumericField(AssertingDocValuesFormat.java:152)
> [junit4] 2> at
> org.apache.lucene.codecs.DocValuesConsumer.mergeSortedNumericField(DocValuesConsumer.java:470)
> [junit4] 2> at
> org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:243)
> [junit4] 2> at
> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:153)
> [junit4] 2> at
> org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:167)
> [junit4] 2> at
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
> [junit4] 2> at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4320)
> [junit4] 2> at
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
> [junit4] 2> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
> [junit4] 2> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
> [junit4] 2>
> [junit4] 2> NOTE: download the large Jenkins line-docs file by running
> 'ant get-jenkins-line-docs' in the lucene directory.
> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexSorting
> -Dtests.method=testRandom3 -Dtests.seed=5F8898DCABBFD056 -Dtests.multiplier=2
> -Dtests.nightly=true -Dtests.slow=true
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=he -Dtests.timezone=Canada/Central -Dtests.asserts=true
> -Dtests.file.encoding=ISO-8859-1
> [junit4] ERROR 14.9s J5 | TestIndexSorting.testRandom3 <<<
> [junit4] > Throwable #1:
> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
> [junit4] > at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:748)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:762)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1566)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1315)
> [junit4] > at
> org.apache.lucene.index.TestIndexSorting.testRandom3(TestIndexSorting.java:2019)
> [junit4] > at java.lang.Thread.run(Thread.java:745)
> [junit4] > Caused by: java.lang.AssertionError:
> nextValue=4594289799775307848 vs previous=4606302611760746829
> [junit4] > at
> org.apache.lucene.codecs.asserting.AssertingDocValuesFormat$AssertingDocValuesConsumer.addSortedNumericField(AssertingDocValuesFormat.java:152)
> [junit4] > at
> org.apache.lucene.codecs.DocValuesConsumer.mergeSortedNumericField(DocValuesConsumer.java:470)
> [junit4] > at
> org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:243)
> [junit4] > at
> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:153)
> [junit4] > at
> org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:167)
> [junit4] > at
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4320)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
> [junit4] > at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
> [junit4] > at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)Throwable
> #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=1115, name=Lucene Merge Thread #0,
> state=RUNNABLE, group=TGRP-TestIndexSorting]
> [junit4] > Caused by:
> org.apache.lucene.index.MergePolicy$MergeException: java.lang.AssertionError:
> nextValue=4594289799775307848 vs previous=4606302611760746829
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([5F8898DCABBFD056]:0)
> [junit4] > at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
> [junit4] > at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
> [junit4] > Caused by: java.lang.AssertionError:
> nextValue=4594289799775307848 vs previous=4606302611760746829
> [junit4] > at
> org.apache.lucene.codecs.asserting.AssertingDocValuesFormat$AssertingDocValuesConsumer.addSortedNumericField(AssertingDocValuesFormat.java:152)
> [junit4] > at
> org.apache.lucene.codecs.DocValuesConsumer.mergeSortedNumericField(DocValuesConsumer.java:470)
> [junit4] > at
> org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:243)
> [junit4] > at
> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:153)
> [junit4] > at
> org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:167)
> [junit4] > at
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4320)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
> [junit4] > at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
> [junit4] > at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
> [junit4] 2> NOTE: leaving temporary files on disk at:
> /var/lib/jenkins/jobs/Lucene-Solr-Nightly-6.x/workspace/lucene/build/core/test/J5/temp/lucene.index.TestIndexSorting_5F8898DCABBFD056-001
> [junit4] 2> NOTE: test params are: codec=Asserting(Lucene62):
> {docs=PostingsFormat(name=Direct), norms=PostingsFormat(name=Direct),
> positions=PostingsFormat(name=Memory doPackFST= false),
> id=PostingsFormat(name=Direct),
> term_vectors=PostingsFormat(name=LuceneFixedGap)},
> docValues:{multi_valued_long=DocValuesFormat(name=Lucene54),
> double=DocValuesFormat(name=Direct), foo=DocValuesFormat(name=Lucene54),
> numeric=DocValuesFormat(name=Direct),
> positions=DocValuesFormat(name=Lucene54),
> multi_valued_numeric=DocValuesFormat(name=Asserting),
> float=DocValuesFormat(name=Memory), int=DocValuesFormat(name=Asserting),
> long=DocValuesFormat(name=Memory), points=DocValuesFormat(name=Asserting),
> sorted=DocValuesFormat(name=Direct),
> multi_valued_double=DocValuesFormat(name=Asserting),
> docs=DocValuesFormat(name=Asserting),
> multi_valued_string=DocValuesFormat(name=Asserting),
> norms=DocValuesFormat(name=Asserting), bytes=DocValuesFormat(name=Asserting),
> binary=DocValuesFormat(name=Direct), id=DocValuesFormat(name=Asserting),
> multi_valued_int=DocValuesFormat(name=Direct),
> multi_valued_bytes=DocValuesFormat(name=Direct),
> multi_valued_float=DocValuesFormat(name=Memory),
> term_vectors=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=1312,
> maxMBSortInHeap=5.376962888308618,
> sim=RandomSimilarity(queryNorm=true,coord=yes): {positions=DFR I(ne)3(800.0),
> id=IB LL-L3(800.0), term_vectors=DFR I(ne)B3(800.0)}, locale=he,
> timezone=Canada/Central
> [junit4] 2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation
> 1.8.0_77 (64-bit)/cpus=16,threads=1,free=132261744,total=508559360
> [junit4] 2> NOTE: All tests run in this JVM: [TestFixedBitSet,
> TestSearchAfter, TestMinShouldMatch2, TestStopFilter, TestMmapDirectory,
> TestAutomaton, TestTransactions, TestMultiLevelSkipList, TestForUtil,
> TestFieldType, TestIntBlockPool, TestNumericRangeQuery64, TestBoolean2,
> Test2BPositions, TestDemo, TestSpanBoostQuery, TestSpanMultiTermQueryWrapper,
> TestCharsRefBuilder, TestIOUtils, TestAllFilesHaveChecksumFooter,
> TestMaxTermFrequency, TestConjunctionDISI, TestIndexWriterMerging,
> TestPackedInts, TestLucene50StoredFieldsFormat, TestGeoEncodingUtils,
> TestGeoUtils, Test2BPoints, Test2BSortedDocValuesFixedSorted, TestDocValues,
> TestExceedMaxTermLength, TestExitableDirectoryReader,
> TestFilterDirectoryReader, TestIndexSorting]
> [junit4] Completed [345/441 (1!)] on J5 in 39.46s, 43 tests, 1 error <<<
> FAILURES!
> {noformat}
> {noformat}
> Checking out Revision 500f6c7875be31c34ca68c58f850b7e64fd049a9
> (refs/remotes/origin/branch_6x)
> [...]
> [junit4] Suite: org.apache.lucene.index.TestIndexSorting
> [junit4] 2> NOTE: download the large Jenkins line-docs file by running
> 'ant get-jenkins-line-docs' in the lucene directory.
> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexSorting
> -Dtests.method=testRandom3 -Dtests.seed=5D1E2D97B0777D5C -Dtests.multiplier=2
> -Dtests.nightly=true -Dtests.slow=true
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=fi-FI -Dtests.timezone=America/Tegucigalpa
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
> [junit4] ERROR 245s J3 | TestIndexSorting.testRandom3 <<<
> [junit4] > Throwable #1: java.lang.RuntimeException: segment has
> indexSort=<float: "float">! missingValue=0.38226813,<sortedset:
> "multi_valued_bytes">! selector=MIN,<int: "id"> but docID=75 sorts after
> docID=76
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([5D1E2D97B0777D5C:FFC6634DD485545A]:0)
> [junit4] > at
> org.apache.lucene.index.CheckIndex.testSort(CheckIndex.java:876)
> [junit4] > at
> org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:756)
> [junit4] > at
> org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:300)
> [junit4] > at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:865)
> [junit4] > at
> org.apache.lucene.util.IOUtils.close(IOUtils.java:89)
> [junit4] > at
> org.apache.lucene.util.IOUtils.close(IOUtils.java:76)
> [junit4] > at
> org.apache.lucene.index.TestIndexSorting.testRandom3(TestIndexSorting.java:2087)
> [junit4] > at java.lang.Thread.run(Thread.java:745)
> [junit4] 2> NOTE: leaving temporary files on disk at:
> /var/lib/jenkins/jobs/Lucene-Solr-Nightly-6.x/workspace/lucene/build/core/test/J3/temp/lucene.index.TestIndexSorting_5D1E2D97B0777D5C-001
> [junit4] 2> NOTE: test params are: codec=CheapBastard,
> sim=RandomSimilarity(queryNorm=true,coord=no): {positions=DFR I(F)BZ(0.3),
> id=IB SPL-DZ(0.3), term_vectors=DFR I(ne)3(800.0)}, locale=fi-FI,
> timezone=America/Tegucigalpa
> [junit4] 2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation
> 1.8.0_77 (64-bit)/cpus=16,threads=1,free=252779544,total=525336576
> [junit4] 2> NOTE: All tests run in this JVM: [TestBooleanRewrites,
> TestSearchForDuplicates, TestSwappedIndexFiles, TestClassicSimilarity,
> TestNeverDelete, TestAllFilesCheckIndexHeader, TestBooleanScorer,
> TestPayloads, TestDelegatingAnalyzerWrapper, TestWeakIdentityMap,
> TestRollingBuffer, TestParallelCompositeReader, TestExitableDirectoryReader,
> TestCollectionUtil, TestLegacyNumericUtils, TestBagOfPostings, TestNot,
> TestPhrasePrefixQuery, TestSimpleExplanationsWithFillerDocs,
> TestPriorityQueue, TestBlockPostingsFormat2, TestIntroSorter,
> FiniteStringsIteratorTest, TestIndexWriterReader, TestNumericRangeQuery64,
> TestDeletionPolicy, TestBooleanOr, TestSloppyPhraseQuery, TestNRTThreads,
> TestPolygon, TestPolygon2D, TestDocValues, TestFilterDirectoryReader,
> TestIndexSorting]
> [junit4] Completed [431/441 (1!)] on J3 in 263.65s, 43 tests, 1 error <<<
> FAILURES!
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
