[
https://issues.apache.org/jira/browse/LUCENE-7574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695686#comment-15695686
]
Ferenczi Jim commented on LUCENE-7574:
--------------------------------------
{quote}Can you explain the first patch a bit more? It felt correct to me to
take whether the sort is reversed into account in order to compute the missing
ordinal?{quote}
Yes but if we do it twice and that reverts the sort on missing ordinals, so my
patch is just removing one of them. Line 176 in
MultiSorter.CrossReaderComparator reverseMul is applied to the result of the
ordinals comparison. This is ok to do it on missing ordinals as well but we
already applied the reverseMul on missing ordinal line 160 so the result is
reversed. I hope it makes sense.
> Another TestIndexSorting failures
> ---------------------------------
>
> Key: LUCENE-7574
> URL: https://issues.apache.org/jira/browse/LUCENE-7574
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
> Affects Versions: 6.x, master (7.0)
> Reporter: Ferenczi Jim
> Attachments: LUCENE-7574-1.patch, LUCENE-7574-2.patch
>
>
> TestIndexSorting still fails with some seeds:
> {noformat}
> [junit4] Suite: org.apache.lucene.index.TestIndexSorting
> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexSorting
> -Dtests.method=testRandom3 -Dtests.seed=6E45BA611FCD7241 -Dtests.slow=true
> -Dtests.locale=ko-KR -Dtests.timezone=America/Cordoba -Dtests.asserts=true
> -Dtests.file.encoding=ISO-8859-1
> [junit4] FAILURE 0.64s J1 | TestIndexSorting.testRandom3 <<<
> [junit4] > Throwable #1: org.junit.ComparisonFailure: expected:<[449]>
> but was:<[650]>
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([6E45BA611FCD7241:CC9DF4BB7B3F5B47]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexSorting.testRandom3(TestIndexSorting.java:2264)
> [junit4] > at java.lang.Thread.run(Thread.java:745)
> [junit4] 2> NOTE: leaving temporary files on disk at:
> /var/lib/jenkins/workspace/apache+lucene-solr+branch_6x/lucene/build/core/test/J1/temp/lucene.index.TestIndexSorting_6E45BA611FCD7241-001
> [junit4] 2> NOTE: test params are: codec=Asserting(Lucene62):
> {docs=PostingsFormat(name=MockRandom), norms=PostingsFormat(name=MockRandom),
> positions=PostingsFormat(name=Memory doPackFST= true),
> id=PostingsFormat(name=MockRandom), term_vectors=FSTOrd50},
> docValues:{multi_valued_long=DocValuesFormat(name=Direct),
> double=DocValuesFormat(name=Lucene54), foo=DocValuesFormat(name=Direct),
> numeric=DocValuesFormat(name=Lucene54),
> positions=DocValuesFormat(name=Direct),
> multi_valued_numeric=DocValuesFormat(name=Memory),
> float=DocValuesFormat(name=Lucene54), int=DocValuesFormat(name=Memory),
> long=DocValuesFormat(name=Lucene54), points=DocValuesFormat(name=Memory),
> sorted=DocValuesFormat(name=Lucene54),
> multi_valued_double=DocValuesFormat(name=Memory),
> docs=DocValuesFormat(name=Memory),
> multi_valued_string=DocValuesFormat(name=Memory),
> norms=DocValuesFormat(name=Memory), bytes=DocValuesFormat(name=Memory),
> binary=DocValuesFormat(name=Lucene54), id=DocValuesFormat(name=Memory),
> multi_valued_int=DocValuesFormat(name=Lucene54),
> multi_valued_bytes=DocValuesFormat(name=Lucene54),
> multi_valued_float=DocValuesFormat(name=Lucene54),
> term_vectors=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=419,
> maxMBSortInHeap=7.394324294878203,
> sim=RandomSimilarity(queryNorm=true,coord=yes): {positions=DFR I(n)Z(0.3),
> id=IB SPL-L1, term_vectors=DFR I(ne)B3(800.0)}, locale=ko-KR,
> timezone=America/Cordoba
> [junit4] 2> NOTE: Linux 3.12.60-52.54-default amd64/Oracle Corporation
> 1.8.0_111 (64-bit)/cpus=4,threads=1,free=105782608,total=240648192
> [junit4] 2> NOTE: All tests run in this JVM: [TestPayloads,
> TestSnapshotDeletionPolicy, TestDocValues, FuzzyTermOnShortTermsTest,
> TestNoMergeScheduler, TestPointValues, TestSegmentInfos, TestStressIndexing2,
> TestSimpleExplanationsOfNonMatches, TestPrefixInBooleanQuery, TestTermQuery,
> TestSegmentMerger, TestByteArrayDataInput, TestTransactions, TestMultiFields,
> TestNRTReaderCleanup, TestPackedInts, TestIndexWriterExceptions,
> TestSleepingLockWrapper, TestBlockPostingsFormat2, TestLSBRadixSorter,
> TestSwappedIndexFiles, TestIndexWriterCommit, TestPrefixRandom,
> Test4GBStoredFields, TestFuzzyQuery, TestCodecUtil,
> TestSimpleSearchEquivalence, TestWeakIdentityMap, TestIndexWriterOnDiskFull,
> TestTopDocsMerge, TestOmitTf, TestDuelingCodecs, TestRAMDirectory,
> TestFlushByRamOrCountsPolicy, TestDemo, TestSimpleExplanationsWithFillerDocs,
> TestByteSlices, TestParallelLeafReader, TestSortedSetSelector,
> TestBagOfPostings, TestDemoParallelLeafReader, TestTopFieldCollector,
> TestSearchForDuplicates, TestStringHelper, TestTragicIndexWriterDeadlock,
> TestDirectPacked, TestSloppyMath, TestPrefixQuery, TestSimpleFSDirectory,
> TestFixedLengthBytesRefArray, TestIndexingSequenceNumbers, TestCharArraySet,
> TestRollingBuffer, TestPagedBytes, TestFixedBitSet, TestAutomaton,
> TestPhrasePrefixQuery, TestMultiPhraseEnum, TestBytesRefAttImpl,
> TestDocsAndPositions, TestCharsRefBuilder, TestDeterminizeLexicon,
> TestNIOFSDirectory, TestConjunctionDISI, TestLiveFieldValues, TestBoolean2,
> TestHighCompressionMode, TestIndexWriterUnicode, TestCachingCollector,
> TestMultiDocValues, TestFilterWeight, TestPerFieldPostingsFormat2,
> TestBytesRefHash, TestBooleanQueryVisitSubscorers, TestMatchAllDocsQuery,
> TestBinaryTerms, TestPositionIncrement, TestNumericTokenStream,
> TestDateTools, Test2BPostings, TestBinaryDocument, TestBooleanScorer,
> TestNot, TestReaderClosed, TestNGramPhraseQuery, TestSimpleAttributeImpl,
> Test2BPostingsBytes, Test2BTerms, TestReusableStringReader,
> TestLucene50StoredFieldsFormat, TestLucene50TermVectorsFormat, TestPolygon,
> Test2BPoints, Test2BSortedDocValuesFixedSorted, Test2BSortedDocValuesOrds,
> TestAllFilesCheckIndexHeader, TestAllFilesDetectTruncation,
> TestBinaryDocValuesUpdates, TestDocIDMerger, TestDuelingCodecsAtNight,
> TestExitableDirectoryReader, TestFilterDirectoryReader, TestIndexReaderClose,
> TestIndexSorting]
> [junit4] Completed [380/442 (1!)] on J1 in 11.77s, 47 tests, 1 failure <<<
> FAILURES!
> {noformat}
> {noformat}
> [junit4] Suite: org.apache.lucene.index.TestIndexSorting
> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexSorting
> -Dtests.method=testNumericAlreadySorted -Dtests.seed=1B0F8A58C7247131
> -Dtests.slow=true -Dtests.locale=tr-TR -Dtests.timezone=Etc/GMT+2
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
> [junit4] ERROR 0.17s J1 | TestIndexSorting.testNumericAlreadySorted <<<
> [junit4] > Throwable #1: java.io.IOException: background merge hit
> exception: _n(6.4.0):c111:[indexSort=<int: "foo">,<int: "id">] into _o
> [maxNumSegments=1]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([1B0F8A58C7247131:80F8B4206297188B]:0)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1910)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1850)
> [junit4] > at
> org.apache.lucene.index.TestIndexSorting.assertNeedsIndexSortMerge(TestIndexSorting.java:197)
> [junit4] > at
> org.apache.lucene.index.TestIndexSorting.testNumericAlreadySorted(TestIndexSorting.java:224)
> [junit4] > at java.lang.Thread.run(Thread.java:745)
> [junit4] > Caused by: java.lang.AssertionError: expected:<true> but
> was:<false>
> [junit4] > at
> org.apache.lucene.index.TestIndexSorting$AssertingNeedsIndexSortCodec$1$1.merge(TestIndexSorting.java:105)
> [junit4] > at
> org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:173)
> [junit4] > at
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:122)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4320)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
> [junit4] > at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
> [junit4] > at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
> [junit4] 2> Kas 24, 2016 11:55:11 PM
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> [junit4] 2> WARNING: Uncaught exception in thread: Thread[Lucene Merge
> Thread #2,5,TGRP-TestIndexSorting]
> [junit4] 2> org.apache.lucene.index.MergePolicy$MergeException:
> java.lang.AssertionError: expected:<true> but was:<false>
> [junit4] 2> at
> __randomizedtesting.SeedInfo.seed([1B0F8A58C7247131]:0)
> [junit4] 2> at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
> [junit4] 2> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
> [junit4] 2> Caused by: java.lang.AssertionError: expected:<true> but
> was:<false>
> [junit4] 2> at org.junit.Assert.fail(Assert.java:93)
> [junit4] 2> at org.junit.Assert.failNotEquals(Assert.java:647)
> [junit4] 2> at org.junit.Assert.assertEquals(Assert.java:128)
> [junit4] 2> at org.junit.Assert.assertEquals(Assert.java:147)
> [junit4] 2> at
> org.apache.lucene.index.TestIndexSorting$AssertingNeedsIndexSortCodec$1$1.merge(TestIndexSorting.java:105)
> [junit4] 2> at
> org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:173)
> [junit4] 2> at
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:122)
> [junit4] 2> at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4320)
> [junit4] 2> at
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
> [junit4] 2> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
> [junit4] 2> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
> [junit4] 2>
> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexSorting
> -Dtests.method=testConcurrentDVUpdates -Dtests.seed=1B0F8A58C7247131
> -Dtests.slow=true -Dtests.locale=tr-TR -Dtests.timezone=Etc/GMT+2
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
> {noformat}
> The first failure is only on branch_6x and is due to a problem in the
> backport of the multi-valued docvalues sort.
> The second failure is a test bug that can appear on master and branch_6x.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
