[
https://issues.apache.org/jira/browse/LUCENE-8253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440073#comment-16440073
]
Steve Rowe commented on LUCENE-8253:
------------------------------------
{{git bisect}} blames commit {{c70ccea}} on this issue for reproducing Solr
{{SegmentsInfoRequestHandlerTest}} failures, e.g. from
[https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21840/]:
{noformat}
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=SegmentsInfoRequestHandlerTest -Dtests.method=testSegmentInfosData
-Dtests.seed=D8FA27F4CB25E126 -Dtests.multiplier=3 -Dtests.slow=true
-Dtests.locale=it -Dtests.timezone=Europe/Kaliningrad -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 0.00s J2 |
SegmentsInfoRequestHandlerTest.testSegmentInfosData <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at
__randomizedtesting.SeedInfo.seed([D8FA27F4CB25E126:A45305AF7DCD56B9]:0)
[junit4] > at
org.apache.lucene.index.IndexWriter$ReaderPool.noDups(IndexWriter.java:867)
[junit4] > at
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:857)
[junit4] > at
org.apache.lucene.index.IndexWriter.numDeletesToMerge(IndexWriter.java:5233)
[junit4] > at
org.apache.lucene.index.LogMergePolicy.sizeDocs(LogMergePolicy.java:153)
[junit4] > at
org.apache.lucene.index.LogDocMergePolicy.size(LogDocMergePolicy.java:44)
[junit4] > at
org.apache.lucene.index.LogMergePolicy.findMerges(LogMergePolicy.java:469)
[junit4] > at
org.apache.solr.handler.admin.SegmentsInfoRequestHandler.getMergeCandidatesNames(SegmentsInfoRequestHandler.java:100)
[junit4] > at
org.apache.solr.handler.admin.SegmentsInfoRequestHandler.getSegmentsInfo(SegmentsInfoRequestHandler.java:59)
[junit4] > at
org.apache.solr.handler.admin.SegmentsInfoRequestHandler.handleRequestBody(SegmentsInfoRequestHandler.java:48)
[junit4] > at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
[junit4] > at
org.apache.solr.core.SolrCore.execute(SolrCore.java:2508)
[junit4] > at
org.apache.solr.util.TestHarness.query(TestHarness.java:337)
[junit4] > at
org.apache.solr.util.TestHarness.query(TestHarness.java:319)
[junit4] > at
org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:890)
[junit4] > at
org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfosData(SegmentsInfoRequestHandlerTest.java:75)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at
java.base/java.lang.reflect.Method.invoke(Method.java:564)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:844)
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=SegmentsInfoRequestHandlerTest -Dtests.method=testSegmentInfos
-Dtests.seed=D8FA27F4CB25E126 -Dtests.multiplier=3 -Dtests.slow=true
-Dtests.locale=it -Dtests.timezone=Europe/Kaliningrad -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 0.00s J2 | SegmentsInfoRequestHandlerTest.testSegmentInfos
<<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at
__randomizedtesting.SeedInfo.seed([D8FA27F4CB25E126:94ADB04B01293976]:0)
[junit4] > at
org.apache.lucene.index.IndexWriter$ReaderPool.noDups(IndexWriter.java:867)
[junit4] > at
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:857)
[junit4] > at
org.apache.lucene.index.IndexWriter.numDeletesToMerge(IndexWriter.java:5233)
[junit4] > at
org.apache.lucene.index.LogMergePolicy.sizeDocs(LogMergePolicy.java:153)
[junit4] > at
org.apache.lucene.index.LogDocMergePolicy.size(LogDocMergePolicy.java:44)
[junit4] > at
org.apache.lucene.index.LogMergePolicy.findMerges(LogMergePolicy.java:469)
[junit4] > at
org.apache.solr.handler.admin.SegmentsInfoRequestHandler.getMergeCandidatesNames(SegmentsInfoRequestHandler.java:100)
[junit4] > at
org.apache.solr.handler.admin.SegmentsInfoRequestHandler.getSegmentsInfo(SegmentsInfoRequestHandler.java:59)
[junit4] > at
org.apache.solr.handler.admin.SegmentsInfoRequestHandler.handleRequestBody(SegmentsInfoRequestHandler.java:48)
[junit4] > at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
[junit4] > at
org.apache.solr.core.SolrCore.execute(SolrCore.java:2508)
[junit4] > at
org.apache.solr.util.TestHarness.query(TestHarness.java:337)
[junit4] > at
org.apache.solr.util.TestHarness.query(TestHarness.java:319)
[junit4] > at
org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:890)
[junit4] > at
org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfos(SegmentsInfoRequestHandlerTest.java:61)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at
java.base/java.lang.reflect.Method.invoke(Method.java:564)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:844)
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=SegmentsInfoRequestHandlerTest
-Dtests.method=testSegmentInfosVersion -Dtests.seed=D8FA27F4CB25E126
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=it
-Dtests.timezone=Europe/Kaliningrad -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 0.00s J2 |
SegmentsInfoRequestHandlerTest.testSegmentInfosVersion <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at
__randomizedtesting.SeedInfo.seed([D8FA27F4CB25E126:2024B21CB3493075]:0)
[junit4] > at
org.apache.lucene.index.IndexWriter$ReaderPool.noDups(IndexWriter.java:867)
[junit4] > at
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:857)
[junit4] > at
org.apache.lucene.index.IndexWriter.numDeletesToMerge(IndexWriter.java:5233)
[junit4] > at
org.apache.lucene.index.LogMergePolicy.sizeDocs(LogMergePolicy.java:153)
[junit4] > at
org.apache.lucene.index.LogDocMergePolicy.size(LogDocMergePolicy.java:44)
[junit4] > at
org.apache.lucene.index.LogMergePolicy.findMerges(LogMergePolicy.java:469)
[junit4] > at
org.apache.solr.handler.admin.SegmentsInfoRequestHandler.getMergeCandidatesNames(SegmentsInfoRequestHandler.java:100)
[junit4] > at
org.apache.solr.handler.admin.SegmentsInfoRequestHandler.getSegmentsInfo(SegmentsInfoRequestHandler.java:59)
[junit4] > at
org.apache.solr.handler.admin.SegmentsInfoRequestHandler.handleRequestBody(SegmentsInfoRequestHandler.java:48)
[junit4] > at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
[junit4] > at
org.apache.solr.core.SolrCore.execute(SolrCore.java:2508)
[junit4] > at
org.apache.solr.util.TestHarness.query(TestHarness.java:337)
[junit4] > at
org.apache.solr.util.TestHarness.query(TestHarness.java:319)
[junit4] > at
org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:890)
[junit4] > at
org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfosVersion(SegmentsInfoRequestHandlerTest.java:68)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at
java.base/java.lang.reflect.Method.invoke(Method.java:564)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:844)
[...]
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70):
{name=PostingsFormat(name=Memory), id=PostingsFormat(name=Memory)},
docValues:{}, maxPointsInLeafNode=375, maxMBSortInHeap=6.1720643265572495,
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@50ce333a),
locale=it, timezone=Europe/Kaliningrad
[junit4] 2> NOTE: Linux 4.13.0-37-generic amd64/Oracle Corporation 9.0.4
(64-bit)/cpus=8,threads=1,free=136326496,total=536870912
{noformat}
{{IndexWriter.ReaderPool.noDups()}} is:
{code:java|title:IndexWriter.java}
862: // Make sure that every segment appears only once in the
863: // pool:
864: private boolean noDups() {
865: Set<String> seen = new HashSet<>();
866: for(SegmentCommitInfo info : readerMap.keySet()) {
867: assert !seen.contains(info.info.name);
868: seen.add(info.info.name);
869: }
870: return true;
871: }
872: }
{code}
> ForceMergeDeletes does not merge soft-deleted segments
> ------------------------------------------------------
>
> Key: LUCENE-8253
> URL: https://issues.apache.org/jira/browse/LUCENE-8253
> Project: Lucene - Core
> Issue Type: Bug
> Affects Versions: 7.4, master (8.0)
> Reporter: Nhat Nguyen
> Assignee: Simon Willnauer
> Priority: Major
> Attachments: LUCENE-8253.patch, test-merge.patch
>
>
> IndexWriter#forceMergeDeletes should merge segments having soft-deleted
> documents as hard-deleted documents if we configured "softDeletesField" in an
> IndexWriterConfig.
> Attached is a failed test.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
