[
https://issues.apache.org/jira/browse/LUCENE-8043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16271767#comment-16271767
]
Simon Willnauer commented on LUCENE-8043:
-----------------------------------------
[~jpountz] [[email protected]] [~mikemccand] I think I found the issue. It
seems like we try to drop the same segment reader from the reader pool multiple
times during applying deletes which I am not 100% sure is expected or not. Yet,
due to that we also reduce the counter for that segment multiple times. With
this patch I can run the test 1k times without a failure. I am happy to provide
a patch for it but I wonder if this is an expected state? [~mikemccand] can you
tell.
{code}diff --git
a/lucene/core/src/java/org/apache/lucene/index/IndexWriter.java
b/lucene/core/src/java/org/apache/lucene/index/IndexWriter.java
index 7f47e42d45..586a294915 100644
--- a/lucene/core/src/java/org/apache/lucene/index/IndexWriter.java
+++ b/lucene/core/src/java/org/apache/lucene/index/IndexWriter.java
@@ -551,13 +551,15 @@ public class IndexWriter implements Closeable,
TwoPhaseCommit, Accountable {
return true;
}
- public synchronized void drop(SegmentCommitInfo info) throws IOException {
+ public synchronized boolean drop(SegmentCommitInfo info) throws
IOException {
final ReadersAndUpdates rld = readerMap.get(info);
if (rld != null) {
assert info == rld.info;
readerMap.remove(info);
rld.dropReaders();
+ return true;
}
+ return false;
}
public synchronized long ramBytesUsed() {
@@ -1616,10 +1618,9 @@ public class IndexWriter implements Closeable,
TwoPhaseCommit, Accountable {
// segment, we leave it in the readerPool; the
// merge will skip merging it and will then drop
// it once it's done:
- if (mergingSegments.contains(info) == false) {
+ if (mergingSegments.contains(info) == false && readerPool.drop(info)) {
segmentInfos.remove(info);
pendingNumDocs.addAndGet(-info.info.maxDoc());
- readerPool.drop(info);
}
}{code}
> Attempting to add documents past limit can corrupt index
> --------------------------------------------------------
>
> Key: LUCENE-8043
> URL: https://issues.apache.org/jira/browse/LUCENE-8043
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
> Affects Versions: 4.10, 7.0, master (8.0)
> Reporter: Yonik Seeley
> Attachments: LUCENE-8043.patch
>
>
> The IndexWriter check for too many documents does not always work, resulting
> in going over the limit. Once this happens, Lucene refuses to open the index
> and throws a CorruptIndexException: Too many documents.
> This appears to affect all versions of Lucene/Solr (the check was first
> implemented in LUCENE-5843 in v4.9.1/4.10 and we've seen this manifest in
> 4.10)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
