[jira] [Updated] (LUCENE-8328) ReadersAndUpdates#getLatestReader should execute under lock
[ https://issues.apache.org/jira/browse/LUCENE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nhat Nguyen updated LUCENE-8328: Attachment: LUCENE-8328.patch > ReadersAndUpdates#getLatestReader should execute under lock > --- > > Key: LUCENE-8328 > URL: https://issues.apache.org/jira/browse/LUCENE-8328 > Project: Lucene - Core > Issue Type: Bug >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Priority: Major > Attachments: LUCENE-8328.patch, LUCENE-8328.patch > > > It's possible for a merge thread to acquire an index reader which is closed > before it can incRef. > {noformat} > Merge stack trace: > at __randomizedtesting.SeedInfo.seed([136983A068AA2F9D]:0) > at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257) > at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) > at > org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) > at > org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) > at > org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) > at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) > at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) > at > org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140) > Refresh stack trace: > at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) > at > org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) > at > org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686) > at > org.apache.lucene.index.ReadersAndUpdates.getLatestReader(ReadersAndUpdates.java:260) > at > org.elasticsearch.index.shard.ElasticsearchMergePolicy.keepFullyDeletedSegment(ElasticsearchMergePolicy.java:143) > at > org.apache.lucene.index.ReadersAndUpdates.keepFullyDeletedSegment(ReadersAndUpdates.java:769) > at org.apache.lucene.index.IndexWriter.isFullyDeleted(IndexWriter.java:5124) > at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3306) > at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:514) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) > at > org.apache.lucene.index.FilterDirectoryReader.doOpenIfChanged(FilterDirectoryReader.java:104) > at > org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140){noformat} > The problem is that `ReadersAndUpdates#getLatestReader` is executed > concurrently without holding lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8328) ReadersAndUpdates#getLatestReader should execute under lock
[ https://issues.apache.org/jira/browse/LUCENE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nhat Nguyen updated LUCENE-8328: Attachment: (was: LUCENE-8328.patch) > ReadersAndUpdates#getLatestReader should execute under lock > --- > > Key: LUCENE-8328 > URL: https://issues.apache.org/jira/browse/LUCENE-8328 > Project: Lucene - Core > Issue Type: Bug >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Priority: Major > Attachments: LUCENE-8328.patch > > > It's possible for a merge thread to acquire an index reader which is closed > before it can incRef. > {noformat} > Merge stack trace: > at __randomizedtesting.SeedInfo.seed([136983A068AA2F9D]:0) > at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257) > at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) > at > org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) > at > org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) > at > org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) > at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) > at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) > at > org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140) > Refresh stack trace: > at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) > at > org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) > at > org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686) > at > org.apache.lucene.index.ReadersAndUpdates.getLatestReader(ReadersAndUpdates.java:260) > at > org.elasticsearch.index.shard.ElasticsearchMergePolicy.keepFullyDeletedSegment(ElasticsearchMergePolicy.java:143) > at > org.apache.lucene.index.ReadersAndUpdates.keepFullyDeletedSegment(ReadersAndUpdates.java:769) > at org.apache.lucene.index.IndexWriter.isFullyDeleted(IndexWriter.java:5124) > at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3306) > at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:514) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) > at > org.apache.lucene.index.FilterDirectoryReader.doOpenIfChanged(FilterDirectoryReader.java:104) > at > org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140){noformat} > The problem is that `ReadersAndUpdates#getLatestReader` is executed > concurrently without holding lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8328) ReadersAndUpdates#getLatestReader should execute under lock
[ https://issues.apache.org/jira/browse/LUCENE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nhat Nguyen updated LUCENE-8328: Description: It's possible for a merge thread to acquire an index reader which is closed before it can incRef. {noformat} Merge stack trace: at __randomizedtesting.SeedInfo.seed([136983A068AA2F9D]:0) at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257) at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140) Refresh stack trace: at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) at org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) at org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686) at org.apache.lucene.index.ReadersAndUpdates.getLatestReader(ReadersAndUpdates.java:260) at org.elasticsearch.index.shard.ElasticsearchMergePolicy.keepFullyDeletedSegment(ElasticsearchMergePolicy.java:143) at org.apache.lucene.index.ReadersAndUpdates.keepFullyDeletedSegment(ReadersAndUpdates.java:769) at org.apache.lucene.index.IndexWriter.isFullyDeleted(IndexWriter.java:5124) at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3306) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:514) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.FilterDirectoryReader.doOpenIfChanged(FilterDirectoryReader.java:104) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140){noformat} The problem is that `ReadersAndUpdates#getLatestReader` is executed concurrently without holding lock. was: It's possible for a merge thread to acquire an index reader which is closed before it can incRef. *Merge stack trace:* {noformat} Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140){noformat} *Refresh stack trace:* {noformat} at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) at org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) at org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686) at org.apache.lucene.index.ReadersAndUpdates.getLatestReader(ReadersAndUpdates.java:260) at org.elasticsearch.index.shard.ElasticsearchMergePolicy.keepFullyDeletedSegment(ElasticsearchMergePolicy.java:143) at org.apache.lucene.index.ReadersAndUpdates.keepFullyDeletedSegment(ReadersAndUpdates.java:769) at org.apache.lucene.index.IndexWriter.isFullyDeleted(IndexWriter.java:5124) at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3306) at org.apache.lucene.index.IndexWriter.getReader
[jira] [Updated] (LUCENE-8328) ReadersAndUpdates#getLatestReader should execute under lock
[ https://issues.apache.org/jira/browse/LUCENE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nhat Nguyen updated LUCENE-8328: Attachment: LUCENE-8328.patch > ReadersAndUpdates#getLatestReader should execute under lock > --- > > Key: LUCENE-8328 > URL: https://issues.apache.org/jira/browse/LUCENE-8328 > Project: Lucene - Core > Issue Type: Bug >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Priority: Major > Attachments: LUCENE-8328.patch > > > It's possible for a merge thread to acquire an index reader which is closed > before it can incRef. > *Merge stack trace:* > {noformat} > Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader > is closed at > org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) at > org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) > at > org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) at > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) at > org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) > at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) at > org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) at > org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) > at > org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140){noformat} > > *Refresh stack trace:* > {noformat} > at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) > at > org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) > at > org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686) > at > org.apache.lucene.index.ReadersAndUpdates.getLatestReader(ReadersAndUpdates.java:260) > at > org.elasticsearch.index.shard.ElasticsearchMergePolicy.keepFullyDeletedSegment(ElasticsearchMergePolicy.java:143) > at > org.apache.lucene.index.ReadersAndUpdates.keepFullyDeletedSegment(ReadersAndUpdates.java:769) > at org.apache.lucene.index.IndexWriter.isFullyDeleted(IndexWriter.java:5124) > at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3306) > at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:514) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) > at > org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) > at > org.apache.lucene.index.FilterDirectoryReader.doOpenIfChanged(FilterDirectoryReader.java:104) > at > org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140){noformat} > > The problem is that `ReadersAndUpdates#getLatestReader` is executed > concurrently without holding lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8328) ReadersAndUpdates#getLatestReader should execute under lock
[ https://issues.apache.org/jira/browse/LUCENE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nhat Nguyen updated LUCENE-8328: Description: It's possible for a merge thread to acquire an index reader which is closed before it can incRef. *Merge stack trace:* {noformat} Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140){noformat} *Refresh stack trace:* {noformat} at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) at org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) at org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686) at org.apache.lucene.index.ReadersAndUpdates.getLatestReader(ReadersAndUpdates.java:260) at org.elasticsearch.index.shard.ElasticsearchMergePolicy.keepFullyDeletedSegment(ElasticsearchMergePolicy.java:143) at org.apache.lucene.index.ReadersAndUpdates.keepFullyDeletedSegment(ReadersAndUpdates.java:769) at org.apache.lucene.index.IndexWriter.isFullyDeleted(IndexWriter.java:5124) at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3306) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:514) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.FilterDirectoryReader.doOpenIfChanged(FilterDirectoryReader.java:104) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140){noformat} The problem is that `ReadersAndUpdates#getLatestReader` is executed concurrently without holding lock. was: It's possible for a merge thread to acquire an index reader which is closed before it can incRef. *Merge stack trace:* {noformat} Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed at __randomizedtesting.SeedInfo.seed([136983A068AA2F9D]:0) at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257) at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140) at {noformat} *Refresh stack trace:* {noformat} at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) at org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) at org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686) at org.apache.lucene.index.ReadersAndUpdates.getLatestReader(ReadersAndUpdates.java:260) at org.elasticsearch.index.shard.ElasticsearchMergePolicy.keepFullyDeletedSegment(ElasticsearchMergePolicy.java:143) at org.apache.lucene.index.ReadersAndUpdates.keepFullyDeletedSegment(ReadersAndUpdates.java:769) at org.apache.lucene.index.IndexWriter.isFullyDeleted(IndexWriter.j
[jira] [Updated] (LUCENE-8328) ReadersAndUpdates#getLatestReader should execute under lock
[ https://issues.apache.org/jira/browse/LUCENE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nhat Nguyen updated LUCENE-8328: Description: It's possible for a merge thread to acquire an index reader which is closed before it can incRef. *Merge stack trace:* {noformat} Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed at __randomizedtesting.SeedInfo.seed([136983A068AA2F9D]:0) at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257) at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140) at {noformat} *Refresh stack trace:* {noformat} at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) at org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) at org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686) at org.apache.lucene.index.ReadersAndUpdates.getLatestReader(ReadersAndUpdates.java:260) at org.elasticsearch.index.shard.ElasticsearchMergePolicy.keepFullyDeletedSegment(ElasticsearchMergePolicy.java:143) at org.apache.lucene.index.ReadersAndUpdates.keepFullyDeletedSegment(ReadersAndUpdates.java:769) at org.apache.lucene.index.IndexWriter.isFullyDeleted(IndexWriter.java:5124) at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3306) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:514) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.FilterDirectoryReader.doOpenIfChanged(FilterDirectoryReader.java:104) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140){noformat} The problem is that `ReadersAndUpdates#getLatestReader` is executed concurrently without holding lock. was: It's possible for a merge thread to acquire an index reader which is closed before it can incRef. *Merge stack trace:* {noformat} Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed at __randomizedtesting.SeedInfo.seed([136983A068AA2F9D]:0) at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257) at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140) at {noformat} *Refresh stack trace:* {noformat} at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) at org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) at org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686) at org.apache.lucene.index.ReadersAndUpdates.getLatestReader(ReadersAndUpdates.java:260) at org.elasticsearch.index.shard.ElasticsearchMergePolicy.keepFullyDeletedSegment(ElasticsearchMergePolicy.java:143) at org.apache.lucene.ind
[jira] [Updated] (LUCENE-8328) ReadersAndUpdates#getLatestReader should execute under lock
[ https://issues.apache.org/jira/browse/LUCENE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nhat Nguyen updated LUCENE-8328: Description: It's possible for a merge thread to acquire an index reader which is closed before it can incRef. *Merge stack trace:* {noformat} Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed at __randomizedtesting.SeedInfo.seed([136983A068AA2F9D]:0) at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257) at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140) at {noformat} *Refresh stack trace:* {noformat} at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) at org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) at org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686) at org.apache.lucene.index.ReadersAndUpdates.getLatestReader(ReadersAndUpdates.java:260) at org.elasticsearch.index.shard.ElasticsearchMergePolicy.keepFullyDeletedSegment(ElasticsearchMergePolicy.java:143) at org.apache.lucene.index.ReadersAndUpdates.keepFullyDeletedSegment(ReadersAndUpdates.java:769) at org.apache.lucene.index.IndexWriter.isFullyDeleted(IndexWriter.java:5124) at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3306) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:514) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.FilterDirectoryReader.doOpenIfChanged(FilterDirectoryReader.java:104) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140){noformat} The problem is that `ReadersAndUpdates#getLatestReader` is executed concurrently without holding lock. was: It's possible for a merge thread to acquire an index reader which is closed before it can incRef. *Merge stack trace:* {noformat} Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed at __randomizedtesting.SeedInfo.seed([136983A068AA2F9D]:0) at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257) at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140) at org.apache.lucene.index.TestSoftDeletesRetentionMergePolicy.lambda$testMergeAndRefreshDeletedSegmentsConcurrently$21(TestSoftDeletesRetentionMergePolicy.java:597) ... 1 more {noformat} *Refresh stack trace:* {noformat} at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) at org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) at org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686) at org.apache.lucene.index.ReadersAndUpdates.getLatestReader
[jira] [Updated] (LUCENE-8328) ReadersAndUpdates#getLatestReader should execute under lock
[ https://issues.apache.org/jira/browse/LUCENE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nhat Nguyen updated LUCENE-8328: Description: It's possible for a merge thread to acquire an index reader which is closed before it can incRef. {noformat} Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed at __randomizedtesting.SeedInfo.seed([136983A068AA2F9D]:0) at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257) at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140) at org.apache.lucene.index.TestSoftDeletesRetentionMergePolicy.lambda$testMergeAndRefreshDeletedSegmentsConcurrently$21(TestSoftDeletesRetentionMergePolicy.java:597) ... 1 more {noformat} {noformat} at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) at org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) at org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686) at org.apache.lucene.index.ReadersAndUpdates.getLatestReader(ReadersAndUpdates.java:260) at org.elasticsearch.index.shard.ElasticsearchMergePolicy.keepFullyDeletedSegment(ElasticsearchMergePolicy.java:143) at org.apache.lucene.index.ReadersAndUpdates.keepFullyDeletedSegment(ReadersAndUpdates.java:769) at org.apache.lucene.index.IndexWriter.isFullyDeleted(IndexWriter.java:5124) at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3306) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:514) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.FilterDirectoryReader.doOpenIfChanged(FilterDirectoryReader.java:104) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140){noformat} The problem is that `ReadersAndUpdates#getLatestReader` is executed concurrently without holding lock. was: It's possible for a merge thread to acquire an index reader which is closed before it can incRef. ** Merge stack trace:* {noformat} Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed at __randomizedtesting.SeedInfo.seed([136983A068AA2F9D]:0) at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257) at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140) at org.apache.lucene.index.TestSoftDeletesRetentionMergePolicy.lambda$testMergeAndRefreshDeletedSegmentsConcurrently$21(TestSoftDeletesRetentionMergePolicy.java:597) ... 1 more {noformat} ** Refresh stack trace:* {noformat} at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) at org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) at org.apache.lucene.in
[jira] [Updated] (LUCENE-8328) ReadersAndUpdates#getLatestReader should execute under lock
[ https://issues.apache.org/jira/browse/LUCENE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nhat Nguyen updated LUCENE-8328: Description: It's possible for a merge thread to acquire an index reader which is closed before it can incRef. *Merge stack trace:* {noformat} Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed at __randomizedtesting.SeedInfo.seed([136983A068AA2F9D]:0) at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257) at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140) at org.apache.lucene.index.TestSoftDeletesRetentionMergePolicy.lambda$testMergeAndRefreshDeletedSegmentsConcurrently$21(TestSoftDeletesRetentionMergePolicy.java:597) ... 1 more {noformat} *Refresh stack trace:* {noformat} at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) at org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) at org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686) at org.apache.lucene.index.ReadersAndUpdates.getLatestReader(ReadersAndUpdates.java:260) at org.elasticsearch.index.shard.ElasticsearchMergePolicy.keepFullyDeletedSegment(ElasticsearchMergePolicy.java:143) at org.apache.lucene.index.ReadersAndUpdates.keepFullyDeletedSegment(ReadersAndUpdates.java:769) at org.apache.lucene.index.IndexWriter.isFullyDeleted(IndexWriter.java:5124) at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3306) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:514) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.FilterDirectoryReader.doOpenIfChanged(FilterDirectoryReader.java:104) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140){noformat} The problem is that `ReadersAndUpdates#getLatestReader` is executed concurrently without holding lock. was: It's possible for a merge thread to acquire an index reader which is closed before it can incRef. {noformat} Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed at __randomizedtesting.SeedInfo.seed([136983A068AA2F9D]:0) at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257) at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198) at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542) at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263) at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253) at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140) at org.apache.lucene.index.TestSoftDeletesRetentionMergePolicy.lambda$testMergeAndRefreshDeletedSegmentsConcurrently$21(TestSoftDeletesRetentionMergePolicy.java:597) ... 1 more {noformat} {noformat} at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238) at org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675) at org.apache.lucene.index.Reader