Mark Payne created NIFI-8210:
--------------------------------
Summary: If Provenance Repository ages off a Lucene Index while
it's being searched, can lead to index corruption
Key: NIFI-8210
URL: https://issues.apache.org/jira/browse/NIFI-8210
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
When all events have been aged off from a Provenance Lucene Index, the lucene
index can be deleted. However, before deleting it, the repository needs to
ensure that there are no active readers or writers. Currently, the repository
only ensures that there are no active writers. As a result, we can have an
active Index Reader/Index Searcher and still delete the index directory. When
this happens, we end up with an error similar to:
{code:java}
org.apache.lucene.index.IndexNotFoundException: no segments* file found in
MMapDirectory@/Users/mpayne/devel/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/target/6e442c6d-14f3-48bb-9500-534c136942aa
lockFactory=org.apache.lucene.store.NativeFSLockFactory@161b062a: files:
[]org.apache.lucene.index.IndexNotFoundException: no segments* file found in
MMapDirectory@/Users/mpayne/devel/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/target/6e442c6d-14f3-48bb-9500-534c136942aa
lockFactory=org.apache.lucene.store.NativeFSLockFactory@161b062a: files: []
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:675)
at
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:81)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63) at
org.apache.nifi.provenance.lucene.StandardIndexManager.borrowIndexSearcher(StandardIndexManager.java:128)
{code}
We need to ensure that we account for any Readers/Searchers that are active also
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
