[
https://issues.apache.org/jira/browse/NIFI-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16909231#comment-16909231
]
Mark Payne commented on NIFI-6559:
----------------------------------
I don't have any solution off the top of my head. The question I would have is
what happened to the file? The way that the logic behaves, in order for the
overflow file to be considered part of the repo, the entire file must be
entirely written and then fsync'ed to ensure that all contents have been
flushed to disk. Only then do we update the FlowFile Repository to point to it
because we know the file is in a good state. If arbitrary parts of the store's
state/data are manually (or otherwise) deleted outside of the application, I
would not expect it to start up and keep functioning, just as I would not
expect a database to function properly if a user started deleting some of its
files.
> FlowFile Repo Journal Recovery Should not Fail if External Overflow Files are
> Missing
> -------------------------------------------------------------------------------------
>
> Key: NIFI-6559
> URL: https://issues.apache.org/jira/browse/NIFI-6559
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Reporter: Peter Wicks
> Assignee: Peter Wicks
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When NiFi is journaling the FlowFile repository changes to disk it sometimes
> writes Overflow files if it exceeds a certain memory threshold.
> These files are tracked inside of the *.journal files as External File
> References. If one of these external file references is deleted or lost the
> entire journal fails to recover.
> Instead, I feel this should work more like FlowFile's that lose their queue,
> or Content in the Content Repository that has lost it's FlowFile. Log it,
> and move on.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
