[ https://issues.apache.org/jira/browse/NIFI-631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026137#comment-15026137 ]
Joseph Witt commented on NIFI-631: ---------------------------------- "wired ListFile and FetchFile together with defaults, they did not work, which I wasn't expecting" This is a reasonable expectation. Mark/Joe: Is this something we can tweak prior to merge? Tony's expectation seems right on. > Create ListFile and FetchFile processors > ---------------------------------------- > > Key: NIFI-631 > URL: https://issues.apache.org/jira/browse/NIFI-631 > Project: Apache NiFi > Issue Type: Improvement > Reporter: Mark Payne > Assignee: Joe Skora > Fix For: 0.4.0 > > Attachments: > 0001-NIFI-631-Initial-implementation-of-FetchFile-process.patch, > NIFI-631.000.patch, Test_ListFile_FetchFile.xml > > > This pair of Processors will provide several benefits over the existing > GetFile processor: > 1. Currently, GetFile will continually pull the same files if the "Keep > Source File" property is set to true. There is no way to pull the file and > leave it in the directory without continually pulling the same file. We could > implement state here, but it would either be a huge amount of state to > remember everything pulled or it would have to always pull the oldest file > first so that we can maintain just the Last Modified Date of the last file > pulled plus all files with the same Last Modified Date that have already been > pulled. > 2. If pulling from a network attached storage such as NFS, this would allow a > single processor to run ListFiles and then distribute those FlowFiles to the > cluster so that the cluster can share the work of pulling the data. > 3. There are use cases when we may want to pull a specific file (for example, > in conjunction with ProcessHttpRequest/ProcessHttpResponse) rather than just > pull all files in a directory. GetFile does not support this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)