[
https://issues.apache.org/jira/browse/NIFI-994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14978745#comment-14978745
]
Joe Skora commented on NIFI-994:
--------------------------------
In general, I don't think the contract of "tail" guarantees every bit of
content will be seen. The GNU Tail source mentions in [this
comment|http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=src/tail.c;h=f916d7460395f0cee52c592bc3d160ac94697e73;hb=HEAD#l1199]
that if the file size shrinks tail will restart from the beginning, but if the
file is truncated and regrows past the last size check it appears that tail
will not detect the change and only return content beyond the last size check.
I share the concerns about using checksums, even though I brought them up.
Logs and such are highly repetitive, which could be a problem for the "last N
bytes" approach unless the checksum window size is large enough to cover a
typical line or record length. It would be great to be able to set the windows
size and have an option for a 0 byte windows size that altogether eliminates
the checksum processing.
Regards,
Joe
> Processor to tail files
> -----------------------
>
> Key: NIFI-994
> URL: https://issues.apache.org/jira/browse/NIFI-994
> Project: Apache NiFi
> Issue Type: New Feature
> Affects Versions: 0.4.0
> Reporter: Joseph Percivall
> Assignee: Mark Payne
> Fix For: 0.4.0
>
> Attachments: 0001-NIFI-994-Initial-import-of-TailFile.patch
>
>
> It's a very common data ingest situation to want to input text into the
> system by "tailing" a file, most commonly log files. Currently we don't have
> an easy way to do this.
> A simple processor to tail a file would benefit many users. There would need
> to be an option to not just tail a file but pick up where the processor left
> off if it is interrupted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
