Philippe Lantin created FLUME-2866:
--------------------------------------
Summary: Add fileTimeMinOffsetSeconds property to Spooling
Directory Source
Key: FLUME-2866
URL: https://issues.apache.org/jira/browse/FLUME-2866
Project: Flume
Issue Type: New Feature
Components: Sinks+Sources
Reporter: Philippe Lantin
Priority: Minor
When using a spooling directory source, it would be useful to have the ability
to specify that files must have a last modified timestamp that is off by a
configurable amount of seconds, either in the future or the past.
For example, if I copy a large file to the spooling directory and it takes
several minutes to copy, I do not want my file to started being processed
before it is completed. A practical way to do this is by looking at the last
modified timestamp: files that are being transferred will update this timestamp.
In many filesystems, it is possible for clients to set the time in the future,
though this is usually done after a file has been completed transferred. For
example "cp -p" on linux.
I propose a new property for the Spooling Directory Source:
fileTimeMinOffsetSeconds. The default would be "0", preserving current behavior.
If fileTimeMinOffsetSeconds=60, files will only be picked up if the last
modified time is +/- 60 seconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
