Ming Zhekai created FLUME-3341:
----------------------------------
Summary: Taildir source may cause file handle leak and data
duplication
Key: FLUME-3341
URL: https://issues.apache.org/jira/browse/FLUME-3341
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: 1.9.0, 1.8.0
Reporter: Ming Zhekai
Fix For: 1.8.0
As is described in Flume-3342, renaming a file may cause data duplication. But
moreover, if the original file was opened by flume before and not closed yet,
flume reopens it while not freeing its original file handles. And then I delete
the new file, but after the idle time, flume just closes the new file handle
and forgets to close its original file handles, leading to a handle leak. It
can be found by "lsof | grep "deleted".
This bug is triggered by using Log4j to roll log files. To avoid losing data
when rolling, I use regex to include both current log file and old log files in
the taildir path.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
