[
https://issues.apache.org/jira/browse/FLUME-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Zhekai reopened FLUME-3341:
--------------------------------
> Taildir source may cause file handle leak and data duplication
> --------------------------------------------------------------
>
> Key: FLUME-3341
> URL: https://issues.apache.org/jira/browse/FLUME-3341
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: 1.8.0, 1.9.0
> Reporter: Ming Zhekai
> Priority: Major
> Fix For: 1.8.0
>
> Attachments: FLUME-3341.patch
>
> Original Estimate: 120h
> Time Spent: 10m
> Remaining Estimate: 119h 50m
>
> As is described in Flume-3342, renaming a file may cause data duplication.
> But moreover, if the original file was opened by flume before and not closed
> yet, flume reopens it while not freeing its original file handles. And then I
> delete the new file, but after the idle time, flume just closes the new file
> handle and forgets to close its original file handles, leading to a handle
> leak. It can be found by "lsof | grep "deleted".
> This bug is triggered by using Log4j to roll log files. To avoid losing data
> when rolling, I use regex to include both current log file and old log files
> in the taildir path.
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
