[
https://issues.apache.org/jira/browse/FLUME-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286245#comment-13286245
]
Juhani Connolly commented on FLUME-1238:
----------------------------------------
I just ran into something similar to this too. On our test setup we're naming
files by the hour. The last rolled file for each hour isn't getting closed
properly, with the name still being xyz-timestamp.tmp(I guess it eventually
will once we hit a file limit? I think there was some code floating about to
limit the number of open files, closing the oldest as necessary)
Any thoughts on how to decide when to roll? Add a configuration for a
fileroller, allowing configuration for how long it waits till closing inactive
files?
> Support active rolling of files created by HDFS Event Sink
> ----------------------------------------------------------
>
> Key: FLUME-1238
> URL: https://issues.apache.org/jira/browse/FLUME-1238
> Project: Flume
> Issue Type: Improvement
> Reporter: Arvind Prabhakar
>
> The HDFS Event Sink uses lazy rolling for closing files that are being
> written to. This results in many files being open for longer than their
> expected roll-interval if they are not actively written to and can even last
> in open state until the sink shuts down.
> It will be preferable to have these files roll proactively rather than
> waiting for a write to trigger the roll.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
