[
https://issues.apache.org/jira/browse/FLUME-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13914246#comment-13914246
]
Gopinathan A commented on FLUME-2331:
-------------------------------------
Any error log saying "Hit max consecutive under-replication rotations" is there
in ur log file?
> Large TMP files created and never closed
> ----------------------------------------
>
> Key: FLUME-2331
> URL: https://issues.apache.org/jira/browse/FLUME-2331
> Project: Flume
> Issue Type: Bug
> Components: File Channel
> Affects Versions: v1.4.0
> Reporter: Krishna Kumar
> Attachments: flumelog.zip
>
>
> We are currently writing files to Hadoop partitioned by year, month, day via
> Flume. File rollovers are done every 5 minutes. Recently, we noticed that
> this file rollover stops happening sometime during the day and that further
> data is written to an open TMP file. Because there are no further file
> rollovers, this TMP file becomes very large. At the end of the day, the TMP
> is not closed also and Flume goes to the next day, creating new files.
> We use a "." prefix to prevent Hive from complaining of the open TMP file.
> Because of this issue where the TMP file is never closed, the file remains
> hidden to Hive even after the day ends.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
