[
https://issues.apache.org/jira/browse/FLUME-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasad Mujumdar resolved FLUME-883.
-----------------------------------
Resolution: Fixed
Patch committed to trunk
> Flume E2E sink could send incorrect ACKs if there are HDFS file close errors
> -----------------------------------------------------------------------------
>
> Key: FLUME-883
> URL: https://issues.apache.org/jira/browse/FLUME-883
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v0.9.4
> Reporter: Prasad Mujumdar
> Assignee: Prasad Mujumdar
> Fix For: v0.9.5
>
> Attachments: Flume-883.patch.1
>
>
> The E2E collector sink saves the batch tags as the batches are passed to the
> downstream sinks. The ACKs are flushed when the roller close the file.
> Currently for the HDFS sink, the close is the only operation that guarantees
> that data is safely stored. Hence the acks are sent on close. If for some
> reason, the writes fail then we don't send the acks assuming the data is
> lost. The E2E mechanism then resends the data.
> The problem is that if the close fails then we don't clear the accumulated
> acks for that current rolltag. Hence its possible that the next successful
> roll could send those acks and hence the batch will not be resent.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
