[
https://issues.apache.org/jira/browse/CHUKWA-369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739576#action_12739576
]
Ari Rabkin commented on CHUKWA-369:
-----------------------------------
Eric --
The difficulty is that with HDFS, there's no easy way to know if data has
committed: it can happen that a call to write() succeeds, but the collector
then crashes before the data left the process. Flush in 0.21 may help, but I'm
not sure we want to flush on every write. The purpose of the proposed
mechanism is to decouple commit-detection from the collector, and let the Agent
determine on its own if data committed.
> proposed reliability mechanism
> ------------------------------
>
> Key: CHUKWA-369
> URL: https://issues.apache.org/jira/browse/CHUKWA-369
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: data collection
> Affects Versions: 0.3.0
> Reporter: Ari Rabkin
> Fix For: 0.3.0
>
>
> We like to say that Chukwa is a system for reliable log collection. It isn't,
> quite, since we don't handle collector crashes. Here's a proposed
> reliability mechanism.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
