[
https://issues.apache.org/jira/browse/FLUME-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840372#comment-13840372
]
Hari Shreedharan commented on FLUME-2181:
-----------------------------------------
Thanks for the feedback Brock.
1. Yes, we must at least flush, but we do actually fsync during a close anyway.
I will add a null check for syncExecutor.
2. When the channel is corrupt (there is a partial event in the file or like I
mentioned above in case 2 - there is an fsync-ed take for a not yet fsync-ed
put), we basically need to ignore the take. In reality, all takes past that
event is really not valid - we can check for this by simply checking the length
of the file before doing a take - though this won't handle partial events. For
partial events, we actually will need to read the event and have a separate
exception to show that we read in a partial event. Does that make approach
sound good? Basically, skip partial or non-existent events and log it - make it
as efficient as we can.
> Optionally disable File Channel fsyncs
> ---------------------------------------
>
> Key: FLUME-2181
> URL: https://issues.apache.org/jira/browse/FLUME-2181
> Project: Flume
> Issue Type: Improvement
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Attachments: FLUME-2181.patch
>
>
> This will give File Channel performance a big boost, at the cost of possible
> data loss if a crash happens between checkpoints.
> Also we should make it configurable, with default to false. If the user does
> not mind slight inconsistencies, this feature can be explicitly enabled
> through configuration. So if it is not configured, then the behavior will be
> exactly as it is now.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
