[
https://issues.apache.org/jira/browse/FLUME-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840421#comment-13840421
]
Brock Noland commented on FLUME-2181:
-------------------------------------
1) the close won't occur if the process is killed with SIGKILL so we need to
flush explicitly.
2)
bq. In reality, all takes past that event is really not valid
Are we assuming that the disk will be written to disk sequentially? I don't
believe that will always be the case. For example without any fsyncs the last
half of a file could be written to disk before the first half. Thus it's
possible the first half of the file is corrupt but no the second half. How do
we handle this case?
Additionally it's possible that the corruption occurs inside an event. That is
the event header is correct and the next event header is correct, but the
inside of the event is all null's. In this case we will be sending invalid data
downstream.
bq we can check for this by simply checking the length of the file before doing
a take
The file is pre-allocated so I don't follow how this will work?
> Optionally disable File Channel fsyncs
> ---------------------------------------
>
> Key: FLUME-2181
> URL: https://issues.apache.org/jira/browse/FLUME-2181
> Project: Flume
> Issue Type: Improvement
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Attachments: FLUME-2181.patch
>
>
> This will give File Channel performance a big boost, at the cost of possible
> data loss if a crash happens between checkpoints.
> Also we should make it configurable, with default to false. If the user does
> not mind slight inconsistencies, this feature can be explicitly enabled
> through configuration. So if it is not configured, then the behavior will be
> exactly as it is now.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
