[
https://issues.apache.org/jira/browse/KUDU-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238049#comment-15238049
]
Mike Percy commented on KUDU-1414:
----------------------------------
I guess it's worth adding that there are some corruptions that are still
undetectable under the above scheme, although at some point one could argue
that probability is on our side. For example, a post-fsync corruption with a
streak of zeros at the end of a record is fully indistinguishable from a
partial write.
> Corrupting multiple log entries at the end of a WAL file may go undetected
> --------------------------------------------------------------------------
>
> Key: KUDU-1414
> URL: https://issues.apache.org/jira/browse/KUDU-1414
> Project: Kudu
> Issue Type: Bug
> Components: log
> Affects Versions: 0.8.0
> Reporter: Mike Percy
>
> While looking at KUDU-1377, I investigated how we are handling WAL truncation
> when corruption is detected. The way the code is written today, a trailing
> series of corrupt log entries are truncated with only a log warning message.
> I'll post a unit test demonstrating this behavior.
> One way to get around this is to ensure that we only accept zeros following a
> truncated record, instead of just bad records, in order to consider it a
> partially-written record that we can safely truncate. We would have to
> maintain this invariant when preallocating space and truncating partial
> records before continuing to write.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
