[
https://issues.apache.org/jira/browse/KAFKA-310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235636#comment-13235636
]
Jun Rao commented on KAFKA-310:
-------------------------------
ByteBufferMessageSet.validBytes currently makes a deep iteration of all
messages, which means that we need to decompress messages. To avoid this
overhead, we should change ByteBufferMessageSet.validBytes to use a shallow
iterator.
> Incomplete message set validation checks in kafka.log.Log's append API can
> corrupt on disk log
> ----------------------------------------------------------------------------------------------
>
> Key: KAFKA-310
> URL: https://issues.apache.org/jira/browse/KAFKA-310
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.7
> Reporter: Neha Narkhede
> Priority: Critical
> Attachments: kafka-310.patch
>
>
> The behavior of the ByteBufferMessageSet's iterator is to ignore and return
> false if some trailing bytes are found that cannot be de serialized into a
> Kafka message. The append API in Log, iterates through a ByteBufferMessageSet
> and validates the checksum of each message. Though, while appending data to
> the log, it just uses the underlying ByteBuffer that forms the
> ByteBufferMessageSet. Now, due to some bug, if the ByteBuffer has some
> trailing data, that will get appended to the on-disk log too. This can cause
> corruption of the log.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
