[
https://issues.apache.org/jira/browse/KAFKA-804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602414#comment-13602414
]
Jay Kreps commented on KAFKA-804:
---------------------------------
I am not sure about this change. There are two choices to add to the index: the
offset of the wrapper message which is the offset of the last message in the
compressed set or the offset of the first message in the compressed set. Either
of these are correct since the index entry needs to be a lower bound on the
position, so this is not a bug. I do agree that we should ideally produce
the same index entries on the leader and follower, but I am not sure if we
should choose the first or last message. The problem with choosing the first
message is that the integrity of the index can only be validated using deep
iteration on the log and the index then points to a message which doesn't have
the index entry given in the index which is very odd.
This code looks very confusing to me. I can't tell if this change is making it
worse or the code is already just very confusing. I recommend we leave it as is
on 0.8 and fix it on trunk where that code has been cleaned up. Let's also
figure out the pros and cons of which offset the index should contain.
> Incorrect index in the log of a follower
> ----------------------------------------
>
> Key: KAFKA-804
> URL: https://issues.apache.org/jira/browse/KAFKA-804
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Assignee: Jun Rao
> Priority: Blocker
> Attachments: kafka-804.patch
>
>
> In the follower, in log.append(), we use the offset of the 1st compressed
> message as the offset for appending index entries. This means that the index
> is pointing to an earlier position in the file than it should. Instead, it
> should use the offset of the 1st uncompressed message for appending index.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
