[
https://issues.apache.org/jira/browse/KAFKA-12331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jose Armando Garcia Sancio updated KAFKA-12331:
-----------------------------------------------
Description:
KafkaMetadataLog's appendAsLeader expects the base offset to match the LEO.
This is enforced when KafkaRaftClient uses the BatchAccumulator to create
batches. When creating the control batch for the LeaderChangeMessage the
KafkaRaftClient doesn't use the BatchAccumulator and instead creates the batch
with the default base offset of 0.
This causes the validation in KafkaMetadataLog to fail with the following
exception:
{code:java}
kafka.common.UnexpectedAppendOffsetException: Unexpected offset in append to
@metadata-0. First offset 0 is less than the next offset 5. First 10 offsets in
append: ArrayBuffer(0), last offset in append: 0. Log start offset = 0
at kafka.log.Log.append(Log.scala:1217)
at kafka.log.Log.appendAsLeader(Log.scala:1092)
at kafka.raft.KafkaMetadataLog.appendAsLeader(KafkaMetadataLog.scala:92)
at
org.apache.kafka.raft.KafkaRaftClient.appendAsLeader(KafkaRaftClient.java:1158)
at
org.apache.kafka.raft.KafkaRaftClient.appendLeaderChangeMessage(KafkaRaftClient.java:449)
at
org.apache.kafka.raft.KafkaRaftClient.onBecomeLeader(KafkaRaftClient.java:409)
at
org.apache.kafka.raft.KafkaRaftClient.maybeTransitionToLeader(KafkaRaftClient.java:463)
at
org.apache.kafka.raft.KafkaRaftClient.handleVoteResponse(KafkaRaftClient.java:663)
at
org.apache.kafka.raft.KafkaRaftClient.handleResponse(KafkaRaftClient.java:1530)
at
org.apache.kafka.raft.KafkaRaftClient.handleInboundMessage(KafkaRaftClient.java:1652)
at org.apache.kafka.raft.KafkaRaftClient.poll(KafkaRaftClient.java:2183)
at kafka.raft.KafkaRaftManager$RaftIoThread.doWork(RaftManager.scala:52)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
{code}
We should make the following changes:
# Fix MockLog to perform similar validation as KafkaMetadataLog::appendAsLeader
# Use the LEO when creating the control batch for the LeaderChangedMessage
was:
KafkaMetadataLog's appendAsLeader expects the base offset to match the LEO.
This is enforced when KafkaRaftClient uses the BatchAccumulator to create
batches. When creating the control batch for the LeaderChangeMessage the
KafkaRaftClient doesn't use the BatchAccumulator and instead creates the batch
with the default base offset of 0.
This causes the validation in KafkaMetadataLog to fail with the following
exception:
{code:java}
kafka.common.UnexpectedAppendOffsetException: Unexpected offset in append to
@metadata-0. First offset 0 is less than the next offset 5. First 10 offsets in
append: ArrayBuffer(0), last offset in append: 0. Log start offset = 0
at kafka.log.Log.append(Log.scala:1217)
at kafka.log.Log.appendAsLeader(Log.scala:1092)
at kafka.raft.KafkaMetadataLog.appendAsLeader(KafkaMetadataLog.scala:92)
at
org.apache.kafka.raft.KafkaRaftClient.appendAsLeader(KafkaRaftClient.java:1158)
at
org.apache.kafka.raft.KafkaRaftClient.appendLeaderChangeMessage(KafkaRaftClient.java:449)
at
org.apache.kafka.raft.KafkaRaftClient.onBecomeLeader(KafkaRaftClient.java:409)
at
org.apache.kafka.raft.KafkaRaftClient.maybeTransitionToLeader(KafkaRaftClient.java:463)
at
org.apache.kafka.raft.KafkaRaftClient.handleVoteResponse(KafkaRaftClient.java:663)
at
org.apache.kafka.raft.KafkaRaftClient.handleResponse(KafkaRaftClient.java:1530)
at
org.apache.kafka.raft.KafkaRaftClient.handleInboundMessage(KafkaRaftClient.java:1652)
at org.apache.kafka.raft.KafkaRaftClient.poll(KafkaRaftClient.java:2183)
at kafka.raft.KafkaRaftManager$RaftIoThread.doWork(RaftManager.scala:52)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
{code}
> KafkaRaftClient should use the LEO when appending LeaderChangeMessage
> ---------------------------------------------------------------------
>
> Key: KAFKA-12331
> URL: https://issues.apache.org/jira/browse/KAFKA-12331
> Project: Kafka
> Issue Type: Sub-task
> Components: replication
> Reporter: Jose Armando Garcia Sancio
> Priority: Major
>
> KafkaMetadataLog's appendAsLeader expects the base offset to match the LEO.
> This is enforced when KafkaRaftClient uses the BatchAccumulator to create
> batches. When creating the control batch for the LeaderChangeMessage the
> KafkaRaftClient doesn't use the BatchAccumulator and instead creates the
> batch with the default base offset of 0.
> This causes the validation in KafkaMetadataLog to fail with the following
> exception:
> {code:java}
> kafka.common.UnexpectedAppendOffsetException: Unexpected offset in append to
> @metadata-0. First offset 0 is less than the next offset 5. First 10 offsets
> in append: ArrayBuffer(0), last offset in append: 0. Log start offset = 0
> at kafka.log.Log.append(Log.scala:1217)
> at kafka.log.Log.appendAsLeader(Log.scala:1092)
> at kafka.raft.KafkaMetadataLog.appendAsLeader(KafkaMetadataLog.scala:92)
> at
> org.apache.kafka.raft.KafkaRaftClient.appendAsLeader(KafkaRaftClient.java:1158)
> at
> org.apache.kafka.raft.KafkaRaftClient.appendLeaderChangeMessage(KafkaRaftClient.java:449)
> at
> org.apache.kafka.raft.KafkaRaftClient.onBecomeLeader(KafkaRaftClient.java:409)
> at
> org.apache.kafka.raft.KafkaRaftClient.maybeTransitionToLeader(KafkaRaftClient.java:463)
> at
> org.apache.kafka.raft.KafkaRaftClient.handleVoteResponse(KafkaRaftClient.java:663)
> at
> org.apache.kafka.raft.KafkaRaftClient.handleResponse(KafkaRaftClient.java:1530)
> at
> org.apache.kafka.raft.KafkaRaftClient.handleInboundMessage(KafkaRaftClient.java:1652)
> at org.apache.kafka.raft.KafkaRaftClient.poll(KafkaRaftClient.java:2183)
> at kafka.raft.KafkaRaftManager$RaftIoThread.doWork(RaftManager.scala:52)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
> {code}
> We should make the following changes:
> # Fix MockLog to perform similar validation as
> KafkaMetadataLog::appendAsLeader
> # Use the LEO when creating the control batch for the LeaderChangedMessage
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
