[
https://issues.apache.org/jira/browse/KAFKA-16185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lianet Magrans updated KAFKA-16185:
-----------------------------------
Priority: Critical (was: Blocker)
> Fix client reconciliation of same assignment received in different epochs
> --------------------------------------------------------------------------
>
> Key: KAFKA-16185
> URL: https://issues.apache.org/jira/browse/KAFKA-16185
> Project: Kafka
> Issue Type: Sub-task
> Components: clients, consumer
> Reporter: Lianet Magrans
> Assignee: Lianet Magrans
> Priority: Critical
> Labels: client-transitions-issues, kip-848-client-support
> Fix For: 3.8.0
>
>
> Currently, the intention in the client state machine is that the client
> always reconciles whatever it has pending and sends an ack for it, but in
> cases where the same assignment is received in different epochs this does not
> work as expected.
> 1 - Client might get stuck JOINING/RECONCILING, with a pending reconciliation
> (delayed), and it receives the same assignment, but in a new epoch (ex. after
> being FENCED). First time it receives the assignment it takes no action, as
> it already has it as pending to reconcile, but when the reconciliation
> completes it discards the result because the epoch changed. And this is
> wrong. Note that after sending the assignment with the new epoch one time,
> the broker continues to send null assignments.
> Here is a sample sequence leading to the client stuck JOINING:
> - client joins, epoch 0
> - client receives assignment tp1, stuck RECONCILING, epoch 1
> - member gets FENCED on the coord, coord bumps epoch to 2
> - client tries to rejoin (JOINING), epoch 0 provided by the client
> - new member added to the group (group epoch bumped to 3), client receives
> same assignment that is currently trying to reconcile (tp1), but with epoch 3
> - previous reconciliation completes, but will discard the result because it
> will notice that the memberHasRejoined (memberEpochOnReconciliationStart !=
> memberEpoch). Client is stuck JOINING, with the server sending null target
> assignment because it hasn't changed since the last one sent (tp1)
> We should end up with a test similar to the existing
> #testDelayedReconciliationResultDiscardedIfMemberRejoins but with the case
> that the member receives the same assignment after being fenced and rejoining
> 2 - Client is not sending ack back to the broker in cases where it finishes a
> reconciliation for the same assignment that it sent in the last HB (builder
> will not include the assignment). Following sequence:
> - client owns T1-1 (last HB sent included ack for T1-1)
> - client receives [T1-1, T2-1] and start reconciling
> - client receives T1-1 (meaning T2-1 needs to be revoked)
> - ongoing reconciliation for [T1-1, T2-1] fails so ack never sent for it
> - next reconciliation starts for T1-1 and completes, but ack is not sent
> because the builder sees it's the same it sent on the last HB, leaving the
> broker waiting for an ack that won't arrive.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
