Sean Quah created KAFKA-21582:
---------------------------------
Summary: Upgrading a classic group during a rebalance via static
member replacement does not trigger a new target assignment
Key: KAFKA-21582
URL: https://issues.apache.org/jira/browse/KAFKA-21582
Project: Kafka
Issue Type: Bug
Components: group-coordinator
Affects Versions: 4.0.0
Reporter: Sean Quah
Assignee: Sean Quah
{{ConsumerGroup#fromClassicGroup}} sets the group epoch and the target
assignment epoch to the classic group's generation, and it takes the previous
generation's assignments as the target assignment. When the classic group is in
PreparingRebalance, it is waiting for a new assignment, so these values are
stale. The upgrade is supposed to get a fresh assignment from the epoch bump
caused by the joining member. When a consumer-protocol static member replaces a
classic static member with the same subscription, nothing bumps the epoch, and
the coordinator treats the stale target assignment as up to date.
*Steps to reproduce*
# Set {{group.consumer.migration.policy}} to {{{}upgrade{}}}, and set
{{group.initial.rebalance.delay.ms}} high enough that the classic group's
initial rebalance doesn't complete during the test.
# A classic static member with {{group.instance.id}} X joins a new group G.
The group goes to PreparingRebalance at generation 0 and the JoinGroup stays
pending.
# Before the initial rebalance completes, a consumer-protocol member with a
new member id sends ConsumerGroupHeartbeat to G with member epoch 0,
{{instanceId}} X, and the same subscription as the classic member.
# The coordinator upgrades G to a consumer group at group epoch 0 and target
assignment epoch 0. The consumer member replaces the classic static member and
is reset to epoch 0.
# The subscription is unchanged, so the group epoch isn't bumped. The
coordinator considers the target assignment at epoch 0 up to date, and
reconciles the new member to it.
# The heartbeat response returns member epoch 0.
*Expected:* the upgrade triggers a new target assignment, and the member gets
an epoch greater than 0. Epoch 0 is reserved for members joining a group.
When the group has completed at least one generation, the same sequence returns
a valid epoch, but the members keep the previous generation's target
assignment, which may be empty for members that joined during the rebalance.
Partitions stay unassigned or keep their old owners until an unrelated epoch
bump.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)