nileshkumar3 opened a new pull request, #23377:
URL: https://github.com/apache/kafka/pull/23377
### Problem
`test_static_consumer_bounce_with_eager_assignment` verifies that partitions
are not
re-assigned when a static member is bounced, but only for eager assignment.
As the
test's own docstring notes, it "relies on that eager behaviour": under eager
rebalancing every member revokes everything on any rebalance, so the
revocation count
of the member kept alive distinguishes "a global rebalance happened" from
"it did not".
That signal does not exist without eager rebalancing, where a member only
revokes the
partitions that actually move. The test therefore cannot simply be
parametrized over
`CooperativeStickyAssignor` or the new consumer group protocol — its
dynamic-membership
branch asserts `num_revokes_after_bounce != 0`, which is not a valid
expectation once
rebalancing is incremental. This left static membership under the KIP-848
consumer
protocol without system test coverage for assignment stability.
### Change
Adds `test_static_consumer_bounce_preserves_assignment_when_not_eager`,
which asserts
on the assignment itself rather than on revocation counts:
- the member kept alive revokes nothing across the bounces, so no partition
is taken
from a member that never left, and
- the partition assignment of the whole group is identical before and after,
so the
bounced static members get their own partitions back.
Covered configurations: classic protocol with `CooperativeStickyAssignor`,
and the
consumer group protocol with both the `range` and `uniform` remote
assignors, each
across `all` and `rolling` bounce modes.
The test uses a new 6-partition topic. `OffsetValidationTest.TOPIC` has a
single
partition shared by three consumers, so two of the three members own
nothing, which
makes both revocation counts and assignment comparisons meaningless for this
check.
`assert not consumer.is_eager()` guards the intent, so a future change to
assignor
defaults cannot silently reduce this to a duplicate of the eager test.
### Testing
Verified that all six parametrized cases collect with the expected injected
arguments
via `ducktape --collect-only`. I do not have a ducktape cluster available to
execute
the run — if someone can trigger it, or if there is a preferred way to get
system test
results on a PR, I am happy to follow up.
### Questions
1. Should this also cover dynamic membership? I deliberately left it out
because the
expected behaviour is not obvious to me under incremental rebalancing:
when a
dynamic member is bounced its partitions do have to be covered while it
is down, so
the surviving member may legitimately gain and then lose partitions. I
did not want
to encode a guess as an assertion. Happy to add it if you can confirm the
intended
expectation.
2. Is asserting equality of the *whole* group assignment too strict for the
consumer
protocol? A server-side assignor could in principle reshuffle for reasons
unrelated
to the bounce, which would make this flaky rather than wrong. The
narrower assertion
is to check only that the kept-alive member's partitions are untouched. I
went with
the stronger one since it is what the ticket describes, but I will weaken
it if you
would rather trade coverage for stability.
commit messgae
KAFKA-16628: Add system test for static consumer bounce with non-eager
assignment
test_static_consumer_bounce_with_eager_assignment verifies that partitions
are
not re-assigned when a static member is bounced, but it can only do so by
counting revocations on the member kept alive: under eager rebalancing every
member revokes everything on any rebalance. That signal does not exist with
incremental rebalancing, where a member only revokes the partitions that
actually move, so the test cannot simply be parametrized over the cooperative
sticky assignor or the consumer group protocol.
Add a test that asserts on the assignment instead: the member kept alive
revokes nothing, and the partition assignment of the whole group is identical
before and after the bounces. Covers the classic protocol with
CooperativeStickyAssignor, and the consumer group protocol with the range and
uniform remote assignors, across both bounce modes.
The test uses a new 6-partition topic because OffsetValidationTest.TOPIC has
a
single partition shared by three consumers, leaving two members owning
nothing.
Generated-by: Cursor (Claude Opus 4.5)
Per CONTRIBUTING.md, the AI trailer is expected on AI-assisted changes —
drop that last line if you'd rather attribute it differently.
modified: tests/kafkatest/tests/client/consumer_test.py
Untracked files:
(use "git add <file>..." to include in what will be committed)
.gradle-local/
clients/src/main/java/org/apache/kafka/clients/consumer/internals/test
clients/src/main/java/org/apache/kafka/clients/consumer/internals/test-file
docs/EAB Research Paper-Dr. Veena Kumari.docx - Google Docs.html
examples/.gradle-local/
kafka-8073-loop-results.txt
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
