sjhajharia opened a new pull request, #22786:
URL: https://github.com/apache/kafka/pull/22786
This adds the first system-test coverage for share group dead-letter queues
(KIP-1191, part of KAFKA-19469), plus the client-side tooling changes needed to
drive it — there was previously no way for any ducktape test client to
explicitly REJECT or RELEASE a share-consumed record; VerifiableShareConsumer
only ever implicitly accepted.
`VerifiableShareConsumer.java`
- New `--ack-pattern` CLI flag: a comma-separated cycle of
`accept|release|reject|renew`, applied per record via `(offset %
pattern.length)` — keyed off the immutable record offset so the assignment is
stable across redeliveries.
- When set, the consumer is switched to
`share.acknowledgement.mode=explicit` and requires `--acknowledgement-mode
sync|async` (rejects auto at arg-parse time, since explicit per-record acks
still need a commit to ship).
- `offsets_acknowledged` events now optionally include an `ackTypeCounts`
breakdown `({"ACCEPT": n, "RELEASE": n, ...})`, omitted entirely when
`--ack-pattern` isn't used — no change to the existing JSON schema for any
current caller.
`verifiable_share_consumer.py` (ducktape service)
- Plumbs `ack_pattern` through to the CLI, and adds
`total_accepted()/total_released()/total_rejected()/total_renewed()` accessors
backed by the new `ackTypeCounts` data.
`kafka.py`
- `set_share_group_dlq_config(group, topic_name, copy_record_enable)` — sets
the GroupConfig DLQ configs via kafka-configs.sh.
- `set_share_group_delivery_count_limit(group, limit)` — sets
`share.delivery.count.limit` (the per-group override).
- New share_version constructor param to bootstrap a cluster with `--feature
share.version=<n>`. Since DLQ (share.version=2) is above
`ShareVersion.LATEST_PRODUCTION`, this also flips on
`unstable.feature.versions.enable` in the broker properties.
New: `tests/kafkatest/tests/client/share_consumer_dlq_test.py`
Three tests on a real 3-broker isolated-KRaft cluster, verifying actual DLQ
topic content (not just consumer-side counters):
- `test_single_partition_dlq_reject` — every record REJECTed, DLQ'd exactly
once, values null (copy-record disabled, the default).
- `test_single_partition_dlq_release` — every record RELEASEd repeatedly;
once delivery count exceeds a lowered share.delivery.count.limit, it lands in
the DLQ.
- `test_single_partition_dlq_mixed` — records cycled reject/release/accept
by offset % 3, copy-record enabled; asserts the DLQ'd record set and values
exactly match the subset of produced values that should have been
rejected/released (cross-checked against VerifiableProducer.acked_values, not
just "value is non-null").
All three verified passing end-to-end against a live ducker cluster
(tests/docker/run_tests.sh).
```
> TC_PATHS="tests/kafkatest/tests/client/share_consumer_dlq_test.py" bash
tests/docker/run_tests.sh
================================================================================
SESSION REPORT (ALL TESTS)
ducktape version: 0.14.0
session_id: 2026-07-07--010
run time: 3 minutes 12.121 seconds
tests run: 3
passed: 3
flaky: 0
failed: 0
ignored: 0
================================================================================
test_id:
kafkatest.tests.client.share_consumer_dlq_test.ShareConsumerDLQTest.test_single_partition_dlq_mixed.metadata_quorum=ISOLATED_KRAFT
status: PASS
run time: 1 minute 5.817 seconds
--------------------------------------------------------------------------------
test_id:
kafkatest.tests.client.share_consumer_dlq_test.ShareConsumerDLQTest.test_single_partition_dlq_reject.metadata_quorum=ISOLATED_KRAFT
status: PASS
run time: 1 minute 2.198 seconds
--------------------------------------------------------------------------------
test_id:
kafkatest.tests.client.share_consumer_dlq_test.ShareConsumerDLQTest.test_single_partition_dlq_release.metadata_quorum=ISOLATED_KRAFT
status: PASS
run time: 1 minute 3.694 seconds
--------------------------------------------------------------------------------
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
