merlimat opened a new pull request, #25652:
URL: https://github.com/apache/pulsar/pull/25652
## Summary
Move DLQ ownership from v4 per-segment `ConsumerImpl`s up to the V5
`ScalableQueueConsumer` so that:
- A single DLQ producer fans messages from every source segment, instead of
one producer per segment.
- The DLQ topic itself can be a scalable (`topic://`) topic. The v4
`ConsumerImpl` creates its DLQ producer via `client.newProducer(...)`, which
rejects the `topic://` domain — V5 now uses its own producer builder, so a
scalable DLQ \"just works\".
- The default DLQ topic name (when the policy doesn't set one) is
`topic://<tenant>/<ns>/<source-local>-DLQ`.
`ScalableTopicProducer`'s send path is now fully async: segment producers
are stored as `CompletableFuture` so callers running on a netty IO thread (e.g.
the V5 DLQ dispatch off a v4 receive callback) can chain instead of blocking on
`.get()`. A per-segment dispatch chain serializes v4 `sendAsync` calls in
user-call order; `flushAsync` awaits the snapshot of in-flight send futures.
## Test plan
- [x] `V5DeadLetterPolicyTest`:
- `testMessageGoesToScalableDlqWhenExplicitlyConfigured` — explicit
scalable DLQ target.
- `testMessageGoesToDefaultScalableDlqTopic` — default DLQ name
(`<source>-DLQ`).
- `testDlqMessagePreservesKeyPropertiesAndOriginMetadata` — verifies key,
user properties, eventTime, and origin metadata (`REAL_TOPIC` /
`REAL_SUBSCRIPTION` / `ORIGIN_MESSAGE_ID`) are attached.
- `testDlqAcrossMultipleSourceSegments` — 3-segment source, 6 keys, single
shared V5-side DLQ producer.
- [x] Full V5 test suite (`org.apache.pulsar.client.api.v5.*`) green:
113/113 pass.
- [x] `pulsar-client-v5` and `pulsar-broker` checkstyle clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
