merlimat opened a new pull request, #25716:
URL: https://github.com/apache/pulsar/pull/25716
### Motivation
The test asserted that the low-priority consumer must see at least `total -
highReceiverQueue * 2 - 1` (= 9) of the 20 messages, on the assumption that the
high-priority consumer's prefetch fills at `receiverQueueSize` and the broker
spills the remainder to low. CI hit:
```
java.lang.AssertionError: low-priority consumer must see overflow once high
consumer's prefetch fills, got: 7
```
The broker log shows the high consumer closed with `pendingAcks=13` — the
broker had dispatched 13 of 20 messages to the "paused" high consumer, leaving
only 7 for low:
```
PersistentDispatcherMultipleConsumers - Removed consumer with pending acks
consumer=...consumerName=high-paused-seg-0..., pendingAcks=13
```
This is a consequence of how V5 queues consumed messages.
`receiverQueueSize` sizes the v4-level internal receive buffer; it isn't a
broker-side backpressure cap.
[`ScalableQueueConsumer.startReceiveLoop`](https://github.com/apache/pulsar/blob/master/pulsar-client-v5/src/main/java/org/apache/pulsar/client/impl/v5/ScalableQueueConsumer.java#L437)
runs a continuous `receiveAsync()` loop that drains v4's queue into an
unbounded V5 `LinkedTransferQueue`, which keeps the v4 consumer sending `FLOW`
commands and refilling broker-side permits. As a result the broker can dispatch
arbitrarily many messages to "high-paused" before yielding to low, regardless
of `receiverQueueSize`.
The test's own comment already states the real semantic guarantee:
> The minimum guarantee is that the low consumer receives strictly more than
zero overflow.
…but the assertion checked `lowSeen.size() >= 9`, which contradicts the
comment and assumes V4-style hard backpressure that V5 doesn't provide today.
Adding a true broker-side pause/backpressure mechanism to V5 is a separate,
larger design change.
### Modifications
- Replace the `lowSeen.size() >= total - highReceiverQueue * 2 - 1`
assertion with `lowSeen.size() > 0`, matching the test's stated minimum
guarantee.
- Update the comment to explain why exact counts can't be asserted under
V5's design.
### Verifying this change
This change is a trivial test relaxation; the test continues to verify that
V5's `priorityLevel(int)` knob causes overflow to reach lower-priority
consumers, just without asserting an exact count.
### Does this pull request potentially affect one of the following parts:
- [ ] Dependencies (add or upgrade a dependency)
- [ ] The public API
- [ ] The schema
- [ ] The default values of configurations
- [ ] The threading model
- [ ] The binary protocol
- [ ] The REST endpoints
- [ ] The admin CLI options
- [ ] The metrics
- [ ] Anything that affects deployment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
