leizhiyuan opened a new issue, #1449: URL: https://github.com/apache/rocketmq-clients/issues/1449
### Before Creating the Bug Report - [x] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq-clients/discussions). - [x] I have searched the [GitHub Issues](https://github.com/apache/rocketmq-clients/issues) and [GitHub Discussions](https://github.com/apache/rocketmq-clients/discussions) of this repository and believe that this is not a duplicate. - [x] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ. ### Programming Language of the Client Rust ### Runtime Platform Environment Linux 6.6 x86_64, 32 cores ### RocketMQ Version of the Client/Server Client: Rust `rocketmq` 5.0.0, master `e90ece7a` Server: a mock proxy that speaks the real gRPC protocol over TCP (see the benchmark in the tracking issue); the defects are also visible by reading the code against the Java client. ### Run or Compiler Version rustc 1.97.1 (also checked with 1.74.0, the MSRV) ### Describe the Bug When an ack / nack / DLQ forward fails, `AckEntryProcessor` puts the message in a retry queue ([push_consumer.rs#L1266-L1330](https://github.com/apache/rocketmq-clients/blob/e90ece7a/rust/src/push_consumer.rs#L1266-L1330)). That queue has three problems: 1. **Head-of-line blocking.** Every 100 ms only the head entry is retried ([#L1303](https://github.com/apache/rocketmq-clients/blob/e90ece7a/rust/src/push_consumer.rs#L1303)); when it fails it stays at the head. Entries behind a failing one are never retried. 2. **`INVALID_RECEIPT_HANDLE` is retried forever.** A receipt handle that expired (e.g. the listener took longer than the invisible duration) can never be acked, so that entry stays at the head for good and blocks every later failed ack of the worker. Those messages become visible again and are consumed again. 3. **Unbounded.** The task moves every channel item into a `VecDeque` ([#L1285](https://github.com/apache/rocketmq-clients/blob/e90ece7a/rust/src/push_consumer.rs#L1285)), so the `mpsc::channel(1024)` gives no backpressure and the queue grows without limit while the server rejects acks. Java (`ProcessQueueImpl`): every message retries on its own after 1 s (`ackMessageLater`, `changeInvisibleDurationLater`, `forwardToDeadLetterQueueLater`); `INVALID_RECEIPT_HANDLE` stops the retry of acks and nacks; memory is bounded because a message stays in the process queue cache until its ack completes and reception pauses when the cache is full (`isCacheFull`). Java also retries other failures without a cap, so "no cap" alone is not the issue. ### Steps to Reproduce 400 messages; the first ack of each fails once with a transient error; 4 receipt handles are invalid. ### What Did You Expect to See? All 396 valid messages acked within a few seconds, the 4 invalid ones dropped. ### What Did You See Instead? | | `master` | fix | |---|---|---| | valid acks done after 15 s | 140–176 of 396, then stuck | 396 after 1.5 s | | ack attempts on the 4 invalid handles in 15 s | 377–413 | 4 | ### Additional Context Part of #1445. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
