leizhiyuan opened a new issue, #1415: URL: https://github.com/apache/rocketmq-clients/issues/1415
### Before Creating the Bug Report - [x] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq-clients/discussions). - [x] I have searched the [GitHub Issues](https://github.com/apache/rocketmq-clients/issues) and [GitHub Discussions](https://github.com/apache/rocketmq-clients/discussions) of this repository and believe that this is not a duplicate. - [x] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ. ### Programming Language of the Client Rust ### Runtime Platform Environment macOS 27 (arm64) ### RocketMQ Version of the Client/Server Client: Rust `rocketmq` 5.0.0, master `a3d1de8b` Server: a managed RocketMQ 5.x service (Tencent Cloud TDMQ for RocketMQ 5), gRPC proxy endpoint ### Run or Compiler Version rustc 1.98.0 ### Describe the Bug `Producer::batch_send` reports success for a batch of lite messages, but the messages are never delivered to lite consumers. More generally, `batch_send` returns whatever receipts the server sends back, even when their number differs from the number of messages sent. **Scenario.** A producer sends 3 lite messages with `batch_send` to save round trips. The call returns `Ok`, so the application treats the messages as published. A `LitePushConsumer` subscribed to the lite topic with `OffsetOption` `MIN` receives none of them, even after 65 seconds. A message sent with `send` to another lite topic in the same run is consumed normally. The same thing happens when the 3 messages belong to different lite topics: the client accepts the batch, and the server returns `OK`. The call returns a single `SendReceipt`, whose `message_id` is the 3 client message ids joined with `,`: ``` batch_send(3 lite messages, same lite topic) -> Ok, 1 receipt: "01020000...0001,01020000...0002,01020000...0003" -> consumed with MIN offset after 65s: [] ``` **What the client sends.** I captured the request at the gRPC layer. The client builds one `SendMessageRequest` that carries 3 `pb::Message`s, each with its own `lite_topic` and `message_id`. This is the same request shape that the Java client builds. The response has `status = OK` and **one** `SendResultEntry`, whose `message_id` is the joined ids. **Java comparison.** Java 5.x has no public batch API, but `ProducerImpl#send(List<Message>, boolean)` is the code path behind the public single send, so I called it via reflection with the same 3 lite messages on the same server: | | Java 5.2.2 | Rust (master) | |---|---|---| | client-side checks before sending | same topic, same message type, same message group for FIFO (no lite topic check) | same topic, same message group | | server response | 1 entry for 3 messages | 1 entry for 3 messages | | result | `InternalErrorException: received send receipt's quantity 1 is not equal to sent message's quantity 3` | `Ok` with 1 receipt | | consumed by a lite consumer | 0 | 0 | So the server behaves the same for both clients. Java fails the send because the receipt count does not match (`ProducerImpl#send0`). Rust reports success. **Server side (from the proxy / broker sources, consistent with the observations above).** - **The proxy sends a multi-message `SendMessageRequest` as one batch to the broker.** It then returns one `SendResultEntry` per broker `SendResult`, and a batch yields exactly one (`// factually, the size of result is always 1` in `SendMessageActivity`). This holds for any batch, lite or not, and is also the case in Apache RocketMQ proxy 5.1.2. - **Only the single-message path dispatches to the lite topic.** In the broker, the single-message path of `SendMessageProcessor` turns the lite topic property into an `INNER_MULTI_DISPATCH` entry, which is what dispatches the message to the lite topic's queue. `sendBatchMessage` has no such step, so a batched lite message is only stored in the parent topic and never reaches lite consumers. There is a related latent panic in `Producer::send`. It returns `results[0].clone()`, which panics if the server answers `OK` with no entry. ### Steps to Reproduce 1. Create a lite topic and a `LitePushConsumer` group. 2. `producer.batch_send(vec![m0, m1, m2])` with 3 messages built by `MessageBuilder::lite_message_builder(topic, body, lite_topic)`. 3. Subscribe to `lite_topic` with the `MIN` offset policy and wait. ### What Did You Expect to See? Either the messages are delivered, or `batch_send` fails, as the Java client does. A batch of lite messages should be rejected with a clear error before sending, since the server cannot deliver it. ### What Did You See Instead? `Ok` with one joined receipt, and no message delivered. ### Additional Context **Fixing this is a breaking change for `batch_send`.** As described above, the proxy returns a single entry for every batch. So once the client checks the receipt count, as Java does, `batch_send` with more than one message fails for normal messages as well, and callers have to use `send` for each message. I think that is the right trade-off. For normal messages the batch is probably stored, but the client gets neither the per-message ids nor the offsets, and returning `Ok` with a joined id is more dangerous than an explicit failure. I will describe it in the PR. --- Tracked in #1412. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
