lizhimins opened a new issue, #8472: URL: https://github.com/apache/rocketmq/issues/8472
### Before Creating the Bug Report - [X] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq/discussions). - [X] I have searched the [GitHub Issues](https://github.com/apache/rocketmq/issues) and [GitHub Discussions](https://github.com/apache/rocketmq/discussions) of this repository and believe that this is not a duplicate. - [X] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ. ### Runtime platform environment None ### RocketMQ version RocketMQ 5.3.0 ### JDK Version JDK11 ### Describe the Bug Pop consumption may cause consumption latency under the following three conditions: 1. The topic's write traffic is heavy. 2. The consumer uses consumption filtering and the number of matched messages is very low. For example, only one in a thousand (0.1% of the traffic) is valid. 3. The number of consumers is small, with only one or a few clients. Detailed reasons: When use pop consumption, the action of popping a message from the queue is eventually used to get messages from the storage. The storage layer imposes a limit on the number of messages filtered per single request. The current configuration is 16,000 / 20 = 800 messages per queue. If there are no matching messages among these 800, the storage will return 'not match' as result. At this point, the pop request does not utilize the network layer to perform an action similar to "recursion," resulting in the notify message generated during new message writes being unable to correctly trigger the response of pending long polling. 同时满足以下三个条件时 pop 消费可能会造成消费延迟： 1. topic 生产 / 写入流量较大 2. 消费者使用消费过滤且有效匹配的消息量很少，例如千分之一的有效流量 5. 消费者的数量较少，只有一个或少数客户端 细节原因： 当 pop 消费时，最终会使用 pop message from queue 的动作从存储层获取消息，存储层会对单次请求向后过滤的消息量进行限制，当前配置为 16000 / 20 = 800 条每队列，当 800 条数据里没有匹配的消息时，存储层会返回 not match。此时 pop 请求没有使用利用网络层进行类似 “递归” 的动作，造成单次新消息写入时产生的 notify message arrive 无法让挂起的长轮询正确的写回响应。 ### Steps to Reproduce Simultaneously meeting the above three conditions, consumer receive message will delay. ### What Did You Expect to See? Instant consumption without delay. ### What Did You See Instead? There is a delay in the pop consumption, with the delay time being within 20 seconds, and an accumulation in consumption has been observed. ### Additional Context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
