imzs opened a new issue, #11272:
URL: https://github.com/apache/rocketmq/issues/11272

   ### Before Creating the Bug Report
   
   - [x] I found a bug, not just asking a question, which should be created in 
[GitHub Discussions](https://github.com/apache/rocketmq/discussions).
   
   - [x] I have searched the [GitHub 
Issues](https://github.com/apache/rocketmq/issues) and [GitHub 
Discussions](https://github.com/apache/rocketmq/discussions)  of this 
repository and believe that this is not a duplicate.
   
   - [x] I have confirmed that this bug belongs to the current repository, not 
other repositories of RocketMQ.
   
   
   ### Runtime platform environment
   
   .
   
   ### RocketMQ version
   
   .
   
   ### JDK Version
   
   .
   
   ### Describe the Bug
   
   We introduced CombineConsumeQueueStore and made the LMQ ConsumeQueue (CQ) 
build
   asynchronous in RocksDB. Specifically, the file-version LMQ CQ construction 
is gated by
   `MultiDispatchUtils.checkMultiDispatchQueue(config, request)`.
   
   When `rocksdbCQDoubleWriteEnable && combineCQUseRocksdbForLmq` both hold, 
this predicate
   returns `false`, so the file-version LMQ CQ is not built during reput. 
Instead, the LMQ
   CQ is written and committed **asynchronously** by RocksGroupCommitService. 
The correct arrival notification for this path is emitted only after the batch 
commit, via `RocksDBConsumeQueueStore#notifyMessageArriveAndClear -> 
notifyMessageArriveIfNecessary(dp)`.
   
   #10173 blocked the file-version LMQ CQ construction, but it did not block 
the LMQ notification on the reput path. 
`DefaultMessageStore#notifyMessageArriveIfNecessary` is a shared entry point 
used by both:
   1. the reput path — `ReputMessageService#doReput` and 
`DispatchService#dispatch`; 
   2. the post-commit path — 
`RocksDBConsumeQueueStore#notifyMessageArriveAndClear`.
   
   On the reput path it still unconditionally invokes 
`notifyMessageArrive4MultiQueue(dispatchRequest)`,
   which fires `MessageArrivingListener#arriving(...)` for every LMQ. So for a
   `CombineConsumeQueueStore` with `combineCQUseRocksdbForLmq=true`, the LMQ 
arrival is notified
   during reput, i.e. before the RocksDB CQ batch has actually been committed, 
then the prematurely woken long-polling consumer reads an empty / stale queue.
   
   ### Steps to Reproduce
   
   .
   
   ### What Did You Expect to See?
   
   .
   
   ### What Did You See Instead?
   
   .
   
   ### Additional Context
   
   .


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to