[GitHub] [rocketmq] azhsmesos commented on issue #5184: Read Commitlogs concurrently to speed up generation of the consumeQueue

[GitBox]

azhsmesos commented on issue #5184:
URL: https://github.com/apache/rocketmq/issues/5184#issuecomment-1264275555

   > @RongtongJin @aaron-ai @lizhanhui @mxsm @Oliverwqcwrw ❤️ ❤️ Excuse me, can 
you help me see if the plan is suitable? Or do i need make a RIP?
   > 
   > **The old way** 
![reput-old](https://user-images.githubusercontent.com/72536832/193395959-99b86489-21c4-4250-995b-d7b4db91b121.png)
   > 
   > **The new way** 
![reput-new](https://user-images.githubusercontent.com/72536832/193395962-f5e6b176-a08d-47d9-b373-3e1732ff6d17.png)
   
   Currently, the bottleneck is that reading Commitlog files is 
single-threaded. In fact, consumerQueue can generate different commitlog files 
in parallel and without locking. In the figure, MSG table needs to be locked , 
so we can replace it with ThreadLocal. Reading a commitLog file builds 
consumerQueue by reading offsets from the ThreadLocal of the current thread.
   Do you think it is possible to use ThreadLocal to store the offset of MSG？ 
It supports lock-free operations.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org