killxdcj opened a new issue #5606: URL: https://github.com/apache/incubator-doris/issues/5606
After increased ScannerThread num, i encountered serious lock competition problems. Judging from the flamegraph, the main problem lies in the construction and destructuring of Memtracker in RowBlock.  Checked the relevant code, i found the current RowBlock's parent MemTracker is root Memtracker(one per process). When the concurrency becomes high, the lock competition must be serious. So should we set RowBlock's parent MemTracker as instance_mem_tracker? On the other hand, for Query, should RowBlock's memory be limited by instance_mem_tracker? Tested after changing RowBlock's Parent MemTracker to instance_mem_tracker, the lock competition problem disappeared, and in our scenario, the performance was also improved several times  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
