liaoxin01 commented on code in PR #33325:
URL: https://github.com/apache/doris/pull/33325#discussion_r1570194029
##########
be/src/olap/memtable_flush_executor.cpp:
##########
@@ -203,15 +203,18 @@ void
FlushToken::_flush_memtable(std::unique_ptr<MemTable> memtable_ptr, int32_t
void MemTableFlushExecutor::init(int num_disk) {
num_disk = std::max(1, num_disk);
- size_t min_threads = std::max(1, config::flush_thread_num_per_store);
- size_t max_threads = num_disk * min_threads;
+ int num_cpus = static_cast<int>(std::thread::hardware_concurrency());
Review Comment:
If the value is not well defined or not computable，hardware_concurrency may
return 0. We should process this case.
##########
be/src/common/config.cpp:
##########
@@ -662,6 +662,12 @@ DEFINE_mInt64(storage_flood_stage_left_capacity_bytes,
"1073741824"); // 1GB
DEFINE_Int32(flush_thread_num_per_store, "6");
// number of thread for flushing memtable per store, for high priority load
task
DEFINE_Int32(high_priority_flush_thread_num_per_store, "6");
+// number of threads = min(flush_thread_num_per_store * num_store,
+// max_flush_thread_num_per_cpu * num_cpu)
+DEFINE_Int32(max_flush_thread_num_per_cpu, "8");
Review Comment:
The default value of 8 is a bit large, and 2 or 4 may be more suitable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
