englefly commented on code in PR #10354:
URL: https://github.com/apache/doris/pull/10354#discussion_r904799723
##########
be/src/vec/exec/volap_scan_node.cpp:
##########
@@ -228,8 +228,12 @@ void VOlapScanNode::scanner_thread(VOlapScanner* scanner) {
std::lock_guard<std::mutex> l(_free_blocks_lock);
_free_blocks.emplace_back(block);
} else {
+ // block should not exceed MAX_BLOCK_SIZE,
+ // in order to avoid generating supper huge block when reading
big-wide-table
+ constexpr size_t MAX_BLOCK_SIZE = 1024 * 1024 * 100;
if (!blocks.empty() &&
- blocks.back()->rows() + block->rows() <=
_runtime_state->batch_size()) {
+ blocks.back()->rows() + block->rows() <=
_runtime_state->batch_size() &&
Review Comment:
将外层while循环的条件调整，遇到 raw_bytes_read >= raw_bytes_threshold
就推出循环，这样可以避免大宽表时生成巨大的block
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
