HappenLee commented on code in PR #67894:
URL: https://github.com/apache/doris/pull/67894#discussion_r4002709220
##########
be/src/exec/common/partition_sort_utils.cpp:
##########
@@ -40,12 +42,15 @@ Status PartitionBlocks::append_block_by_selector(const
Block* input_block, bool
_init_rows = _init_rows - selector_rows;
_current_input_rows = _current_input_rows + selector_rows;
_selector.clear();
- // maybe better could change by user PARTITION_SORT_ROWS_THRESHOLD
if (!eos && _partition_sort_info->_partition_inner_limit != -1 &&
- _current_input_rows >= PARTITION_SORT_ROWS_THRESHOLD &&
+ _current_input_rows >= _partition_sort_rows_threshold &&
_partition_sort_info->_topn_phase !=
TPartTopNPhase::TWO_PHASE_GLOBAL) {
create_or_reset_sorter_state();
RETURN_IF_ERROR(do_partition_topn_sort());
+ // Amortize reprocessing retained peers over at least as many
fresh rows.
+ _partition_sort_rows_threshold =
Review Comment:
Fixed in 9593d49a92567a600bf175391c56117dc1d33624. After a successful
intermediate pruning pass, the code first records the next retained-row
threshold, then immediately calls reset_sorter_state() to release the unread
merge queue and saved peer cursor. The retained output already belongs to
PartitionBlocks::_blocks. The creation helper now only creates the sorter when
needed, avoiding another reset at the next pass or final sink preparation.
Added parameterized RANK/DENSE_RANK coverage that grows the retained group to
four base intervals, shrinks it to two while rejecting six intervals, and
checks that both queue and peer cursor are cleared immediately. The test also
verifies the next pruning interval and final output using the same sorter. All
19 ASAN sorter tests and the formatting, build hygiene, and clang-tidy checks
passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
