HappenLee commented on code in PR #67894:
URL: https://github.com/apache/doris/pull/67894#discussion_r4002362517
##########
be/src/exec/sort/partition_sorter.cpp:
##########
@@ -201,6 +197,9 @@ Status PartitionSorter::_read_row_rank(Block* output_block,
bool* eos, int batch
}
}
+ // A full batch can end inside the last qualifying peer group. Continue
reading
+ // until the next group exceeds the rank limit or the merge queue is
exhausted.
+ *eos = !queue.is_valid();
Review Comment:
Fixed in 0b26d52063e4312f890e34579177e6e7d2421f58. After each intermediate
pruning pass, the next fresh-row threshold is max(the base interval, retained
output rows). This amortizes repeated processing of a growing retained prefix
and restores the base interval after the retained peer group shrinks. Added
parameterized RANK/DENSE_RANK tests for both behaviors; with 16 all-peer input
batches, pruning processes only the prefixes of 1, 2, 4, 8, and 16 batches.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
