alamb commented on code in PR #22416:
URL: https://github.com/apache/datafusion/pull/22416#discussion_r3306567124
##########
datafusion/common/src/utils/mod.rs:
##########
@@ -395,6 +395,70 @@ pub fn longest_consecutive_prefix<T: Borrow<usize>>(
count
}
+/// Splits `vec` at index `n`, returning the first `n` elements and leaving the
+/// remaining `vec.len() - n` elements in `vec`.
+///
+/// Allocates for whichever side is smaller, so the new allocation is
+/// `min(n, vec.len() - n)` rather than always `n` (as
`vec.drain(0..n).collect()`
+/// would). This matters when the split emits a prefix under memory pressure,
+/// where `n` can be close to `vec.len()`.
+pub fn split_vec_min_alloc<T>(vec: &mut Vec<T>, n: usize) -> Vec<T> {
+ if n * 2 <= vec.len() {
+ vec.drain(0..n).collect()
+ } else {
+ let remaining = vec.split_off(n);
Review Comment:
calling `shrink-to-fit` will cause a reallocation (and thus copy the data)
right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
