duongcongtoai commented on issue #17446:
URL: https://github.com/apache/datafusion/issues/17446#issuecomment-3321147365
```
for row_idx in 0..num_rows {
// Create the empty accumulator for converting
let mut converted_accumulator = (self.factory)()?;
// Convert row to states
let values_to_accumulate =
slice_and_maybe_filter(values, opt_filter, &[row_idx,
row_idx + 1])?;
converted_accumulator.update_batch(&values_to_accumulate)?;
let states = converted_accumulator.state()?;
// Resize results to have enough columns according to the
converted states
results.resize_with(states.len(), ||
Vec::with_capacity(num_rows));
// Add the states to results
for (idx, state_val) in states.into_iter().enumerate() {
results[idx].push(state_val);
}
}
```
convert_to_state is invoked when skip aggregation is enabled at runtime
(which make partial aggregation does nothing), but this "do-nothing" cost is a
bit too much. I think as you mention we can try leverage the zero-copy
functionality of ListArrays, which reference the original input values and
adding some offsets of ones. For example given the input
[1,2,3,4,5,....]
Then `convert_to_state` is expected to return [1],[2],[3],[4],[5] ....
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
