[ https://issues.apache.org/jira/browse/ARROW-10026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198197#comment-17198197 ]
Frank Du commented on ARROW-10026: ---------------------------------- Seems most batch for ArrayData now is exactly a copy of current, worth to add a check before Slice? Per perf, no Slice anymore for scalar-arithmetic-benchmark, and the data improved slightly(3%) {code:java} diff --git a/cpp/src/arrow/compute/exec.cc b/cpp/src/arrow/compute/exec.cc index 435d7dd74..7c33a7198 100644 --- a/cpp/src/arrow/compute/exec.cc +++ b/cpp/src/arrow/compute/exec.cc @@ -171,7 +171,12 @@ bool ExecBatchIterator::Next(ExecBatch* batch) { if (args_[i].is_scalar()) { batch->values[i] = args_[i].scalar(); } else if (args_[i].is_array()) { - batch->values[i] = args_[i].array()->Slice(position_, iteration_size); + if (position_ || iteration_size != length_) { + batch->values[i] = args_[i].array()->Slice(position_, iteration_size); + } else { + batch->values[i] = args_[i].array(); + } } else { const ChunkedArray& carr = *args_[i].chunked_array(); const auto& chunk = carr.chunk(chunk_indexes_[i]); {code} > [C++] Improve kernel performance on small batches > ------------------------------------------------- > > Key: ARROW-10026 > URL: https://issues.apache.org/jira/browse/ARROW-10026 > Project: Apache Arrow > Issue Type: Task > Components: C++ > Reporter: Antoine Pitrou > Priority: Major > > It seems that invoking some kernels on smallish batches has quite an overhead: > {code} > ArrayArrayKernel<Add, Int32Type>/32768/100 2860 ns > 2859 ns 245195 bytes_per_second=10.6727G/s > items_per_second=2.86494G/s null_percent=1 size=32.768k > ArrayArrayKernel<Add, Int32Type>/32768/0 2752 ns > 2751 ns 249316 bytes_per_second=11.093G/s items_per_second=2.97775G/s > null_percent=0 size=32.768k > ArrayArrayKernel<Add, Int32Type>/524288/100 18633 ns > 18630 ns 36548 bytes_per_second=26.2097G/s > items_per_second=7.03561G/s null_percent=1 size=524.288k > ArrayArrayKernel<Add, Int32Type>/524288/0 18260 ns > 18257 ns 38245 bytes_per_second=26.7451G/s > items_per_second=7.17933G/s null_percent=0 size=524.288k > {code} > We should investigate and try to lighten the overhead. -- This message was sent by Atlassian Jira (v8.3.4#803005)