westonpace commented on PR #13179: URL: https://github.com/apache/arrow/pull/13179#issuecomment-1175648192
> I noticed another interesting effect here. Intuitively, it would make sense to also use separate output/intermediate buffers for each batch, since especially simple_expression benefits a lot from just writing to the same buffer each time. Turns out arrow already benefits from the same effect, probably due to the allocator giving it back the same memory each time. Without this effect, simple_expression does not benefit from small batch sizes at all: I think we can someday get to a place where these output buffers are thread local temporaries and are not re-allocated each time so let's keep it how it is. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
