pitrou commented on pull request #9742: URL: https://github.com/apache/arrow/pull/9742#issuecomment-803945598
> For example, you can generally filter batch N while filtering batch N+1 at the same time. Would there be a single ExecNode that is run twice? If so won't you have trouble tracking inputs? There would be a single ExecNode that is run twice. Tracking the input is done through the `batch_index` (this is where you pass your N, N+1...). There is even a bit of logic implemented :-) I don't think independent ExecNodes would work, since some operations (e.g. hash aggregation or any vector function) need access to the entire input before starting to compute their output. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
