alamb commented on PR #19462: URL: https://github.com/apache/datafusion/pull/19462#issuecomment-3739805622
> Imagine that there is an `AggregateExec` in the plan. It owns a dynamic filter that is updated when aggregation is executed. For each plan execution, a separate instance of such a filter should be created and somehow pushed down to the child operator so that it can filter its inputs accordingly. Yeah this is a good point > In the suggested approach (this patch), this is solved by splitting filters into two types: planning-time filters and execution-time filters. When `execute(...)` is called, an independent version of the filter is created and then pushed into the children using state. Perhaps we could improve reset_state(...) to support such pushes. It seems we cannot do it without `reset_state` modification as the node that wants to poll filters should take its new version from somewhere (or we can try to re-push it someway). I agree we would need some way / API in the dynamic predicates to update the pointers to the new operator instance, as part of running the plan again 🤔 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
