jayzhan211 commented on PR #26056: URL: https://github.com/apache/datafusion/pull/26056#issuecomment-6358821056
I think in-place reset should be the only pattern, and `HashJoinExec` and `SortExec` should move to it. The filter's identity is what connects the producer to the scan, and `reset_state` can't reconnect it. `reset_plan_states` resets the scan before its producer, and the scan keeps the old filter. So dropping or recreating the filter leaves the scan with one nobody updates: a recursive join prunes 0 rows today, versus 98/100 outside a CTE (#19905). The only alternative is a second pass to reconnect filters (`PushedDynamicFilters` in #19905), which is much more machinery than one `reset()`. The cost is that a reset copy can't run at the same time as the original. Main already has that limitation, since the scan shares the filter either way, and `RecursiveQueryExec` runs iterations one after another. I'll document it on `reset_state`. I'll open follow-up issues to move `HashJoinExec` and `SortExec` to in-place reset. Each conversion is a few lines, plus a `cte.slt` check that the recursive case prunes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
