zhuqi-lucas commented on issue #21596: URL: https://github.com/apache/datafusion/issues/21596#issuecomment-6076935900
One correction to the premise here, found while attempting this in #23599. > This happens when `ProjectionPushdown` runs before `WindowTopN` and embeds a projection into the `FilterExec`. That isn't true of the default rule list. In `datafusion/physical-optimizer/src/optimizer.rs`, `WindowTopN` is at position 117 and `ProjectionPushdown` at 142 — `WindowTopN` runs **first**, so a `FilterExec` reaching it never carries an embedded projection. The guard it hits (`filter.projection().is_some()`) is unreachable from a stock pipeline, and `try_swapping_with_projection` (driven by `ProjectionPushdown`) is the only producer of embedded filter projections. The shape does occur in pipelines that re-run projection pushdown earlier, which is where this came from. But that means the capability can't be exercised by `sqllogictest` as it stands: every test I wrote for it either passed on `main` unchanged, or needed a hand-built non-default rule order, so nothing keeps it honest as the rule list evolves. Per the discussion in #23599, the projection handling is dropped there for that reason (the PR is now just a bail-out guard for `FilterExec::fetch`). #26152 tracks running the slt corpus under an alternative optimizer pipeline, which is what would make this testable — worth treating as a prerequisite rather than writing the handling first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
