2010YOUY01 commented on PR #23599: URL: https://github.com/apache/datafusion/pull/23599#issuecomment-6011901158
Thank you for the great explanation and the fix! Is there any SQL test we can add here, for example an `EXPLAIN ...` test showing that the plan can now be optimized to WindowTopK, whereas previously it could not because of the embedded projection? I suspect this may not be reproducible with the default optimizer rule order, since `WindowTopN` runs before `ProjectionPushdown`. Only in the latter rewrite can a projection be fused into a sibling node, in this case `FilterExec`. Your downstream optimizer rule list seems to include some deeper customization that makes this case possible. This should not be a blocker, though. It seems more like a hidden assumption than a specified rule. I think this points to a deeper problem that makes optimizer rules harder to maintain in general, and I'm thinking about how we could address it systematically. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
