alamb commented on issue #17261: URL: https://github.com/apache/datafusion/issues/17261#issuecomment-3529432027
This is great 🕵️ @logan-keede 🙏 One potential way to improve this would be to teach the optimizer to combine the Projections somehow -- specifically I think the current https://github.com/apache/datafusion/blob/967384e16257bab4deb48bbeff38507ea13c4abd/datafusion/optimizer/src/eliminate_nested_union.rs#L39-L38 rewrites this pattern: ``` Union <plan1> Union <plan2> ``` Into this pattern: ``` Union <plan1> <plan2> ``` Maybe we could extend the code somehow to rewrite this pattern too: ``` Union <plan1> Projection <...> Union <plan2> ``` ``` Union <plan1> Projection <...> <plan2> ``` (aka "look through" the projection) Actually now that I type this, it might hold for any number of single input children between the Union and their children (not just for Projections) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
