alamb commented on issue #5492: URL: https://github.com/apache/datafusion/issues/5492#issuecomment-6402148247
> It is one logical optimizer rule that replaces the three rules DecorrelatePredicateSubquery, ScalarSubqueryToJoin and DecorrelateLateralJoin, and their shared helper PullUpCorrelatedExpr. As a logical rule, it also covers plans from the DataFrame API and Substrait. I really like this property. It also points at an incremental rollout strategy that would be nice -- add a new logical rule and leave the existing rules to co-exist temporarily Then at some point we can deprecate the old rules, and stop running the old phyiscal rules by default (and remove them in a release or two) but > Your strangler-fig plan fits the prototype well, with one change: split the work by operator rather than by shape. Most of the missing-capability shapes need the same core, so the first shape would carry most of the rule. What does "strangler-fig" mean in this case? > The third is a hash join bug on main, unrelated to decorrelation: a LEFT hash join whose probe side is a UNION ALL drops an unmatched row in about a quarter of runs. Do we have a ticket tracking this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
