sunchao commented on PR #56535: URL: https://github.com/apache/spark/pull/56535#issuecomment-4747940075
cc @cloud-fan @MaxGekk @viirya would appreciate reviews on this follow-up, especially alongside #56603. Both address regressions from #56071, but at different layers: - #56535 is correctness-focused: it requires checkpoints to be materialized and the complete filtering plan to be repeatable, preventing DPP from pruning with results different from the join. - #56603 is cost-focused: it avoids unnecessary standalone DPP when a materialized side lacks reliable selectivity evidence. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
