stuhood commented on issue #23817: URL: https://github.com/apache/datafusion/issues/23817#issuecomment-5072815310
I believe that the general thrust of this design makes sense. It feels like the analogy is that while data flows up a plan (via `ExecutionPlan`s), it is partitioned. Likewise, when data flows down a plan (via dynamic filters), is makes sense not to lose that partitioning information either. But the largest problem I see is that the partitioning information will be duplicated in the `ExecutionPlan`s and in the filters. That means that planning would need to coordinate its phases even more carefully than before to ensure that they're not out of sync. But perhaps that can always be backstopped with asserts on the producing/consuming sides? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
