rkrishn7 opened a new issue, #17451: URL: https://github.com/apache/datafusion/issues/17451
### Describe the bug Presently, partition bounds are reported after the build side is collected during a hash join. However, there is no synchronization to wait until all bounds have been reported _before_ we begin polling the right side. This is so we can ensure that upstream APIs like `FileOpener::open` can immediately observe the updated dynamic filter as they may begin using it right away. Without some synchronization, the behavior is racy and it's not guaranteed that that the dynamic filter is built prior to initiating the right side's execution plan. ### To Reproduce Reproducing is flaky given that the behavior is not determinate. I've been able to observe the placeholder dynamic filter expr (`lit(true)`) from time to time by adding a log to `TestOpener::open` in `core::tests::physical_optimizer::filter_pushdown::utils`. ### Expected behavior The updated dynamic filter should be immediately visible on the probe side. ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
