adriangb opened a new pull request, #27345: URL: https://github.com/apache/datafusion/pull/27345
## Which issue does this PR close? - Related to https://github.com/apache/datafusion/issues/21625 (already closed by https://github.com/apache/datafusion/pull/21666). ## Rationale for this change https://github.com/apache/datafusion/pull/21666 fixed a hang in partitioned hash joins with dynamic filters (first seen as TPC-H Q18 hanging). Its only test is a Rust unit test of the coordinator. No SQL-level test covers the plan shape that triggers the hang, so a planner or execution change could bring it back unnoticed. ## What changes are included in this PR? One new file, `join_dynamic_filter_canceled_partition.slt`. It is a small, deterministic reproducer: - A `Partitioned` inner join sits on top of a `Partitioned` child join on the same key, with no repartition between them. - The parent build side holds only `k = 1`, so 3 of the 4 parent partitions have an empty build side. They complete without polling their probe side, so their child partitions are dropped before they report to the shared dynamic-filter coordinator. - The one non-empty parent partition polls its child partition, which waits for the dropped partitions. Before the fix, it waits forever. The hang does not depend on timing: the hash routing of `k = 1` is fixed, and a child partition reports only when its parent polls it. An `EXPLAIN` assertion locks the plan shape the reproducer needs. The file comments link to the issue and describe the sequence of events. ## What is the testing strategy for this PR? - The new test passes on `main`. - With the fix disabled (the `report_canceled_partition` call in `HashJoinStream::drop` replaced by a no-op), the test hangs. It never completed and was killed by a 500 s timeout. - With `datafusion-cli`, the same query hangs 5 of 5 times at `887595660^` and returns 2 rows 5 of 5 times at `887595660`. Note: a regression shows up as a hang, not a failed assertion, so CI would catch it by the job timeout. ## Are there any user-facing changes? No. Test-only. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
