viirya opened a new pull request, #25693: URL: https://github.com/apache/datafusion/pull/25693
## Which issue does this PR close? - Closes #25691. ## Rationale for this change Null-aware `LeftMark` joins use physical null checks for dictionary keys. They miss valid dictionary keys that point to `NULL` dictionary values and return `FALSE` marks where SQL three-valued logic requires `NULL`. ## What changes are included in this PR? Null-aware `LeftMark` joins now use Arrow's logical-null APIs on both probe and build keys. The build-side logical null bitmap is computed once before constructing the mark array. ## What is the testing strategy for this PR? The new `HashJoinExec` tests cover logical dictionary nulls on both build and probe sides, both probe-partition completion orders, and the existing batch-size/configuration matrix. Ablation testing confirmed that all 10 new cases fail without the fix; all 30 related `LeftMark` cases pass with it. The 30 focused `LeftMark` tests pass on this branch. Before the two independent fixes were split into separate branches, they also passed `cargo fmt --all`, the extended workspace test suite, and clippy across the workspace except `datafusion-benchmarks` (the local host has CMake 3.16.3 while `snmalloc-sys` requires 3.21) with two pre-existing Rust 1.98 lints allowed. ## Are there any user-facing changes? Null-aware mark joins over dictionary-encoded keys now return the correct nullable marks. There are no public API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
