jayzhan211 opened a new pull request, #26647:
URL: https://github.com/apache/datafusion/pull/26647
## Which issue does this PR close?
- No issue yet. This is the first of four PRs that give the null-aware rules
their own module; the design issue for the series will be linked here.
## Rationale for this change
The correlated `NOT IN` marking pass and its helpers make up about 370 lines
of `hash_join/stream.rs`. They sit between the regular join stream's own
functions, and the build-side `NullValueBuildRows` lives in `exec.rs`. This PR
starts moving the null-aware rules into their own module
(`hash_join/not_in.rs`) with the code that only needs to move.
## What changes are included in this PR?
- A test for correlated marking when every build value key is NULL: the
full-key map is then empty, and the marking must still run before the empty-map
shortcut. No test covered this before.
- A new `hash_join/not_in.rs`. These items move into it unchanged:
- from `exec.rs`: `NullValueBuildRows` and `null_value_key_mask`;
- from `stream.rs`: `mark_null_candidates_for_probe_batch`,
`retain_value_mismatch_free`, `for_each_scope_match`, `retain_unmarked` and
`for_each_unmarked_cross_product`.
- One signature changes. `mark_null_candidates_for_probe_batch` takes
`left_data`, `probe_batch` and `probe_values` instead of the stream's
`BuildSideReadyState` / `ProcessProbeBatchState`, whose fields are private to
`stream.rs`. The body reads the same values.
`git diff --color-moved` shows the rest as moved lines.
## What is the testing strategy for this PR?
- The new test, `test_null_aware_correlated_all_null_build_values`, covers
LeftAnti and LeftMark with the probe in two batches, across every batch-size
config. It also passes on main.
- Existing tests: physical-plan unit and integration tests, and the
sqllogictest suite, with no changes to expected results.
## Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
