HappenLee opened a new pull request, #68809:
URL: https://github.com/apache/doris/pull/68809
### What problem does this PR solve?
Issue Number: None
Related PR: #65975
Problem Summary:
Backport #65975 to `branch-4.1`.
A single-column string hash join using `<=>` can fail to match two NULL keys
when expression evaluation leaves different bytes in their nested string
columns. For example, joining `CONCAT(COALESCE(p.ch, ''), CAST(p.v % 5 AS
STRING)) <=> c.s` should match a NULL expression result with a NULL `c.s`, but
previously could miss that row depending on block composition.
Normalize NULL keys to an empty `StringRef` in the serialization shared by
build and probe. Real empty strings remain in ordinary hash buckets, separate
from the dedicated NULL bucket. Also avoid passing default empty refs to
`memcmp`, and reject unsupported `<=>` ON predicates in ASOF JOIN.
Resolve the parser conflict by retaining the 4.1 UNNEST handling and
changing only the ASOF equality check. Resolve the BE test conflict by adding
the original NULL normalization test without importing unrelated master-only
iterator tests. Adapt the existing ASOF regression to its `nereids_p0` location
on 4.1.
### Release note
Fix incorrect results of single-column null-safe string joins when NULL keys
come from expressions. ASOF JOIN rejects unsupported null-safe equality ON
predicates.
### Check List (For Author)
- Test:
- BE formatting: repository clang-format/check-format scripts passed
with clang-format 16.0.6.
- Regression: preserve the upstream expression-NULL, mixed-block,
NULL-versus-empty-string, and ASOF rejection suites. The generated expected
output is copied unchanged from the original commit; local cluster execution
has not been run.
- FE Unit Test:
`NereidsParserTest#testParseAsofJoinRejectNullSafeEquality+testParseJoin`
passed locally (2 tests, 0 failures/errors).
- BE Unit Test: the original normalization test is preserved; local ASAN
build is in progress. Initial setup had a missing simdutf library, which was
installed using the repository third-party build script.
- Behavior changed: Yes, correct NULL string key matching and reject
unsupported ASOF ON predicates.
- Does this need documentation: No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
