seawinde opened a new pull request, #63268:
URL: https://github.com/apache/doris/pull/63268
### What problem does this PR solve?
Issue Number: N/A
Related PR: #62492
Problem Summary:
INNER JoinEdge null-reject inference can validate rewriting an INNER JOIN
query by an OUTER JOIN materialized view without adding the required non-null
compensation predicate. The rewritten plan can keep null-padded rows from the
MV side that should be rejected by the original query.
Root cause: In AbstractMaterializedViewRule.predicatesCompensate(), the
previous check treated INNER JoinEdge null-reject inference as proof that an
OUTER JOIN MV rewrite was valid, but the proof was not materialized as a real
IS NOT NULL predicate in the rewritten query.
Change Summary:
| File | Change Description |
|------|--------------------|
| AbstractMaterializedViewRule.java | Split predicate-based null-reject
proof from INNER JoinEdge proof and add query-based IS NOT NULL compensation
when only JoinEdge proof covers required MV nullable sides. Fail rewrite if no
safe MV output slot can carry the compensation predicate. |
| NullRejectInferenceTest.java | Add unit coverage for LEFT/FULL OUTER JOIN
MV rewrites that require INNER JoinEdge null-reject compensation on both sides.
|
| inner_join_null_reject_compensation.groovy | Add regression coverage with
unmatched OUTER JOIN MV rows, including the LEFT JOIN MV to INNER JOIN query
repro with nullable join keys. |
Design rationale: Existing query predicates already flow through normal
predicate compensation, so they do not need extra filters. INNER JoinEdge proof
is only logical evidence; when it is needed to reject null-generated MV rows,
the rewrite must add a real IS NOT NULL predicate on an MV output slot. If no
such slot is available, the rewrite is rejected conservatively.
### Release note
Fixed an issue where OUTER JOIN materialized view rewrite could return extra
null-padded rows for INNER JOIN queries.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
Unit tests / checks:
- Added NullRejectInferenceTest coverage for INNER/FULL join null-reject
compensation on both sides.
- Ran git diff --check.
- Tried ./run-fe-ut.sh --run
org.apache.doris.nereids.rules.exploration.mv.NullRejectInferenceTest, but FE
core compilation failed before tests because generated cloud proto classes miss
Cloud.CreateMetaSyncPointRequest/Response in MetaServiceClient and
MetaServiceProxy.
Regression test:
- Added inner_join_null_reject_compensation.groovy for FULL/LEFT OUTER JOIN
MV rewrites with unmatched null-padded rows.
- Not run locally; the local FE UT build is currently blocked by the cloud
proto compilation issue above.
- Behavior changed:
- [x] Yes. OUTER JOIN MV rewrite now adds real IS NOT NULL compensation
when INNER JoinEdge null-reject inference is required, or rejects the rewrite
if no safe MV output slot can carry that predicate.
- [ ] No.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
Co-authored-by: Copilot <[email protected]>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
