yujun777 opened a new pull request, #69211: URL: https://github.com/apache/doris/pull/69211
### What problem does this PR solve? Related issue: https://github.com/apache/doris/issues/65418 Problem Summary: The append-only detail (DUP) IVM model maintains only inner and cross joins, but it tested the join type alone. A join's mark is orthogonal to its type: an apply rewrite puts a mark on a `LEFT_SEMI`, a `LEFT_ANTI`, a `NULL_AWARE_LEFT_ANTI`, or even a `CROSS` join (`ExistsApplyToJoin` / `ScalarApplyToJoin`), so an `EXISTS` that lands on a `CROSS` join passes a type-only test. A mark join can stop matching a row the MV already holds, so no append can express the refresh, yet the detail normalize and the detail delta rewriter both accepted it while the unique-key model rejected the same query. Reproduction: on a DUP-key base table, create an IVM whose query contains a `NOT IN` / `EXISTS` subquery that the optimizer rewrites to a mark join whose type is `CROSS_JOIN`; before the fix the detail model accepted it, and the refresh could not maintain it. Root cause: `IvmNormalizeDupKeysMTMV#visitLogicalJoin` and `IvmDupDeltaRewriter#visitLogicalJoin` checked only `join.getJoinType().isInnerOrCrossJoin()`, with no `join.isMarkJoin()` gate; `IvmNormalizeUniqKeysMTMV` already had both gates. Fix: the two normalize paths and the detail delta rewriter now share their acceptance gates in one place, `IvmNormalizeCommonChecker`, and the join gate tests the mark before the type, carrying the per-model accepted type set (detail: inner/cross; unique-key: inner/cross/left/right/full outer). `IvmUtil.validateUserOutputColumnNames` moved into the same checker (its only caller) and was deleted from `IvmUtil`. ### Release note Fix an IVM defect where an `EXISTS` / `NOT IN` subquery rewritten to a mark join could be accepted by the append-only detail IVM model even though its refresh cannot be maintained by appends. ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [x] Unit Test - Behavior changed: - [x] Yes. The append-only detail IVM model now rejects a mark join whose type is inner or cross; the same query was previously accepted and could not be refreshed incrementally. - Does this need documentation? - [x] No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
