yujun777 commented on code in PR #68390:
URL: https://github.com/apache/doris/pull/68390#discussion_r4082556600
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/mtmv/MTMVTask.java:
##########
@@ -448,6 +462,17 @@ private List<RefreshAttemptType>
buildAttempts(RefreshRequest request, boolean c
if (shouldUseCompleteForInitialIvmRefresh(containsOneRowRelation)) {
return Lists.newArrayList(RefreshAttemptType.COMPLETE);
}
+ // A schema-level invalidation is not a set of dirty partitions: it
means every partition, including
+ // the ones partition sync has not created yet, and no per-partition
requirement can express that.
+ // IVM only -- a non-IVM MV reaches the same effect through its
cleared snapshot, which its own
+ // refresh already depends on.
+ if (mtmv.isIvm() && !request.explicitPartitions
Review Comment:
**P2 (distinguish a whole-IVM invalidation from a generic `SCHEMA_CHANGE`) —
keeping the current behaviour, and here is why.**
The finding is that `processBaseTableChange` sets `SCHEMA_CHANGE` even when
re-analysis succeeds and no partition epoch was raised (`DROP COLUMN spare`),
and this branch then converts a strict `REFRESH ... INCREMENTAL` into a
whole-MV COMPLETE.
Narrowing it is not a diagnostic change: it hands the refresh to the
incremental path after a change that may have moved the normalized plan, and
that path then re-applies rows the COMPLETE had just rebuilt. This is the same
root as the plan-signature finding below — the evidence recorded there
(`test_ivm_snapshot`, `test_ivm_bitmap_agg_2`, `test_ivm_agg_array_1` each
writing their delta twice once the fresh signature is published) is what
happens when the incremental path takes over after a plan-level change. So
until the incremental path's baseline/offset handling is understood,
`SCHEMA_CHANGE` implies COMPLETE for an IVM MV is the protection rather than
the waste the finding describes — which is also why the design puts it there: a
schema-level invalidation is not a set of dirty partitions, it covers the
partitions partition sync has not created yet, and no per-partition requirement
can express that.
The second half of the finding is met. The RefreshMode /
IvmRebuiltPartitions assertion it asks for is in the regression suite:
`test_ivm_partition_epoch_rebuild` pins a `SCHEMA_CHANGE` escalation as
`SUCCESS / COMPLETE / 2` (status, refresh mode, rebuilt partitions), so a
strict INCREMENTAL that ran as a whole-MV rebuild is visible as such instead of
passing on SUCCESS alone.
Recording the decision here so it is not lost.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
