zclllyybb commented on issue #65775: URL: https://github.com/apache/doris/issues/65775#issuecomment-5003439147
Breakwater-GitHub-Analysis-Slot: slot_e33f6267bed5 This content is generated by AI for reference only. I looked at this against the 4.1.3 FE sync-MV code path. The two reported symptoms look related at the feature level, but they are probably two different problems. 1. The minimal `CREATE MATERIALIZED VIEW` failure is explained by the current sync-MV DDL validation. In `CreateMaterializedViewCommand.validate()` -> `MaterializedViewHandler.processCreateMaterializedView()`, Doris builds the MV output items and then checks every MV output column name against the base table full schema. Therefore: ```sql SELECT k1, k2, MAX(v) AS max_v FROM tmp_mv_probe GROUP BY k1, k2 ``` produces MV output names `k1`, `k2`, `max_v`; `k1` and `k2` already exist in the base table, so the handler raises `ERR_DUP_FIELDNAME`. This is not caused by `AUTO PARTITION` or generated columns. It is a name-collision check before the rollup schema is built. If this restriction is intended, the error should be clearer, for example that sync MV output column names must not duplicate base table column names. If it is not intended, the duplicate check should be narrowed so valid grouped key outputs do not collide with their own base columns. A current workaround is to alias all MV outputs that would otherwise reuse base column names: ```sql CREATE MATERIALIZED VIEW mv_probe AS SELECT k1 AS mv_k1, k2 AS mv_k2, MAX(v) AS max_v FROM tmp_mv_probe GROUP BY k1, k2; ``` 2. The rewrite miss for the already-created aliased MV is not explained by the minimal DDL failure. The production MV aliases the key columns, so it avoids the duplicate-name check above. In 4.1.3, `EXPLAIN` with `enable_materialized_view_rewrite=true` installs `InitMaterializationContextHook`; visible non-base sync MV indexes are collected as candidates, the stored CREATE MV SQL is reparsed, and `SyncMaterializationContext` can generate a scan using the MV index id. So a `FINISHED` MV still scanning the base table means one of these happened: - the sync MV context was not created or was marked unavailable, for example a parse/init exception in FE logs; - the MV rewrite failed to match the query shape; - the rewrite succeeded, but the final costed plan did not choose the MV index. Normal `EXPLAIN` only shows the final scan, so it does not identify which of those happened. Please attach: - `EXPLAIN MEMO <query>` or `EXPLAIN OPTIMIZED <query>` for the failing query, with the same `enable_materialized_view_rewrite` and `pre_materialized_view_rewrite_strategy` settings. The `========== MATERIALIZATIONS ==========` section should show whether `mv_tick_dirty_source_watermark` was `chose`, `not chose`, or `fail`. - `SHOW CREATE TABLE tick_dirty_minute;` - `SHOW CREATE MATERIALIZED VIEW mv_tick_dirty_source_watermark ON tick_dirty_minute;` - `SHOW ALTER TABLE MATERIALIZED VIEW WHERE TableName='tick_dirty_minute' ORDER BY CreateTime DESC LIMIT 5;` - whether `ANALYZE TABLE tick_dirty_minute WITH SYNC;` has been run after the MV reached `FINISHED` and after the Routine Load data was visible. Existing sync-MV rewrite regression checks normally make row counts/statistics ready before asserting that the MV is finally chosen. - any FE WARN log around the EXPLAIN query id containing `createSyncMvContexts`, `can't parse`, `MaterializationContext`, or the MV name. If `EXPLAIN MEMO` says the MV `fail`s for the identical aliased query, this should be treated as a rewrite matching bug. If it says `not chose`, the next step is to inspect stale/missing table/index statistics and CBO costing. If the MV is absent from the materialization summary, focus on candidate initialization or stored MV metadata. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
