924060929 opened a new pull request, #63612: URL: https://github.com/apache/doris/pull/63612
### What problem does this PR solve? Problem Summary: A query cached on a follower or observer can keep returning an old schema after a table change. For example, a heavy `ALTER TABLE ... DROP COLUMN` switches the base index without changing the table's data visible version. The master's ALTER path clears its local SQL cache, while another FE replays the schema change and retains the old cached result. This change invalidates derived query caches while applying existing DML/DDL journals. It replaces the earlier implementation of this PR that emitted a separate metadata-change journal. ### Implementation `Env.onTableChanged` is the shared entry point. Every cache invalidation runs in its own `try/catch`, logs a failure, and allows the remaining cache invalidations to run. Checkpoint replay skips these local derived caches. | Change | Invalidation | |---|---| | Table metadata | SQL result contexts, sorted partition ranges, dependent MTMV rewrite plans, and exact simple-aggregate caches | | Visible DML | SQL contexts whose table-wide or partition dependencies changed; unaffected partition dependencies remain reusable | | Database/catalog namespace | Matching SQL, sorted partition, and MTMV rewrite caches, including cold external objects | The hook runs at the existing mutation boundary: schema-change/rollup completion, transaction visibility, table drop/replacement, and truncate. Remaining replay handlers notify it after applying their mutation. Renames and replacements notify after applying the name mapping, preserving the original names and table IDs under the existing locks. Batch partition changes notify each affected table once. Replay coverage includes table/view creation and removal, recovery, column/table/partition/rollup renames, heavy/light schema and index changes, partition changes/replacement, table replacement/engine/properties/distribution changes, administrative version changes, constraints, MTMV changes, branch/tag operations, and database/catalog changes. External refreshes reuse their existing invalidation protocol and retain the row-count cache's final invalidation barrier. SQL cache publication and lookup share dependency validation: local metadata revisions, table identity/schema/data versions, view definitions, and external catalog properties. Revisions are scoped by table ID/name, database and catalog; each statement retains its own dependency snapshot, with no registry of executing contexts. Publication uses a fresh statement and connector scope, restores the executing statement in finally, and skips cache admission if validation fails. Lookup also rejects a late put that races invalidation. The BE cache key includes the base index/schema identity, and MTMV rewrite invalidation reuses the existing generation barrier. No journal opcode, extra journal record, persisted payload, or FE/BE protocol field is added. Approximate statistics and transaction-scoped auto-partition routing retain their existing lifecycles. ### Release note Fix stale cached query results on follower and observer FEs after table metadata changes. ### Check List (For Author) - Test: Latest-head preset FE UT run: 276 tests across 21 focused classes, 0 failures/errors/skips; FE Checkstyle: 0 violations. Includes FE/BE publication, forced late put, namespace and first-binding revision checks, unchanged-partition controls, fresh external branch/scope validation, failure isolation, and rename/replace notification ordering. Earlier multi-FE regression validation is described below; it was not rerun for this refactor. - Behavior changed: Yes. Existing replayed table changes invalidate their local dependent query caches. - Does this need documentation: No Earlier local multi-FE validation used the then-patched master FE with an existing 4.1.3 BE. The isolated cluster sets `be_exec_version=10` and disables the newer local shuffle planner/local exchange to accommodate that BE. This is not a same-head FE/BE build or a Cloud cluster validation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
