CalvinKirs opened a new pull request, #69883:
URL: https://github.com/apache/doris/pull/69883
### What problem does this PR solve?
Issue Number: None
Problem Summary:
SQL cache currently invalidates an OLAP query whenever the table visible
version changes. For example, a cached query reading historical partition p1 is
discarded after an insert into unrelated partition p2, even though its result
is unchanged.
Record the partition versions actually assigned to the scan, including
selected empty partitions removed during planning. When the table version
advances, reuse the cache only if those dependencies and the schema/partition
topology still match. Use the same stable dependency identity for the BE cache
key. Keep table-wide validation for metadata aggregates, variable-dependent
keys and materialized-view rewrites whose physical scans can hide source
dependencies.
The change is FE-only and preserves the existing BE cache protocol.
Regression cases use only 3–5 source rows and cover unrelated/relevant writes,
empty partitions and branches, self joins, partition DDL, partial MV rewrites
and eliminated empty MVs.
### Release note
Fix unnecessary SQL cache invalidation after writes to unrelated OLAP
partitions.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [ ] Manual test
- [ ] No need to test or manual test.
Validation:
- FE build and Checkstyle passed. The functional patch passed 29 FE unit
tests in 5 classes and 5 regression suites: `sql_cache_partition_version`,
`sql_cache_partial_mtmv`, `sql_cache_empty_mtmv`, `sql_cache` and
`mtmv_with_sql_cache`.
- The final behavior-preserving cleanup additionally passed FE Checkstyle
and
`OlapTableSqlCacheVersionTest#testVariableAwareCacheKeysRetainTableVersionInBothBindingOrders`.
The full 29-test/5-suite selection was not rerun after that cleanup.
- Cloud version-refresh races are covered with deterministic mocked MS
responses; no live cloud-cluster validation was performed. Regression tests
reused an existing BE binary.
- Behavior changed:
- [ ] No.
- [x] Yes. Queries can continue using SQL cache after unrelated
partition writes; actual data dependencies and metadata changes still
invalidate it.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
