HappenLee opened a new pull request, #68826:
URL: https://github.com/apache/doris/pull/68826
### What problem does this PR solve?
Issue Number: None
Related PR: #68672 (replacement rebased onto master at
`157e03eeb16deb8a84ae58d9bed721f56f8fedb3`)
Problem Summary:
Condition cache can reuse false ranges for predicates whose results vary
between executions. Virtual-column predicates can also share a digest although
their separately stored definitions differ. For example, after `abs(k + 1) +
abs(k + 1) = 0`, a cached scan of `abs(k - 9999) + abs(k - 9999) = 0` can
incorrectly omit `k = 9999`.
Share expression eligibility with query cache, reject volatile scan
dependencies and runtime-filter probes (including implicit casts), and pass
scan eligibility to BE before digest generation. Treat seeded and unseeded
`array_shuffle`/`shuffle` as volatile, and recognize scalar and nested
TIME-to-date conversions while preserving valid NULL folding. Return a zero
digest for virtual-column references whose definitions are absent from the
digest.
Add a per-query `ConditionCacheLookup` profile counter for both misses and
hits. Regression coverage uses execution-specific profiles to verify that
unsafe scans bypass cache and deterministic scans retain cache hits.
This replacement preserves the original 24-file implementation. The rebase
conflict in `RuntimeFilterTranslatorBucketPruneTest` was resolved by retaining
both master's grouped non-blocking filter coverage and the original cache
eligibility tests.
### Release note
Prevent incorrect cached results for volatile predicates,
query-date-dependent TIME casts, and virtual-column predicates. Preserve NULL
folding for supported TIME casts and shuffle calls. Expose
`ConditionCacheLookup` in OLAP scan profiles.
Rolling upgrade: upgrade BE before FE. The new optional plan field defaults
to false, so new BE disables condition cache for old FE plans; old BE cannot
enforce the new FE eligibility field.
### Check List (For Author)
- Test:
- [x] Unit Test: `FE_UT_PARALLEL=48 bash run-fe-ut.sh --run
'org.apache.doris.nereids.util.CacheExpressionTest,org.apache.doris.planner.ConditionCacheEligibilityTest,org.apache.doris.nereids.glue.translator.RuntimeFilterTranslatorBucketPruneTest'`;
21 tests passed (8 + 6 + 7), including master's non-blocking grouped-filter
case.
- [x] Manual test: FE Checkstyle passed with zero violations;
clang-format/check-format 16.0.6 and BE build hygiene passed. Compared
added/removed lines in all 24 files against the original PR: identical.
- [ ] Regression test: existing regression and BE unit coverage is
preserved; BE unit tests and regression suites were not rerun locally for this
rebase. Full CI will be requested with `run buildall`.
- Behavior changed:
- [x] Yes. Unsafe scans bypass condition cache; deterministic scans keep
their eligibility. No additional behavior change is introduced by the rebase.
- Does this need documentation?
- [x] No. Correctness fix using existing cache switches; upgrade
considerations are documented above.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
