CalvinKirs opened a new pull request, #66533:
URL: https://github.com/apache/doris/pull/66533
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
External metadata caches are currently bounded only by entry count, so
entries with very different memory footprints are treated equally. This PR adds
framework support for an optional catalog-level `max-weight`, an entry-specific
size estimator contract, mutually exclusive Caffeine size/weight construction,
saturated integer weight conversion, and weighted cache statistics.
It also completes the framework by exposing byte-explicit weight metrics
through `information_schema.catalog_meta_cache_statistics`, supporting weighted
caches with synchronous removal listeners, accepting binary size suffixes such
as `512MB` while retaining bare-byte compatibility, strictly rejecting
malformed, negative, and overflowing values, and keeping statistics reads
lightweight and side-effect free.
Existing entries continue to use `maximumSize` unless they explicitly
register an estimator and configure `max-weight`. Catalog-specific Iceberg and
Paimon estimators are intentionally not included.
### Release note
Support unit-suffixed metadata cache `max-weight` values and expose
`MAX_WEIGHT_BYTES`, `ESTIMATED_WEIGHT_BYTES`, and `EVICTION_WEIGHT_BYTES` in
`information_schema.catalog_meta_cache_statistics`.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- `./run-fe-ut.sh --run
org.apache.doris.common.CacheFactoryTest,org.apache.doris.datasource.metacache.CacheSpecTest,org.apache.doris.datasource.metacache.MetaCacheEntryTest,org.apache.doris.datasource.metacache.AbstractExternalMetaCacheTest,org.apache.doris.catalog.SchemaTableTest,org.apache.doris.tablefunction.MetadataGeneratorTest`
(81 tests passed)
- `./run-fe-ut.sh --run
org.apache.doris.datasource.metacache.CacheSpecTest` (10 tests passed)
- `DISABLE_BUILD_UI=ON ./build.sh --fe` (passed, including
Checkstyle)
- [ ] Manual test
- [ ] No need to test or manual test
- Behavior changed:
- [ ] No.
- [x] Yes. `max-weight` accepts binary size suffixes, invalid values are
rejected, byte-explicit weight statistics are exposed, and statistics reads no
longer trigger Caffeine maintenance.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
