CalvinKirs opened a new pull request, #69966:
URL: https://github.com/apache/doris/pull/69966
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
Cloud MV partition compensation checks each valid mapped MV partition
through a singleton non-empty request. Cached-empty or unknown partitions can
therefore cause many serial MS version requests, repeated across related
partition tables, even when the MV is not ultimately selected.
Collect unique mapped valid MV partition IDs and call the existing bulk
interface in batches of at most 5,000. Reuse the returned ID set across related
tables, preserving partition removal, base-table compensation, cache handling
and exception propagation.
The 5,000-partition cap is a fixed code constant, not a runtime
configuration. It bounds the three request ID lists to at most 165,000 bytes at
the worst protobuf integer width, plus the remaining request fields. This is a
request-size guard, not a claimed performance-optimal batch size. Tests verify
that 1,020 candidates use one bulk call and 10,001 use three; these are mocked
interface call counts, not a live Cloud benchmark.
### Release note
Reduce Cloud query planning overhead when materialized views have many empty
partitions by batching non-empty partition checks with a bounded request size.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- `run-fe-ut.sh --run` with `PartitionCompensatorBatchTest`,
`PartitionCompensatorTest` and `CloudPartitionTest`: 29 tests passed; the
89-module reactor completed successfully.
- Cover batch boundaries, duplicate candidates, shared PCT mappings,
mixed/all-empty results, unmapped/invalid partitions and a failed second batch.
- FE Core Checkstyle: 0 violations; `git diff --check` passed.
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [x] No. Query semantics and partition compensation are preserved;
metadata request grouping changes.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
