hubgeter opened a new pull request, #70276:
URL: https://github.com/apache/doris/pull/70276
### What problem does this PR solve?
Issue Number: N/A
Related PR: #64304
Problem Summary:
On branch-4.2, `expire_snapshots` reads each snapshot's delete manifests
before classifying deleted files. Adjacent snapshots can share an immutable
manifest, so the same manifest is opened and decoded repeatedly. The existing
`putIfAbsent` only deduplicates the resulting delete-file entries.
Track manifest paths within each `buildDeleteFileContentMap` call and read
each distinct delete manifest once. This adapts the existing master
optimization to the legacy FE implementation without bringing over the
connector migration. Classification still uses the table's spec map, and read
failures still abort the operation.
### Release note
Reduce redundant delete manifest reads when expiring Iceberg snapshots.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test: `IcebergExpireSnapshotsActionTest`,
`BaseIcebergActionTest`, and `IcebergSnapshotMaintenanceCoverageTest` via
`run-fe-ut.sh --run`: 11 passed, 2 pre-existing disabled stale-metadata tests
skipped, 0 failures.
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
The new shared-manifest test fails before the fix (an input stream opens 3
times instead of 1) and passes after it. The tests count actual input-stream
opens, cover position/equality deletes across partition specs, preserve the
exact classification map, and check repeated calls and read-failure
propagation. Both changed Java files pass Checkstyle. No SQL regression was
added because the optimization preserves SQL results; the I/O reduction is
asserted at the FileIO boundary.
- Behavior changed:
- [ ] No.
- [x] Yes. Shared delete manifests are read once per classification
call; file classification and cleanup semantics are preserved.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
