zhoulii opened a new issue, #10709: URL: https://github.com/apache/paimon/issues/10709
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation Snapshot expiration loads entire index manifests into `List<IndexManifestEntry>` before filtering, and deletion tasks retain full entries until execution. Processing multiple large manifests concurrently can cause excessive heap usage and OOM. Building the set of protected index files also loads complete metadata just to extract file names. <img width="953" height="410" alt="Image" src="https://github.com/user-attachments/assets/689c8f9f-a330-42ad-a449-03f4af47f4b9"; /> ### Solution - Reuse the existing projected streaming scan: read only file names when collecting protected files, and only the fields needed to resolve paths when planning deletion. - Filter entries as they are read and retain only `Path` objects in deletion tasks. - Bypass the full manifest cache to avoid loading unnecessary metadata, such as deletion vector ranges and global index metadata. ### Anything else? Builds on the projected scan API from [#9095](https://github.com/apache/paimon/pull/9095). The manifest format and file retention semantics remain unchanged. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
