vaultah opened a new issue, #13719: URL: https://github.com/apache/iceberg/issues/13719
### Apache Iceberg version 1.9.2 (latest release) ### Query engine None ### Please describe the bug 🐞 Iceberg's `rewrite_table_path` procedure (`RewriteTablePath` Spark action) does not update sizes of rewritten manifests in manifest lists that reference them. Specifically, when [`RewriteTablePathUtil.rewriteManifestList`](https://github.com/apache/iceberg/blob/95a578d7ad96aa3c82dccd305f556c83b3360528/core/src/main/java/org/apache/iceberg/RewriteTablePathUtil.java#L262) rewrites the manifest list, it copies the contained manifests and only updates their paths. [Manifest lists are rewritten first and manifest files are rewritten second](https://github.com/apache/iceberg/blob/main/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteTablePathSparkAction.java#L289-L297), so the utility has no way of knowing final sizes of the rewritten manifest files. This is very similar to https://github.com/apache/iceberg/issues/12554. Mismatch between manifest size in manifest list and size of the physical manifest file may cause correctness problems in some query engines. For example if the actual manifest file is larger than its size recorded in manifest lists, Trino may only read part of it and ignore the rest, effectively skipping manifest entries (referenced data and delete files) silently. This is reproducible in Iceberg 1.8.1 at least, and based on the source code this is also true for later versions. ### Willingness to contribute - [x] I can contribute a fix for this bug independently - [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [ ] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
