szehon-ho commented on issue #5339: URL: https://github.com/apache/iceberg/issues/5339#issuecomment-1193071314
From my knowledge, when you have a Table object, you usually have in the memory all the ManifestFiles, but that's as far as it goes. What happens during commit is a bit trickier (can lose a lot of time in that code :). There are a few cases of reading old manifests and checking their DataFiles, like 1) in delete operation to check if we can delete any DataFile entirely from the delete expression, or 2) to merge a new manifest with older ones, but I'd be surprised if we load DataFile entries that we don't need. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
