singhpk234 opened a new pull request, #5366: URL: https://github.com/apache/iceberg/pull/5366
### About the Change : Presently whenever a failure is observed in any of the completableFutures in `uploadParts()`, it causes further completable futures fail with stagings files deleted as part of `abortUpload` which result in FileNotFoundException when it tries to create a request for it using `RequestBody.fromFile(file)` https://github.com/apache/iceberg/blob/7a92bf5e31a995c61dc598d2a36f598d5c120756/aws/src/main/java/org/apache/iceberg/aws/s3/S3OutputStream.java#L306 I think this is what was also highlighted in @stevenzwu 's observation here (https://github.com/apache/iceberg/issues/4168#issuecomment-1123025380) This change attempts : * To avoid unnecessary deletes being called for all staging files on every subsequent `completableFutures` which fail due to fileNotFoundException. * Cancels the remaining futures in which would not be required as the abort is called, reduces number of calls to abort multi-part upload as well. ### Testing Done Existing UT's + a UT to see abort is called only once rather than atleast once. cc @rdblue @jackye1995 @RussellSpitzer @stevenzwu @amogh-jahagirdar @rajarshisarkar -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
