kbendick commented on code in PR #5972:
URL: https://github.com/apache/iceberg/pull/5972#discussion_r996384083
##########
core/src/main/java/org/apache/iceberg/hadoop/HadoopFileIO.java:
##########
@@ -78,7 +78,7 @@ public OutputFile newOutputFile(String path) {
@Override
public void deleteFile(String path) {
- Path toDelete = new Path(path);
+ Path toDelete = new Path(path.replaceAll("s3", "s3a"));
Review Comment:
This this does seem to be done in many places though, if it's not presently
configurable in the general case, maybe we should add a function that
normalizes based off of some configuration?
I think that the configurable equivalency check that @flyrain added was for
file cleanup or some other procedure. So if it's not configurable at the
catalog level or the file io level, maybe that would be best to consider?
Though admittedly this change makes me a bit nervous as we have file names
hard coded in places.
That general hesitancy is why I'd like Yufei's input. I'm sure he had to
deal with most cases when he implemented equivalency checking for certain
procedures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
