steveloughran commented on PR #6726: URL: https://github.com/apache/hadoop/pull/6726#issuecomment-2059138849
commented. I've also done a PR #6738 which tunes the API to work with iceberg, having just written a PoC of the iceberg binding. My PR * moved the wrapper methods to a new wrappedio.WrappedIO class * add a probe for the api being available * I also added an availability probe in the interface. not sure about that as we really should make it available everywhere, always. Can you cherrypick this PR onto your branch and then do the review comments. After which, please do not do any rebasing of your PR. That way, it is easier for me too keep my own branch in sync with your changes. Thanks. PoC of iceberg integration, based on their S3FileIO one. https://github.com/steveloughran/iceberg/blob/s3/HADOOP-18679-bulk-delete-api/core/src/main/java/org/apache/iceberg/hadoop/HadoopFileIO.java#L208 The iceberg api passes in a collection of paths, *which may span multiple filesystems*. To handle this, * the bulk delete API should take a Collection, not a list * it needs to be implemented in every FS, because trying to distinguish case-by-case on support would be really complex. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
