cccs-jc commented on a change in pull request #1979:
URL: https://github.com/apache/iceberg/pull/1979#discussion_r549734182
##########
File path: core/src/main/java/org/apache/iceberg/hadoop/HadoopCatalog.java
##########
@@ -139,6 +139,25 @@ public String name() {
return catalogName;
}
+ private boolean isTableDir(Path path) {
+ Path metadataPath = new Path(path, "metadata");
+ // Only the path which contains metadata is the path for table, otherwise
it could be
+ // still a namespace.
+ try {
+ return fs.listStatus(metadataPath, TABLE_FILTER).length >= 1;
+ } catch (IOException e) {
+ return false;
Review comment:
We use the iceberg HadoopCatalog over Azure ABFS, Users use spark to
list the tables in the catalog. However not every user has access to all tables
on disk (we use file system access control list). When a user cannot access a
folder the listStatus function will throw an IOException.
In my ticket I suggested using a flag to configure the HadoopCatalog to
throw errors when unable to list these directories or return false if it is
expected that some users will not be able to list all folders/tables.
What do you think about that proposal?
[https://github.com/apache/iceberg/issues/1941](https://github.com/apache/iceberg/issues/1941)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
