zhangbutao commented on code in PR #5504:
URL: https://github.com/apache/hive/pull/5504#discussion_r1950179197
##########
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/Warehouse.java:
##########
@@ -133,14 +133,8 @@ public FileSystem getFs(Path f) throws MetaException {
/**
- * Hadoop File System reverse lookups paths with raw ip addresses The File
- * System URI always contains the canonical DNS name of the Namenode.
- * Subsequently, operations on paths with raw ip addresses cause an exception
- * since they don't match the file system URI.
Review Comment:
Yes, this is a very old code&comment. :(
Bsed on the test
https://github.com/apache/hive/blob/master/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestWarehouseDnsPath.java
, I didn't find any clues about `Hadoop File System reverse lookups paths`.
But maybe this test can not cover the case which has multiple namenodes(hdfs
federation). I also don't have a hdfs federation cluster to have test. :(
The purpose of this code is to assume that s3 and hdfs have the same
absolute path acquisition method. That is to say
`hive.blobstore.supported.schemes` can also include hdfs, then this can also
reduce some hdfs rpc call. If this, we can simplify this code to remove the
property `hive.blobstore.supported.schemes` and let the optimization apply to
all schemas(s3a, hdfs) by default.
Now If we want to be careful about this optimization, we can just add `hdfs
` into `hive.blobstore.supported.schemes` in MetastoreConf, and people can
remove `hdfs `from this property if finding some issue about getting hdfs path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
