ayushtkn commented on code in PR #4880:
URL: https://github.com/apache/hive/pull/4880#discussion_r1399143374
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##########
@@ -1038,6 +1050,28 @@ static String encodeString(String rawString) {
return
HiveConf.EncoderDecoderFactory.URL_ENCODER_DECODER.encode(rawString);
}
+ static String getLocation(String location, String defaultLocation,
Configuration conf) {
+ if (location == null || defaultLocation == null) {
+ return location;
+ }
+ boolean maskDefaultLocation =
conf.getBoolean(HiveConf.ConfVars.HIVE_ICEBERG_MASK_DEFAULT_LOCATION.varname,
+ HiveConf.ConfVars.HIVE_ICEBERG_MASK_DEFAULT_LOCATION.defaultBoolVal);
+ try {
+ return maskDefaultLocation ?
+ getQualifiedPath(location,
conf).replaceFirst(getQualifiedPath(defaultLocation, conf),
+ TABLE_DEFAULT_LOCATION) :
+ location;
+ } catch (IOException e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ static String getQualifiedPath(String location, Configuration conf) throws
IOException {
+ Path path = new Path(location);
+ FileSystem fs = path.getFileSystem(conf);
Review Comment:
FileSystem is cached, it isn't created always, it is created once & next
time when you try to fetch it, it returns the same instance not a new one. In
the end, all the FS instances are closed for the UGI which were created by Hive
during the query execution, so there isn't no explicit need for closing the FS,
it will get closed in the end.
If we close it in the middle, similar issues will arise like in:
[HIVE-27447](https://issues.apache.org/jira/browse/HIVE-27447)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
