abstractdog commented on code in PR #4753:
URL: https://github.com/apache/hive/pull/4753#discussion_r1345271568
##########
common/src/java/org/apache/hadoop/hive/common/FileUtils.java:
##########
@@ -1387,6 +1388,48 @@ public static RemoteIterator<LocatedFileStatus>
listFiles(FileSystem fs, Path pa
status -> filter.accept(status.getPath()));
}
+ /**
+ * Resolves a symlink on a local filesystem. In case of any exceptions or
scheme other than "file"
+ * it simply returns the original path. Refer to DEBUG level logs for
further details.
+ * @param path input path to be resolved
+ * @param conf a Configuration instance to be used while resolving the
FileSystem
+ * @return the resolved target Path or the original if the input Path is not
a symlink
+ * @throws IOException
+ */
+ public static Path resolveSymlinks(Path path, Configuration conf) throws
IOException {
+ if (path == null) {
+ throw new IllegalArgumentException("Cannot resolve symlink for a null
Path");
+ }
+
+ String scheme = path.toUri().getScheme();
+
+ /*
+ * If you're about to extend this method to e.g. HDFS, simply remove this
check.
+ * There is a known exception reproduced by whroot_external1.q, which can
be referred to,
+ * which is because java.nio is not prepared by default for other schemes
like "hdfs".
+ */
+ if (!"file".equalsIgnoreCase(scheme)) {
+ LOG.debug("scheme '{}' is not supported for resolving symlinks", scheme);
+ return path;
+ }
+
+ FileSystem srcFs;
+ if (scheme != null) {
+ srcFs = path.getFileSystem(conf);
+ } else {
+ srcFs = FileSystem.getLocal(conf);
+ }
Review Comment:
okay, also, I'll add a unit test for paths without scheme, I believe in that
case we need to resolve the symlink but return with a path without a scheme also
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
