ayushtkn commented on a change in pull request #2088:
URL: https://github.com/apache/hadoop/pull/2088#discussion_r443489757
##########
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
##########
@@ -1258,40 +1260,52 @@ public FileStatus getFileStatus(Path f) throws
IOException {
FileStatus status =
((ChRootedFileSystem)link.getTargetFileSystem())
.getMyFs().getFileStatus(new Path(linkedPath));
- result[i++] = new FileStatus(status.getLen(), status.isDirectory(),
- status.getReplication(), status.getBlockSize(),
- status.getModificationTime(), status.getAccessTime(),
- status.getPermission(), status.getOwner(), status.getGroup(),
- null, path);
+ linkStatuses.add(
+ new FileStatus(status.getLen(), status.isDirectory(),
+ status.getReplication(), status.getBlockSize(),
+ status.getModificationTime(), status.getAccessTime(),
+ status.getPermission(), status.getOwner(),
+ status.getGroup(), null, path));
} catch (FileNotFoundException ex) {
LOG.warn("Cannot get one of the children's(" + path
+ ") target path(" + link.getTargetFileSystem().getUri()
+ ") file status.", ex);
throw ex;
}
} else {
- result[i++] =
+ internalDirStatuses.add(
new FileStatus(0, true, 0, 0, creationTime, creationTime,
PERMISSION_555, ugi.getShortUserName(),
- ugi.getPrimaryGroupName(), path);
+ ugi.getPrimaryGroupName(), path));
}
}
+ FileStatus[] internalDirStatusesMergedWithFallBack = internalDirStatuses
+ .toArray(new FileStatus[internalDirStatuses.size()]);
if (fallbackStatuses.length > 0) {
- return consolidateFileStatuses(fallbackStatuses, result);
+ internalDirStatusesMergedWithFallBack =
+ merge(fallbackStatuses, internalDirStatusesMergedWithFallBack);
+ }
+
+ // we don't use target file status as we show the mount link as symlink.
+ if (showMountLinksAsSymlinks) {
+ return merge(internalDirStatusesMergedWithFallBack,
+ linkStatuses.toArray(new FileStatus[linkStatuses.size()]));
Review comment:
Thanx @umamaheswararao for initiating this, The idea seems good. This
fallback mechanism is there in RBF too, through default namespace
implementation, shall be great to bring it to ViewFs as well.
I got the overall idea, Just a doubt regarding the above line.
Why in case `showMountLinkAsSymlinks` is true, we need to give preference to
the fallback paths?
Taking example from the structure in
`testLSOnLinkParentWithFallbackLinkWithSameMountDirectoryTree`
if we do ls on `/user1/hive/warehouse` the `linkStatuses` will have one
partition0 which is mounted and the `internalDirStatusesMergedWithFallBack`
will have another partition0 from the fallback. Shouldn't we give preference to
the one from the mount entry. Since if I perform an operation on
`user1/hive/warehouse/partiton0 the operation will be performed on the mount
link resolved path rather than the Fallback partion0?
Here collision is possible only between `linkStatus` and `fallbackStatus`,
`linkStatus` and `internalDirStatuses` can not collide since ViewFs isn't
allowing to mount /user1/hive -> (some path) as well as /user1/hive/warehouse
-> (some path) which is allowed in RBF. So, In case if the exact mount entry is
present any call will go to mount entry resolved path and the fallback dir will
never be used, so in listing also it could have been overwritten?
Can you help with logic here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
