[jira] [Commented] (HDFS-5294) DistributedFileSystem#getFileLinkStatus should not fully qualify the link target
[
https://issues.apache.org/jira/browse/HDFS-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13785431#comment-13785431
]
Andrew Wang commented on HDFS-5294:
---
Same criticism also applies to {{#getLinkTarget}}. We qualify FileStatus paths
everywhere right now, which I agree makes it painful for wrapper-FSes like
{{ViewFileSystem}}.
> DistributedFileSystem#getFileLinkStatus should not fully qualify the link
> target
>
>
> Key: HDFS-5294
> URL: https://issues.apache.org/jira/browse/HDFS-5294
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>
> The NN returns a {{FileStatus}} containing the exact link target as specified
> by the user at creation. However,
> {{DistributedFileSystem#getFileLinkStatus}} explicit overwrites the target
> with the fully scheme qualified path lookup. This causes multiple issues
> such as:
> # Prevents clients from discerning if the target is relative or absolute
> # Mangles a target that is not intended to be a path
> # Causes incorrect resolution with multi-layered filesystems - ie. the link
> should be resolved relative to a higher level fs (ie. viewfs, chroot,
> filtered, etc)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-5294) DistributedFileSystem#getFileLinkStatus should not fully qualify the link target
[
https://issues.apache.org/jira/browse/HDFS-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13785734#comment-13785734
]
Colin Patrick McCabe commented on HDFS-5294:
There are a ton of FileSystem operations that return paths. If we're going to
switch them all to return unresolve dpaths, that will also affect
getFileStatus, listStatus, getLocatedFileStatus, resolvePath,
listCorruptFileBlocks, globStatus, createSnapshot, etc.
Also, as we've discussed elsewhere, doing this would be a large performance
regression, huge in some cases.
I really don't think we should do this unless we also do symlink resolution
server-side to avoid doing N symlink resolution RPCs every time we use a path
with N symlinks in it.
> DistributedFileSystem#getFileLinkStatus should not fully qualify the link
> target
>
>
> Key: HDFS-5294
> URL: https://issues.apache.org/jira/browse/HDFS-5294
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>
> The NN returns a {{FileStatus}} containing the exact link target as specified
> by the user at creation. However,
> {{DistributedFileSystem#getFileLinkStatus}} explicit overwrites the target
> with the fully scheme qualified path lookup. This causes multiple issues
> such as:
> # Prevents clients from discerning if the target is relative or absolute
> # Mangles a target that is not intended to be a path
> # Causes incorrect resolution with multi-layered filesystems - ie. the link
> should be resolved relative to a higher level fs (ie. viewfs, chroot,
> filtered, etc)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-5294) DistributedFileSystem#getFileLinkStatus should not fully qualify the link target
[
https://issues.apache.org/jira/browse/HDFS-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13785812#comment-13785812
]
Colin Patrick McCabe commented on HDFS-5294:
This is a duplicate of the earlier HADOOP-9780. I propose that we close this
issue as duplicate and move the discussion to there. If we do end up making
this change, it will affect more than just HDFS, so it doesn't make sense to
have this be an HDFS (as opposed to common JIRA) anyway.
> DistributedFileSystem#getFileLinkStatus should not fully qualify the link
> target
>
>
> Key: HDFS-5294
> URL: https://issues.apache.org/jira/browse/HDFS-5294
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>
> The NN returns a {{FileStatus}} containing the exact link target as specified
> by the user at creation. However,
> {{DistributedFileSystem#getFileLinkStatus}} explicit overwrites the target
> with the fully scheme qualified path lookup. This causes multiple issues
> such as:
> # Prevents clients from discerning if the target is relative or absolute
> # Mangles a target that is not intended to be a path
> # Causes incorrect resolution with multi-layered filesystems - ie. the link
> should be resolved relative to a higher level fs (ie. viewfs, chroot,
> filtered, etc)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-5294) DistributedFileSystem#getFileLinkStatus should not fully qualify the link target
[
https://issues.apache.org/jira/browse/HDFS-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786311#comment-13786311
]
Daryn Sharp commented on HDFS-5294:
---
It's a dup in concept, but addressing the issue described will require a
specific change to {{DistributedFileSystem}}'s {{getFileLinkStatus}} and
{{getLinkTarget}} so I think it should stay open in addition to the common jira.
bq. [...] t will also affect getFileStatus, listStatus, getLocatedFileStatus,
resolvePath, listCorruptFileBlocks, globStatus, createSnapshot, etc. [...] I
really don't think we should do this unless we also do symlink resolution
server-side to avoid doing N symlink resolution RPCs every time we use a path
with N symlinks in it.
I think there's some misunderstanding. The issue of whether the client is able
to obtain the exact symlink target is orthogonal from whether the other methods
return a {{FileStatus}} with a (un)qualified path.
> DistributedFileSystem#getFileLinkStatus should not fully qualify the link
> target
>
>
> Key: HDFS-5294
> URL: https://issues.apache.org/jira/browse/HDFS-5294
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>
> The NN returns a {{FileStatus}} containing the exact link target as specified
> by the user at creation. However,
> {{DistributedFileSystem#getFileLinkStatus}} explicit overwrites the target
> with the fully scheme qualified path lookup. This causes multiple issues
> such as:
> # Prevents clients from discerning if the target is relative or absolute
> # Mangles a target that is not intended to be a path
> # Causes incorrect resolution with multi-layered filesystems - ie. the link
> should be resolved relative to a higher level fs (ie. viewfs, chroot,
> filtered, etc)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-5294) DistributedFileSystem#getFileLinkStatus should not fully qualify the link target
[
https://issues.apache.org/jira/browse/HDFS-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786825#comment-13786825
]
Colin Patrick McCabe commented on HDFS-5294:
OK. If I understand correctly, it sounds like you're proposing that
{{getFileLinkStatus}} return symlinks with relative links as relative, rather
than trying to qualify them. I am +1 on this idea, but if we do it, we need to
do it for all Filesystems, not just HDFS.
> DistributedFileSystem#getFileLinkStatus should not fully qualify the link
> target
>
>
> Key: HDFS-5294
> URL: https://issues.apache.org/jira/browse/HDFS-5294
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>
> The NN returns a {{FileStatus}} containing the exact link target as specified
> by the user at creation. However,
> {{DistributedFileSystem#getFileLinkStatus}} explicit overwrites the target
> with the fully scheme qualified path lookup. This causes multiple issues
> such as:
> # Prevents clients from discerning if the target is relative or absolute
> # Mangles a target that is not intended to be a path
> # Causes incorrect resolution with multi-layered filesystems - ie. the link
> should be resolved relative to a higher level fs (ie. viewfs, chroot,
> filtered, etc)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-5294) DistributedFileSystem#getFileLinkStatus should not fully qualify the link target
[
https://issues.apache.org/jira/browse/HDFS-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13788190#comment-13788190
]
Daryn Sharp commented on HDFS-5294:
---
Yes, that's correct. The symlink target should simply be returned as-is.
Agreed that it should be the behavior for all filesystems.
> DistributedFileSystem#getFileLinkStatus should not fully qualify the link
> target
>
>
> Key: HDFS-5294
> URL: https://issues.apache.org/jira/browse/HDFS-5294
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>
> The NN returns a {{FileStatus}} containing the exact link target as specified
> by the user at creation. However,
> {{DistributedFileSystem#getFileLinkStatus}} explicit overwrites the target
> with the fully scheme qualified path lookup. This causes multiple issues
> such as:
> # Prevents clients from discerning if the target is relative or absolute
> # Mangles a target that is not intended to be a path
> # Causes incorrect resolution with multi-layered filesystems - ie. the link
> should be resolved relative to a higher level fs (ie. viewfs, chroot,
> filtered, etc)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
