[jira] [Commented] (HDFS-6373) Remove support for extended attributes on symlinks
[
https://issues.apache.org/jira/browse/HDFS-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13998533#comment-13998533
]
Charles Lamb commented on HDFS-6373:
What I can't figure out is where exactly in the code the stack trace is
being put into the log.
> Remove support for extended attributes on symlinks
> --
>
> Key: HDFS-6373
> URL: https://issues.apache.org/jira/browse/HDFS-6373
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
>Reporter: Andrew Wang
>Assignee: Charles Lamb
> Fix For: HDFS XAttrs (HDFS-2006)
>
> Attachments: HDFS-6373.1.patch, HDFS-6373.2.patch
>
>
> Looking in the Linux source code, we see the following:
> http://lxr.linux.no/linux+v3.14.3/fs/xattr.c
> {code}
> 60/*
> 61 * In the user.* namespace, only regular files and directories
> can have
> 62 * extended attributes. For sticky directories, only the owner and
> 63 * privileged users can write attributes.
> 64 */
> {code}
> We should consider removing {{XAttrFeature}} from {{INodeSymlink}}.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6373) Remove support for extended attributes on symlinks
[
https://issues.apache.org/jira/browse/HDFS-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996992#comment-13996992
]
Andrew Wang commented on HDFS-6373:
---
+1 on the patch, thanks Charles.
Thanks for the confirmation also Chris, I guess this is really just for code
consistency for INodeSymlink. I filed HDFS-6396 and assigned it to Charles so
he can do a similar fix.
> Remove support for extended attributes on symlinks
> --
>
> Key: HDFS-6373
> URL: https://issues.apache.org/jira/browse/HDFS-6373
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
>Reporter: Andrew Wang
>Assignee: Charles Lamb
> Attachments: HDFS-6373.1.patch, HDFS-6373.2.patch
>
>
> Looking in the Linux source code, we see the following:
> http://lxr.linux.no/linux+v3.14.3/fs/xattr.c
> {code}
> 60/*
> 61 * In the user.* namespace, only regular files and directories
> can have
> 62 * extended attributes. For sticky directories, only the owner and
> 63 * privileged users can write attributes.
> 64 */
> {code}
> We should consider removing {{XAttrFeature}} from {{INodeSymlink}}.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6373) Remove support for extended attributes on symlinks
[
https://issues.apache.org/jira/browse/HDFS-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995598#comment-13995598
]
Andrew Wang commented on HDFS-6373:
---
Related, I also found this page, which indicates that having ACLs on symlinks
is dubious:
http://0xced.blogspot.com/2009/03/chmod-acl-and-symbolic-links_23.html
This makes sense to me, since symlink permissions are always ignored. Linux
actually doesn't even let you change symlink permissions, though it is allowed
by the Unix spec. [~cnauroth], could you comment on whether this was
intentional, or just an oversight? It seems like we might want to remove
ACLs-on-symlinks, which is compatible since symlinks are still hard-disabled.
> Remove support for extended attributes on symlinks
> --
>
> Key: HDFS-6373
> URL: https://issues.apache.org/jira/browse/HDFS-6373
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
>Reporter: Andrew Wang
>
> Looking in the Linux source code, we see the following:
> http://lxr.linux.no/linux+v3.14.3/fs/xattr.c
> {code}
> 60/*
> 61 * In the user.* namespace, only regular files and directories
> can have
> 62 * extended attributes. For sticky directories, only the owner and
> 63 * privileged users can write attributes.
> 64 */
> {code}
> We should consider removing {{XAttrFeature}} from {{INodeSymlink}}.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6373) Remove support for extended attributes on symlinks
[
https://issues.apache.org/jira/browse/HDFS-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996856#comment-13996856
]
Andrew Wang commented on HDFS-6373:
---
That stacktrace is probably just from how symlink resolution works. The client
code catches the UnresolvedLinkException, attempts to resolve it, then retries
with the new path. Maybe we should squish the trace since this is expected
behavior.
> Remove support for extended attributes on symlinks
> --
>
> Key: HDFS-6373
> URL: https://issues.apache.org/jira/browse/HDFS-6373
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
>Reporter: Andrew Wang
>Assignee: Charles Lamb
>
> Looking in the Linux source code, we see the following:
> http://lxr.linux.no/linux+v3.14.3/fs/xattr.c
> {code}
> 60/*
> 61 * In the user.* namespace, only regular files and directories
> can have
> 62 * extended attributes. For sticky directories, only the owner and
> 63 * privileged users can write attributes.
> 64 */
> {code}
> We should consider removing {{XAttrFeature}} from {{INodeSymlink}}.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6373) Remove support for extended attributes on symlinks
[
https://issues.apache.org/jira/browse/HDFS-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996975#comment-13996975
]
Chris Nauroth commented on HDFS-6373:
-
{quote}
This makes sense to me, since symlink permissions are always ignored. Linux
actually doesn't even let you change symlink permissions, though it is allowed
by the Unix spec. Chris Nauroth, could you comment on whether this was
intentional, or just an oversight? It seems like we might want to remove
ACLs-on-symlinks, which is compatible since symlinks are still hard-disabled.
{quote}
It is impossible to set an ACL on a symlink. Any ACL operation run on a
symlink automatically dereferences the symlink and operates on its target, i.e.
by passing {{true}} to {{FSDirectory#getINodesInPath4Write}}. We wrote tests
asserting this behavior, so I don't think there is any way to store an
{{AclFeature}} on an {{INodeSymlink}}.
I quickly skimmed the patch, and it looks like you're going for some additional
prevention inside {{INodeSymlink}} to prevent mistakes. That seems like a good
idea, and we could do similar for ACLs.
> Remove support for extended attributes on symlinks
> --
>
> Key: HDFS-6373
> URL: https://issues.apache.org/jira/browse/HDFS-6373
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
>Reporter: Andrew Wang
>Assignee: Charles Lamb
> Attachments: HDFS-6373.1.patch, HDFS-6373.2.patch
>
>
> Looking in the Linux source code, we see the following:
> http://lxr.linux.no/linux+v3.14.3/fs/xattr.c
> {code}
> 60/*
> 61 * In the user.* namespace, only regular files and directories
> can have
> 62 * extended attributes. For sticky directories, only the owner and
> 63 * privileged users can write attributes.
> 64 */
> {code}
> We should consider removing {{XAttrFeature}} from {{INodeSymlink}}.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6373) Remove support for extended attributes on symlinks
[
https://issues.apache.org/jira/browse/HDFS-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996912#comment-13996912
]
Charles Lamb commented on HDFS-6373:
Yeah, good idea. I've uploaded a new patch.
> Remove support for extended attributes on symlinks
> --
>
> Key: HDFS-6373
> URL: https://issues.apache.org/jira/browse/HDFS-6373
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
>Reporter: Andrew Wang
>Assignee: Charles Lamb
> Attachments: HDFS-6373.1.patch, HDFS-6373.2.patch
>
>
> Looking in the Linux source code, we see the following:
> http://lxr.linux.no/linux+v3.14.3/fs/xattr.c
> {code}
> 60/*
> 61 * In the user.* namespace, only regular files and directories
> can have
> 62 * extended attributes. For sticky directories, only the owner and
> 63 * privileged users can write attributes.
> 64 */
> {code}
> We should consider removing {{XAttrFeature}} from {{INodeSymlink}}.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6373) Remove support for extended attributes on symlinks
[
https://issues.apache.org/jira/browse/HDFS-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996905#comment-13996905
]
Andrew Wang commented on HDFS-6373:
---
This is basically what I had in mind, but how do you feel about throwing some
kind of RuntimeException, e.g. UnsupportedOperationException? It should prevent
anyone from trying this in the future.
> Remove support for extended attributes on symlinks
> --
>
> Key: HDFS-6373
> URL: https://issues.apache.org/jira/browse/HDFS-6373
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
>Reporter: Andrew Wang
>Assignee: Charles Lamb
> Attachments: HDFS-6373.1.patch
>
>
> Looking in the Linux source code, we see the following:
> http://lxr.linux.no/linux+v3.14.3/fs/xattr.c
> {code}
> 60/*
> 61 * In the user.* namespace, only regular files and directories
> can have
> 62 * extended attributes. For sticky directories, only the owner and
> 63 * privileged users can write attributes.
> 64 */
> {code}
> We should consider removing {{XAttrFeature}} from {{INodeSymlink}}.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6373) Remove support for extended attributes on symlinks
[
https://issues.apache.org/jira/browse/HDFS-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996425#comment-13996425
]
Charles Lamb commented on HDFS-6373:
While investigating this, I noticed this in the TestNameNodeXAttr output. I
suspect it's not intended, but I'll investigate further:
2014-05-13 10:04:33,584 INFO hdfs.StateChange
(FSNamesystem.java:completeFile(3028)) - DIR* completeFile: /symdir2/target is
closed by DFSClient_NONMAPREDUCE_239063403_1
2014-05-13 10:04:33,590 INFO FSNamesystem.audit
(FSNamesystem.java:logAuditMessage(7980)) - allowed=true ugi=cwl
(auth:SIMPLE) ip=/127.0.0.1 cmd=createSymlink src=/symdir1/link
dst=/symdir2/target perm=cwl:supergroup:rwxrwxrwx proto=rpc
2014-05-13 10:04:33,601 INFO FSNamesystem.audit
(FSNamesystem.java:logAuditMessage(7980)) - allowed=true ugi=cwl
(auth:SIMPLE) ip=/127.0.0.1 cmd=setXAttrsrc=/symdir2/target
dst=nullperm=cwl:supergroup:rw-r--r-- proto=rpc
2014-05-13 10:04:33,605 INFO FSNamesystem.audit
(FSNamesystem.java:logAuditMessage(7980)) - allowed=true ugi=cwl
(auth:SIMPLE) ip=/127.0.0.1 cmd=setXAttrsrc=/symdir2/target
dst=nullperm=cwl:supergroup:rw-r--r-- proto=rpc
2014-05-13 10:04:33,608 INFO ipc.Server (Server.java:run(2120)) - IPC Server
handler 9 on 38622, call
org.apache.hadoop.hdfs.protocol.ClientProtocol.getXAttrs from 127.0.0.1:41722
Call#18 Retry#0
org.apache.hadoop.hdfs.protocol.UnresolvedPathException: /symdir2/target
at
org.apache.hadoop.hdfs.server.namenode.INodesInPath.resolve(INodesInPath.java:203)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.getLastINodeInPath(FSDirectory.java:3135)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.getXAttrs(FSDirectory.java:2952)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getXAttrs(FSNamesystem.java:7858)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getXAttrs(NameNodeRpcServer.java:1396)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getXAttrs(ClientNamenodeProtocolServerSideTranslatorPB.java:1294)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:605)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2099)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2095)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1608)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2093)
> Remove support for extended attributes on symlinks
> --
>
> Key: HDFS-6373
> URL: https://issues.apache.org/jira/browse/HDFS-6373
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
>Reporter: Andrew Wang
>Assignee: Charles Lamb
>
> Looking in the Linux source code, we see the following:
> http://lxr.linux.no/linux+v3.14.3/fs/xattr.c
> {code}
> 60/*
> 61 * In the user.* namespace, only regular files and directories
> can have
> 62 * extended attributes. For sticky directories, only the owner and
> 63 * privileged users can write attributes.
> 64 */
> {code}
> We should consider removing {{XAttrFeature}} from {{INodeSymlink}}.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6373) Remove support for extended attributes on symlinks
[
https://issues.apache.org/jira/browse/HDFS-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995982#comment-13995982
]
Yi Liu commented on HDFS-6373:
--
Thanks Andrew.
Let's remove XAttrFeature from INodeSysmlink, so symlinks don't have xattrs of
their own.
In design doc we have
{quote}
Symlinks do not have XAttrs of their own. Operations that modify the XAttrs of
a sysmlink
instead modify the XAttr of the symlinkâs target
{quote}
> Remove support for extended attributes on symlinks
> --
>
> Key: HDFS-6373
> URL: https://issues.apache.org/jira/browse/HDFS-6373
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
>Reporter: Andrew Wang
>Assignee: Charles Lamb
>
> Looking in the Linux source code, we see the following:
> http://lxr.linux.no/linux+v3.14.3/fs/xattr.c
> {code}
> 60/*
> 61 * In the user.* namespace, only regular files and directories
> can have
> 62 * extended attributes. For sticky directories, only the owner and
> 63 * privileged users can write attributes.
> 64 */
> {code}
> We should consider removing {{XAttrFeature}} from {{INodeSymlink}}.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
