[
https://issues.apache.org/jira/browse/HDFS-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13991038#comment-13991038
]
Chris Nauroth commented on HDFS-6326:
-------------------------------------
Daryn, thank you for taking a look at this.
bq. It's probably more appropriate for WebHdfsFileSystem to detect if acls are
supported upon the first call. It keeps the ugliness out of FsShell. It would
be great to also push the hdfs-specific exception handling into
DistributedFileSystem.
I agree this would look cleaner, but then I would worry about a long-lived
client process recording that ACLs aren't supported, then a NameNode gets
upgraded/reconfigured to support ACLs, but the client continues to think it's
unsupported. This isn't a problem for the shell, because the process is
short-lived. Maybe allow for a periodic retry to check for ACL support again?
bq. Given that we now have extensible protobufs and json, is there any reason
why FileStatus, or FSPermission, etc don't return a boolean if there's an acl
on the file? Then an acl call is may only be made when necessary.
I discussed this in my first comment on this issue. Here are some more details.
Regarding {{FileStatus}}, the same question came up on MAPREDUCE-5809 today.
Here is a copy-paste of my response:
We considered adding the ACLs to {{FileStatus}}, but this would have been a
backwards-incompatible change. {{FileStatus}} implements {{Writable}}
serialization, which is more brittle to version compared to something like
protobuf. {{FileStatus#write}} doesn't embed any kind of version number, so
there is no reliable way to tell at runtime if we are deserializing a pre-ACLs
{{FileStatus}} or a post-ACLs {{FileStatus}}. This would have had a high risk
of breaking downstream code or mixed versions that had used the {{Writable}}
serialization. An alternative would have been to skip serializing ACLs in
{{FileStatus#write}}, but then there would have been a risk of NPE for clients
expecting a fully serialized object. This is discussed further in the HDFS-4685
design doc on page 12:
https://issues.apache.org/jira/secure/attachment/12627729/HDFS-ACLs-Design-3.pdf
Regarding {{FsPermission}}, at one point the HDFS-4685 feature branch was using
a previously unused bit in the {{FsPermssion}} short as an ACL indicator. This
got pushback though on HDFS-5923, and the ACL bit changes were backed out
completely. Here is a relevant comment:
https://issues.apache.org/jira/browse/HDFS-5923?focusedCommentId=13898370&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13898370
If you look at my attached HDFS-6326.1.patch, it reintroduces the ACL bit as a
transient field, toggled on the outbound {{FileStatus}} responses from
NameNode, but not persisted to fsimage. Making it transient addresses some of
the objections in HDFS-5923, but still leaves the specific objections mentioned
in the linked comment.
[~daryn] and [~wheat9], the two of you have been most involved in the debate on
this, so would you please review this comment, reconsider the trade-offs, and
post your current opinion? At this point, considering all of the problems, I'm
in favor of reintroducing the ACL bit (patch v1). The one problem with this
approach is that if a 2.4.1 shell talks to a 2.4.0 NameNode, then it wouldn't
display the '+', because the 2.4.0 NameNode wouldn't be toggling the ACL bit.
I think it's worth accepting this relatively minor bug for one release to get
us past the larger issues.
> WebHdfs ACL compatibility is broken
> -----------------------------------
>
> Key: HDFS-6326
> URL: https://issues.apache.org/jira/browse/HDFS-6326
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Affects Versions: 3.0.0, 2.4.0
> Reporter: Daryn Sharp
> Assignee: Chris Nauroth
> Priority: Blocker
> Attachments: HDFS-6326.1.patch, HDFS-6326.2.patch
>
>
> 2.4 ACL support is completely incompatible with <2.4 webhdfs servers. The NN
> throws an {{IllegalArgumentException}} exception.
> {code}
> hadoop fs -ls webhdfs://nn/
> Found 21 items
> ls: Invalid value for webhdfs parameter "op": No enum constant
> org.apache.hadoop.hdfs.web.resources.GetOpParam.Op.GETACLSTATUS
> [... 20 more times...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
