[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Issue Type: Bug (was: Sub-task)
Parent: (was: HDFS-4685)
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: HDFS ACLs (HDFS-4685)
>
> Attachments: HDFS-5594.1.patch, HDFS-5594.2.patch, HDFS-5594.3.patch,
> HDFS-5594.4.patch, HDFS-5594.5.patch, HDFS-5594.6.patch, HDFS-5594.7.patch
>
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Resolution: Fixed
Fix Version/s: HDFS ACLs (HDFS-4685)
Status: Resolved (was: Patch Available)
I've committed this to the HDFS-4685 branch. Thank you to Vinay, Haohui and
Nicholas for the code reviews and some great suggestions. Also, thank you to
Vinay for starting work on the CLI tasks.
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: HDFS ACLs (HDFS-4685)
>
> Attachments: HDFS-5594.1.patch, HDFS-5594.2.patch, HDFS-5594.3.patch,
> HDFS-5594.4.patch, HDFS-5594.5.patch, HDFS-5594.6.patch, HDFS-5594.7.patch
>
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Attachment: HDFS-5594.7.patch
Thanks, Nicholas. I'm uploading patch version 7. I plan to commit this
version to the HDFS-4685 branch tomorrow unless there is more feedback by then.
bq. In FileSystem, should modifyAclEntries be called as addAclEntries?
This method actually merges the requested modifications into the file's
existing ACL. The aclSpec can contain a mix of adding new ACL entries and
modifying the permissions associated with existing ACL entries. If the file
has existing ACL entries that are not mentioned in the aclSpec, then those
entries are retained. I think {{modifyAclEntries}} is the appropriate name,
but I also think my JavaDocs didn't explain this clearly enough. I've tried to
improve on this in the new patch.
bq. Typo in Acl: associatd
Thanks for catching this. I fixed it.
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-5594.1.patch, HDFS-5594.2.patch, HDFS-5594.3.patch,
> HDFS-5594.4.patch, HDFS-5594.5.patch, HDFS-5594.6.patch, HDFS-5594.7.patch
>
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HDFS-5594:
-
Hadoop Flags: Reviewed
+1 patch looks good. Minor question/typo:
- In FileSystem, should modifyAclEntries be called as addAclEntries? Both
"add" and "remove" are modifying the entries but "modify" here means "add".
- Typo in Acl: associatd
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-5594.1.patch, HDFS-5594.2.patch, HDFS-5594.3.patch,
> HDFS-5594.4.patch, HDFS-5594.5.patch, HDFS-5594.6.patch
>
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Attachment: HDFS-5594.6.patch
bq. Since you are touching FileSystem.java can clean following unused imports
Sure thing. Here is patch version 6 to make that change.
bq. One more thing, (general), why cant we check of trailing space in modified
lines using Hadoop QA? This is done in bookkeeper's QA.
I'm not sure if this is an enforced standard for the project, although it is
something I strive to do in my patches. I suggest filing a separate jira to do
this, and if anyone objects, then they can discuss there. It's definitely
feasible to add a check for this in test-patch.sh. It would need to be smart
enough to detect trailing whitespace only on new code, not diff context from
existing code. i.e. something like:
{code}
grep '^\+.*\s$'
{code}
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-5594.1.patch, HDFS-5594.2.patch, HDFS-5594.3.patch,
> HDFS-5594.4.patch, HDFS-5594.5.patch, HDFS-5594.6.patch
>
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Attachment: HDFS-5594.5.patch
Ah, I hadn't checked for trailing whitespace on the comment lines. Here is
patch version 5 to clean that up.
I didn't see any remaining imports of the commons-lang stuff, so patch version
5 doesn't change anything related to that.
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-5594.1.patch, HDFS-5594.2.patch, HDFS-5594.3.patch,
> HDFS-5594.4.patch, HDFS-5594.5.patch
>
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Attachment: HDFS-5594.4.patch
Here is version 4 of the patch with following changes:
# {{equals}} and {{hashCode}} methods have switched from Commons Lang to Guava.
# {{Acl#entries}} is converted from {{Set}} to {{List}}. Let's go with that,
and the NameNode implementation can still enforce the unique constraint that I
mentioned.
# I implemented {{AclEntry#compareTo}} and sorted the entries on the way out of
{{Acl#Builder}}. I added another test for this.
Haohui, the only trailing whitespace I saw was in the diff context for existing
code, not the new code. {{ViewFileSystem}} has a lot of trailing whitepsace
right now, and I'd prefer not to roll a full cleanup into this patch. If you
see trailing whitespace elsewhere in the new code, please let me know.
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-5594.1.patch, HDFS-5594.2.patch, HDFS-5594.3.patch,
> HDFS-5594.4.patch
>
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Attachment: HDFS-5594.3.patch
Here is patch version 3 with the following changes:
# Previously, {{Acl}} contained the associated file/owner/group. This would
have made the class cumbersome to use in the NameNode implementation, where we
want to conserve memory by referencing the same {{Acl}} from multiple files.
I've removed file/owner/group.
# However, we still do need a way to get the association between an ACL and a
file/owner/group. The getfacl CLI prints this information, and clients who do
a recursive get also need a way to link each ACL in the result set back to its
file. To support this, I've introduced a new {{AclStatus}} class that contains
ACL, file, owner and group.
# {{AclStatus}} is returned from {{listAclStatus}}, which was previously called
{{getAcls}}. I did this rename for consistency with existing methods like
{{listStatus}}.
# It looks like the getfacl command traditionally prints the additional flags
on the file too, like sticky bit. To support this, I've added sticky bit as a
member of {{Acl}}. That's also helpful for the NameNode implementation, where
we'll need a distinct {{Acl}} instance for different sticky bit settings even
if the ACL entries are otherwise identical.
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-5594.1.patch, HDFS-5594.2.patch, HDFS-5594.3.patch
>
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Issue Type: Sub-task (was: Improvement)
Parent: HDFS-4685
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-5594.1.patch, HDFS-5594.2.patch
>
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Issue Type: Improvement (was: Sub-task)
Parent: (was: HDFS-4685)
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-5594.1.patch, HDFS-5594.2.patch
>
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Attachment: HDFS-5594.2.patch
Here is a new version of the patch that updates {{TestHarFileSystem}} to fix
the prior test failure.
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-5594.1.patch, HDFS-5594.2.patch
>
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Attachment: HDFS-5594.1.patch
This patch provides the {{FileSystem}} API described in the design doc for
reading and writing ACLs. Here are a few additional notes on this patch:
# The design doc mentioned an {{AclSpec}} class for use in the modification
operations. It turns out that we can simplify this to a {{Set}}, so
I didn't code a separate {{AclSpec}} class. I'll plan to update the design doc
accordingly.
# It might look odd that we have separate enums for read flags and write flags,
both of which just contain a {{RECURSIVE}} option. It's possible that these
flags will diverge over time, so I'd like to keep the enums separate. For
example, Linux getfacl has various additional filtering flags that wouldn't
make sense in the context of a write operation.
# The new objects are following patterns that we've started to use recently on
things like the cache management APIs. I made the objects immutable and
provided builders to avoid an explosion of multiple constructors.
# I'm reusing {{FsAction}} in the ACL model. This class is perfect for
representing the permissions portion of an ACL entry, and it has convenience
methods for computing union and intersection of permissions, which will help
later. I've expanded visiblity of the class from
{{LimitedPrivate(\{"HDFS"\})}}/{{Unstable}} to {{Public}}/{{Stable}}. I
checked revision history, and this class actually has been quite stable. The
last code change was 5 years ago, and that was an internal implementation
change that didn't alter the interface. I see very little risk in expanding
the visibility of this class.
The branch hasn't deviated from trunk yet, so I'm going to submit this for a
Jenkins run.
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-5594.1.patch
>
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Status: Patch Available (was: Open)
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-5594.1.patch
>
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Description: Add new methods to {{FileSystem}} for manipulating ACLs.
(was: Add new methods to {{FileSystem}} and {{FileContext}} for manipulating
ACLs.)
I've updated the description to state that this issue covers just
{{FileSystem}}. For adding the API to {{FileContext}} and
{{AbstractFileSystem}}, I created HDFS-5638.
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>
> Add new methods to {{FileSystem}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HDFS-5594) FileSystem API for ACLs.
[
https://issues.apache.org/jira/browse/HDFS-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-5594:
Target Version/s: HDFS ACLs (HDFS-4685)
Affects Version/s: HDFS ACLs (HDFS-4685)
> FileSystem API for ACLs.
>
>
> Key: HDFS-5594
> URL: https://issues.apache.org/jira/browse/HDFS-5594
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: HDFS ACLs (HDFS-4685)
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>
> Add new methods to {{FileSystem}} and {{FileContext}} for manipulating ACLs.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
