[
https://issues.apache.org/jira/browse/MAPREDUCE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989992#comment-13989992
]
Chris Nauroth commented on MAPREDUCE-5809:
------------------------------------------
Thanks for taking a look at this patch, Nicholas.
bq. CopyCommitter should not get and use source FileSystem since it will be
much slower. We should change listing SequenceFile value to something like
FileStatusWithACL (a new class). Then, CopyCommitter could read ACL from it.
If we do that, then we'll lose the parallelism benefit we get from doing the
RPC calls inside the MR tasks. All of the {{getAclStatus}} calls would have to
run in serial while creating the copy listing, before job submission. I think
distcp will get better overall performance by keeping the {{getAclStatus}}
calls in the MR tasks, but let me know if you disagree.
bq. Should AclsNotSupportedException extend IOException instead of
RuntimeException?
I chose {{RuntimeException}} for consistency with the existing exceptions like
{{CopyListing#DuplicateFileException}} and
{{CopyListing#InvalidInputException}}. We could switch to {{IOException}} if
you prefer, but then I'd like to switch all of them to keep it consistent. I
don't think it makes a big difference either way. Let me know.
bq. Let's move AclsNotSupportedException and
DistCpUtils.checkFileSystemAclSupport(..) to Common. They are also useful for
other cases.
I'm going to wait until the fix for HDFS-6326 is committed before I do this.
We might be simplifying the error handling quite a bit, in which case the
{{checkFileSystemAclSupport}} method here in distcp will become quite trivial.
> Enhance distcp to support preserving HDFS ACLs.
> -----------------------------------------------
>
> Key: MAPREDUCE-5809
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5809
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: distcp
> Affects Versions: 2.4.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: MAPREDUCE-5809.1.patch, MAPREDUCE-5809.2.patch,
> MAPREDUCE-5809.3.patch
>
>
> This issue tracks enhancing distcp to add a new command-line argument for
> preserving HDFS ACLs from the source at the copy destination.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
