[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-07-20 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386887#comment-15386887
 ] 

John Zhuge edited comment on HDFS-6962 at 7/21/16 5:41 AM:
---

Thanks [~eddyxu] for the review! Great comments. I will fix the issues..

bq. Is it true that after creation, the createMode is not useful for INode, as 
its mode has already been established? I feel that we do not need to store it 
as a feature in INode.

Yes it is true, create modes are only needed during inode creation. It is a 
stretch to call CreateModes a feature, but it has be stored somewhere in inode 
or another data structure 1-1 tied to inode. Suggestions are welcome.


was (Author: jzhuge):
Thanks [~eddyxu] for the review! Great comments. I will fix the issues..

bq. Is it true that after creation, the createMode is not useful for INode, as 
its mode has already been established? I feel that we do not need to store it 
as a feature in INode.

Yes it is true, create modes are only needed during inode creation. It is a 
stretch to call CreateModes a feature, but it has be stored somewhere in 
something hanging off inode or another data structure 1-1 tied to inode. 
Suggestions are welcome.

> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.002.patch, 
> HDFS-6962.003.patch, HDFS-6962.004.patch, HDFS-6962.005.patch, 
> HDFS-6962.006.patch, HDFS-6962.007.patch, HDFS-6962.1.patch, 
> disabled_new_client.log, disabled_old_client.log, enabled_new_client.log, 
> enabled_old_client.log, run
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in 
> hdfs-site.xml, everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for 
> inhéritance.
> default:user:toto:rwx allow toto user with rwx access for inhéritance.
> default:mask::rwx inhéritance mask is rwx, so no mask
> 4/ Create a subdir to test inheritance of ACL
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs
> 5/ check ACLs /tmp/ACLS/hdfs
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs
> # file: /tmp/ACLS/hdfs
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:r-x
> group::r-x
> group:readwrite:rwx #effective:r-x
> mask::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> Here we can see that the readwrite group has rwx ACL bu only r-x is effective 
> because the mask is r-x (mask::r-x) in spite of default mask for inheritance 
> is set to default:mask::rwx on /tmp/ACLS/
> 6/ Modifiy hdfs-site.xml et restart namenode
> 
> dfs.umaskmode
> 010
> 
> 7/ Create a subdir to test inheritance of ACL with new parameter umaskmode
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs2
> 8/ Check ACL on /tmp/ACLS/hdfs2
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs2
> # file: /tmp/ACLS/hdfs2
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:rw-
> group::r-x  #effective:r--
> group:readwrite:rwx #effective:rw-
> mask::rw-
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> So HDFS masks the ACL value (user, group and other  -- exepted the POSIX 
> owner -- ) with the group mask of dfs.umaskmode properties when creating 
> directory with inherited ACL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-07-20 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386399#comment-15386399
 ] 

John Zhuge edited comment on HDFS-6962 at 7/20/16 10:40 PM:


[~cnauroth] and [~eddyxu] Please review patch 007.

Diff from 006:
* Add CLI tests {{TestAclCLIWithPosixAclInheritance}} based on {{TestAclCLI}}
* No longer add field {{createModes}} to {{INodeWithAdditionalFields}}. 
Instead, add new feature {{CreateModesFeature}} to store create modes. In this 
way, no penalty when POSIX ACL inheritance is disable or for any inode not 
being created.
* Remove the feature {{CreateModesFeature}} once default ACL has been 
processed, so the feature only exists in inode for a short period of time.
* There is cost of adding and removing the new feature.

TODO:
* Support webhdfs


was (Author: jzhuge):
[~cnauroth] and [~eddyxu] Please review patch 007.

Diff from 006:
* Add CLI tests {{TestAclCLIWithPosixAclInheritance}} based on {{TestAclCLI}}
* No longer add field {{createModes}} to {{INodeWithAdditionalFields}}. 
Instead, add new feature {{CreateModesFeature}} to store create modes. In this 
way, no penalty when POSIX ACL inheritance is disable or for any inode not 
associated with a create request.
* Remove the feature {{CreateModesFeature}} once default ACL has been processed.
* There is added cost of adding and removing the new feature.

TODO:
* Support webhdfs

> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.002.patch, 
> HDFS-6962.003.patch, HDFS-6962.004.patch, HDFS-6962.005.patch, 
> HDFS-6962.006.patch, HDFS-6962.007.patch, HDFS-6962.1.patch, 
> disabled_new_client.log, disabled_old_client.log, enabled_new_client.log, 
> enabled_old_client.log, run
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in 
> hdfs-site.xml, everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for 
> inhéritance.
> default:user:toto:rwx allow toto user with rwx access for inhéritance.
> default:mask::rwx inhéritance mask is rwx, so no mask
> 4/ Create a subdir to test inheritance of ACL
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs
> 5/ check ACLs /tmp/ACLS/hdfs
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs
> # file: /tmp/ACLS/hdfs
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:r-x
> group::r-x
> group:readwrite:rwx #effective:r-x
> mask::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> Here we can see that the readwrite group has rwx ACL bu only r-x is effective 
> because the mask is r-x (mask::r-x) in spite of default mask for inheritance 
> is set to default:mask::rwx on /tmp/ACLS/
> 6/ Modifiy hdfs-site.xml et restart namenode
> 
> dfs.umaskmode
> 010
> 
> 7/ Create a subdir to test inheritance of ACL with new parameter umaskmode
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs2
> 8/ Check ACL on /tmp/ACLS/hdfs2
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs2
> # file: /tmp/ACLS/hdfs2
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:rw-
> group::r-x  #effective:r--
> group:readwrite:rwx #effective:rw-
> mask::rw-
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> So HDFS masks the ACL value (user, group and other  -- exepted the POSIX 
> owner -- ) with the group mask of dfs.umaskmode properties when creating 
> directory with inherited ACL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: 

[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-07-04 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359304#comment-15359304
 ] 

John Zhuge edited comment on HDFS-6962 at 7/4/16 8:02 AM:
--

Saw {{TestDataNodeLifeline}} and {{TestOfflineEditsViewer}} failures on my 
local Mac with stock trunk, so they are not regressions. 
{{TestMover,TestNNThroughputBenchmark,TestNNThroughputBenchmark,TestStripedINodeFile,TestRetryCacheWithHA}}
 failures are regressions.


was (Author: jzhuge):
Looking into the unit test failures. They are regressions.

> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.002.patch, 
> HDFS-6962.003.patch, HDFS-6962.004.patch, HDFS-6962.1.patch, 
> disabled_new_client.log, disabled_old_client.log, enabled_new_client.log, 
> enabled_old_client.log, run
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in 
> hdfs-site.xml, everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for 
> inhéritance.
> default:user:toto:rwx allow toto user with rwx access for inhéritance.
> default:mask::rwx inhéritance mask is rwx, so no mask
> 4/ Create a subdir to test inheritance of ACL
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs
> 5/ check ACLs /tmp/ACLS/hdfs
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs
> # file: /tmp/ACLS/hdfs
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:r-x
> group::r-x
> group:readwrite:rwx #effective:r-x
> mask::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> Here we can see that the readwrite group has rwx ACL bu only r-x is effective 
> because the mask is r-x (mask::r-x) in spite of default mask for inheritance 
> is set to default:mask::rwx on /tmp/ACLS/
> 6/ Modifiy hdfs-site.xml et restart namenode
> 
> dfs.umaskmode
> 010
> 
> 7/ Create a subdir to test inheritance of ACL with new parameter umaskmode
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs2
> 8/ Check ACL on /tmp/ACLS/hdfs2
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs2
> # file: /tmp/ACLS/hdfs2
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:rw-
> group::r-x  #effective:r--
> group:readwrite:rwx #effective:rw-
> mask::rw-
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> So HDFS masks the ACL value (user, group and other  -- exepted the POSIX 
> owner -- ) with the group mask of dfs.umaskmode properties when creating 
> directory with inherited ACL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-06-28 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354400#comment-15354400
 ] 

John Zhuge edited comment on HDFS-6962 at 6/29/16 2:56 AM:
---

Patch 004 passed system and compatibility tests on a pseudo cluster. Upload 
test logs. Focus on group WRITE permission. The test script is {{run}}, the 
command line is
{code}
test_tag=disabled_new_client ; script ~/test/$test_tag.log ~/test/run $test_tag
{code}

Test server is running {{3.0.0-alpha1 + Patch 004}} with POSIX ACL Inheritance 
either enabled or disabled. There are 2 versions of test clients. The POSIX ACL 
inheritance is only in effect when the flag is true and the requests come from 
a compatible client. The following table shows the matrix for the log files.

|| Client Version\POSIX ACL Inheritance || Enabled || Disabled ||
| 2.6.0 | enabled_old_client.log | disabled_old_client.log |
| 3.0.0-alpha1 + Patch 004 | enabled_new_client.log | disabled_new_client.log |

In each log file, there is a test matrix of the following dimensions:
* Parent dir has default ACL or not
* Umask 002 or 027
* Create a file (create request) or a sub-directory (mkdirs request)


was (Author: jzhuge):
Patch 004 passed system and compatibility tests on a pseudo cluster. Upload 
test logs. Focus on group permission, especially WRITE. The test script is 
{{run}}, the command line is
{code}
test_tag=disabled_new_client ; script ~/test/$test_tag.log ~/test/run $test_tag
{code}

Test server is running {{3.0.0-alpha1 + Patch 004}} with POSIX ACL Inheritance 
either enabled or disabled. There are 2 versions of test clients. The POSIX ACL 
inheritance is only in effect when the flag is true and the requests come from 
a compatible client. The following table shows the matrix for the log files.

|| Client Version\POSIX ACL Inheritance || Enabled || Disabled ||
| 2.6.0 | enabled_old_client.log | disabled_old_client.log |
| 3.0.0-alpha1 + Patch 004 | enabled_new_client.log | disabled_new_client.log |

In each log file, there is a test matrix of the following dimensions:
* Parent dir has default ACL or not
* Umask 002 or 027
* Create a file (create request) or a sub-directory (mkdirs request)

> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.002.patch, 
> HDFS-6962.003.patch, HDFS-6962.1.patch, disabled_new_client.log, 
> disabled_old_client.log, enabled_new_client.log, enabled_old_client.log, run
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in 
> hdfs-site.xml, everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for 
> inhéritance.
> default:user:toto:rwx allow toto user with rwx access for inhéritance.
> default:mask::rwx inhéritance mask is rwx, so no mask
> 4/ Create a subdir to test inheritance of ACL
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs
> 5/ check ACLs /tmp/ACLS/hdfs
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs
> # file: /tmp/ACLS/hdfs
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:r-x
> group::r-x
> group:readwrite:rwx #effective:r-x
> mask::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> Here we can see that the readwrite group has rwx ACL bu only r-x is effective 
> because the mask is r-x (mask::r-x) in spite of default mask for inheritance 
> is set to default:mask::rwx on /tmp/ACLS/
> 6/ Modifiy hdfs-site.xml et restart namenode
> 
> dfs.umaskmode
> 010
> 
> 7/ Create a subdir to test inheritance of ACL with new parameter umaskmode
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs2
> 8/ Check ACL on /tmp/ACLS/hdfs2
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs2
> # file: /tmp/ACLS/hdfs2
> # owner: hdfs
> # group: hadoop
> user::rwx

[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-06-28 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354400#comment-15354400
 ] 

John Zhuge edited comment on HDFS-6962 at 6/29/16 2:48 AM:
---

Patch 004 passed system and compatibility tests on a pseudo cluster. Upload 
test logs. Focus on group permission, especially WRITE. The test script is 
{{run}}, the command line is
{code}
test_tag=disabled_new_client ; script ~/test/$test_tag.log ~/test/run $test_tag
{code}

Test server is running {{3.0.0-alpha1 + Patch 004}} with POSIX ACL Inheritance 
either enabled or disabled. There are 2 versions of test clients. The POSIX ACL 
inheritance is only in effect when the flag is true and the requests come from 
a compatible client. The following table shows the matrix for the log files.

|| Client Version\POSIX ACL Inheritance || Enabled || Disabled ||
| 2.6.0 | enabled_old_client.log | disabled_old_client.log |
| 3.0.0-alpha1 + Patch 004 | enabled_new_client.log | disabled_new_client.log |

In each log file, there is a test matrix of the following dimensions:
* Parent dir has default ACL or not
* Umask 002 or 027
* Create a file (create request) or a sub-directory (mkdirs request)


was (Author: jzhuge):
Pass system and compatibility tests with Patch 004 on a pseudo cluster. Upload 
test logs. Focus on group permission, especially WRITE. The test script is 
{{run}}, the command line is
{code}
test_tag=disabled_new_client ; script ~/test/$test_tag.log ~/test/run $test_tag
{code}

Test server is running {{3.0.0-alpha1 + Patch 004}} with POSIX ACL Inheritance 
either enabled or disabled. There are 2 versions of test clients. The POSIX ACL 
inheritance is only in effect when the flag is true and the requests come from 
a compatible client. The following table shows the matrix for the log files.

|| Client Version\POSIX ACL Inheritance || Enabled || Disabled ||
| 2.6.0 | enabled_old_client.log | disabled_old_client.log |
| 3.0.0-alpha1 + Patch 004 | enabled_new_client.log | disabled_new_client.log |

In each log file, there is a test matrix of the following dimensions:
* Parent dir has default ACL or not
* Umask 002 or 027
* Create a file (create request) or a sub-directory (mkdirs request)

> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.002.patch, 
> HDFS-6962.003.patch, HDFS-6962.1.patch, disabled_new_client.log, 
> disabled_old_client.log, enabled_new_client.log, enabled_old_client.log, run
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in 
> hdfs-site.xml, everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for 
> inhéritance.
> default:user:toto:rwx allow toto user with rwx access for inhéritance.
> default:mask::rwx inhéritance mask is rwx, so no mask
> 4/ Create a subdir to test inheritance of ACL
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs
> 5/ check ACLs /tmp/ACLS/hdfs
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs
> # file: /tmp/ACLS/hdfs
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:r-x
> group::r-x
> group:readwrite:rwx #effective:r-x
> mask::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> Here we can see that the readwrite group has rwx ACL bu only r-x is effective 
> because the mask is r-x (mask::r-x) in spite of default mask for inheritance 
> is set to default:mask::rwx on /tmp/ACLS/
> 6/ Modifiy hdfs-site.xml et restart namenode
> 
> dfs.umaskmode
> 010
> 
> 7/ Create a subdir to test inheritance of ACL with new parameter umaskmode
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs2
> 8/ Check ACL on /tmp/ACLS/hdfs2
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs2
> # file: /tmp/ACLS/hdfs2
> # owner: hdfs
> # group: 

[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-06-28 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354400#comment-15354400
 ] 

John Zhuge edited comment on HDFS-6962 at 6/29/16 2:40 AM:
---

Pass system and compatibility tests with Patch 004 on a pseudo cluster. Upload 
test logs. Focus on group permission, especially WRITE. The test script is 
{{run}}, the command line is
{code}
test_tag=disabled_new_client ; script ~/test/$test_tag.log ~/test/run $test_tag
{code}

Test server is running {{3.0.0-alpha1 + Patch 004}} with POSIX ACL Inheritance 
either enabled or disabled. There are 2 versions of test clients. The POSIX ACL 
inheritance is only in effect when the flag is true and the requests come from 
a compatible client. The following table shows the matrix for the log files.

|| Client Version\POSIX ACL Inheritance || Enabled || Disabled ||
| 2.6.0 | enabled_old_client.log | disabled_old_client.log |
| 3.0.0-alpha1 + Patch 004 | enabled_new_client.log | disabled_new_client.log |

In each log file, there is a test matrix of the following dimensions:
* Parent dir has default ACL or not
* Umask 002 or 027
* Create a file (create request) or a sub-directory (mkdirs request)


was (Author: jzhuge):
Upload system and compatibility test logs for Patch 004 on a pseudo cluster. 
Pay attention to group permission, especially WRITE bit. The test script is 
{{run}}, the command line is
{code}
test_tag=disabled_new_client ; script ~/test/$test_tag.log ~/test/run $test_tag
{code}

Test server is running {{3.0.0-alpha1 + Patch 004}} with POSIX ACL Inheritance 
either enabled or disabled. There are 2 versions of test clients. The POSIX ACL 
inheritance is only in effect when the flag is true and the requests come from 
a compatible client. The following table shows the matrix for the log files.

|| Client Version\POSIX ACL Inheritance || Enabled || Disabled ||
| 2.6.0 | enabled_old_client.log | disabled_old_client.log |
| 3.0.0-alpha1 + Patch 004 | enabled_new_client.log | disabled_new_client.log |

In each log file, there is a test matrix of the following dimensions:
* Parent dir has default ACL or not
* Umask 002 or 027
* Create a file (create request) or a sub-directory (mkdirs request)

> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.002.patch, 
> HDFS-6962.003.patch, HDFS-6962.1.patch, disabled_new_client.log, 
> disabled_old_client.log, enabled_new_client.log, enabled_old_client.log, run
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in 
> hdfs-site.xml, everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for 
> inhéritance.
> default:user:toto:rwx allow toto user with rwx access for inhéritance.
> default:mask::rwx inhéritance mask is rwx, so no mask
> 4/ Create a subdir to test inheritance of ACL
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs
> 5/ check ACLs /tmp/ACLS/hdfs
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs
> # file: /tmp/ACLS/hdfs
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:r-x
> group::r-x
> group:readwrite:rwx #effective:r-x
> mask::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> Here we can see that the readwrite group has rwx ACL bu only r-x is effective 
> because the mask is r-x (mask::r-x) in spite of default mask for inheritance 
> is set to default:mask::rwx on /tmp/ACLS/
> 6/ Modifiy hdfs-site.xml et restart namenode
> 
> dfs.umaskmode
> 010
> 
> 7/ Create a subdir to test inheritance of ACL with new parameter umaskmode
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs2
> 8/ Check ACL on /tmp/ACLS/hdfs2
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs2
> # file: /tmp/ACLS/hdfs2
> # owner: hdfs
> # group: 

[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-06-23 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345214#comment-15345214
 ] 

John Zhuge edited comment on HDFS-6962 at 6/24/16 5:20 AM:
---

[~cnauroth] Are there automated tests for section {{Default Acls}} in test plan 
{{Test-Plan-for-Extended-Acls-2.pdf}}? Either extend that section or add a new 
section in test plan to cover ACL inheritance. Currently these are ACL related 
tests: 
TestAcl,TestAclCommands,TestAclCLI,TestViewFileSystemWithAcls,TestViewFsWithAcls,TestAclWithSnapshot,TestAclConfigFlag,TestAclTransformation,TestFileContextAcl,TestFSImageWithAcl,TestNameNodeAcl,TestAclsEndToEnd,TestOfflineImageViewerForAcl,TestWebHDFSAcl.
 Will add a new test class {{namenode/TestAclInheritance}}.


was (Author: jzhuge):
[~cnauroth] Are there automated tests for section {{Default Acls}} in test plan 
{{Test-Plan-for-Extended-Acls-2.pdf}}? Either extend that section or add a new 
section in test plan to cover ACL inheritance. Currently these are ACL related 
tests: 
TestAcl,TestAclCommands,TestAclCLI,TestViewFileSystemWithAcls,TestViewFsWithAcls,FSAclBaseTest,TestAclWithSnapshot,TestAclConfigFlag,TestAclTransformation,TestFileContextAcl,TestFSImageWithAcl,TestNameNodeAcl,TestAclsEndToEnd,TestOfflineImageViewerForAcl,TestWebHDFSAcl.
 Will add a new test class {{namenode/TestAclInheritance}}.

> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.002.patch, 
> HDFS-6962.1.patch
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in 
> hdfs-site.xml, everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for 
> inhéritance.
> default:user:toto:rwx allow toto user with rwx access for inhéritance.
> default:mask::rwx inhéritance mask is rwx, so no mask
> 4/ Create a subdir to test inheritance of ACL
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs
> 5/ check ACLs /tmp/ACLS/hdfs
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs
> # file: /tmp/ACLS/hdfs
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:r-x
> group::r-x
> group:readwrite:rwx #effective:r-x
> mask::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> Here we can see that the readwrite group has rwx ACL bu only r-x is effective 
> because the mask is r-x (mask::r-x) in spite of default mask for inheritance 
> is set to default:mask::rwx on /tmp/ACLS/
> 6/ Modifiy hdfs-site.xml et restart namenode
> 
> dfs.umaskmode
> 010
> 
> 7/ Create a subdir to test inheritance of ACL with new parameter umaskmode
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs2
> 8/ Check ACL on /tmp/ACLS/hdfs2
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs2
> # file: /tmp/ACLS/hdfs2
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:rw-
> group::r-x  #effective:r--
> group:readwrite:rwx #effective:rw-
> mask::rw-
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> So HDFS masks the ACL value (user, group and other  -- exepted the POSIX 
> owner -- ) with the group mask of dfs.umaskmode properties when creating 
> directory with inherited ACL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-06-22 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345214#comment-15345214
 ] 

John Zhuge edited comment on HDFS-6962 at 6/22/16 9:44 PM:
---

[~cnauroth] Are there automated tests for section {{Default Acls}} in test plan 
{{Test-Plan-for-Extended-Acls-2.pdf}}? Either extend that section or add a new 
section in test plan to cover ACL inheritance. Currently these are ACL related 
tests: 
TestAcl,TestAclCommands,TestAclCLI,TestViewFileSystemWithAcls,TestViewFsWithAcls,FSAclBaseTest,TestAclWithSnapshot,TestAclConfigFlag,TestAclTransformation,TestFileContextAcl,TestFSImageWithAcl,TestNameNodeAcl,TestAclsEndToEnd,TestOfflineImageViewerForAcl,TestWebHDFSAcl.
 Will add a new test class {{namenode/TestAclInheritance}}.


was (Author: jzhuge):
[~cnauroth] Are there automated tests for section {{Default Acls}} in test plan 
{{Test-Plan-for-Extended-Acls-2.pdf}}? Either extend that section or add a new 
section in test plan to cover ACL inheritance. Will add a new test class 
{{namenode/TestAclInheritance}}.

> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.002.patch, 
> HDFS-6962.1.patch
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in 
> hdfs-site.xml, everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for 
> inhéritance.
> default:user:toto:rwx allow toto user with rwx access for inhéritance.
> default:mask::rwx inhéritance mask is rwx, so no mask
> 4/ Create a subdir to test inheritance of ACL
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs
> 5/ check ACLs /tmp/ACLS/hdfs
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs
> # file: /tmp/ACLS/hdfs
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:r-x
> group::r-x
> group:readwrite:rwx #effective:r-x
> mask::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> Here we can see that the readwrite group has rwx ACL bu only r-x is effective 
> because the mask is r-x (mask::r-x) in spite of default mask for inheritance 
> is set to default:mask::rwx on /tmp/ACLS/
> 6/ Modifiy hdfs-site.xml et restart namenode
> 
> dfs.umaskmode
> 010
> 
> 7/ Create a subdir to test inheritance of ACL with new parameter umaskmode
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs2
> 8/ Check ACL on /tmp/ACLS/hdfs2
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs2
> # file: /tmp/ACLS/hdfs2
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:rw-
> group::r-x  #effective:r--
> group:readwrite:rwx #effective:rw-
> mask::rw-
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> So HDFS masks the ACL value (user, group and other  -- exepted the POSIX 
> owner -- ) with the group mask of dfs.umaskmode properties when creating 
> directory with inherited ACL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-06-11 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323941#comment-15323941
 ] 

John Zhuge edited comment on HDFS-6962 at 6/11/16 6:41 AM:
---

Patch 002 (prototype):
* Create a new class {{FsPermissionDuo}} that extends {{FsPermission}} to store 
both masked and unmasked permissions. HDFS client uses it to sneak unmasked 
permission from FileContext/FileSystem -> AFS -> Hdfs -> DFSClient -> RPC. NN 
uses it to sneak unmasked permission from RPC -> NameNodeRpcServer.create 
(placed into PermissionStatus) -> FSNamesystem.startFile -> 
FSDirWriterFileOp.startFile/addFile -> INodeFile -> INodeWithAdditionalFields.
* Add field {{unmasked}} to protobuf message {{CreateRequestProto}} and 
{{MkdirsRequestProto}}
* Modify {{copyINodeDefaultAcl}} to switch between old and new ACL inheritance 
behavior.

Questions:
* How to query NN config in {{copyINodeDefaultAcl}}?
* Anyway to avoid adding field FsPermissionDuo to {{INodeWithAdditionalFields}}?
* {{PermissionStatus#applyUMask}} never used, remove it?
* {{DFSClient#mkdirs}} and DFSClient#primitiveMkdir}} use file default if 
permission is null. Should use dir default permission?

TODO:
* Investigate the use of permissions in FSDirMkdirOp.createAncestorDirectories 
call tree
* Add and run unit tests
* Run system compatibility tests
* Update {{HdfsPermissionsGuide.md}}

[~cnauroth] Could you please take a look at Patch 002 to check whether 
backwards compatibility can be achieved with an NN flag and the changes are 
going in the right direction? Thanks.


was (Author: jzhuge):
Patch 002 (prototype):
* Create a new class {{FsPermissionDuo}} that extends {{FsPermission}} to store 
both masked and unmasked permissions. HDFS client uses it to sneak unmasked 
permission from FileContext/FileSystem -> AFS -> Hdfs -> DFSClient -> RPC. NN 
uses it to sneak unmasked permission from RPC -> NameNodeRpcServer.create 
(placed into PermissionStatus) -> FSNamesystem.startFile -> 
FSDirWriterFileOp.startFile/addFile -> INodeFile -> INodeWithAdditionalFields.
* Add field {{unmasked}} to protobuf message {{CreateRequestProto}} and 
{{MkdirsRequestProto}}
* Modify {{copyINodeDefaultAcl}} to switch between old and new ACL inheritance 
behavior.

Questions:
* How to query NN config in {{copyINodeDefaultAcl}}?
* Anyway to avoid adding field FsPermissionDuo to {{INodeWithAdditionalFields}}?
* {{PermissionStatus#applyUMask}} never used, remove it?
* {{DFSClient#mkdirs}} and DFSClient#primitiveMkdir}} use file default if 
permission is null. Should use dir default permission?

TODO:
* Investigate the use of permissions in FSDirMkdirOp.createAncestorDirectories 
call tree
* Add and run unit tests
* Run system compatibility tests
* Update {{HdfsPermissionsGuide.md}}


> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.002.patch, 
> HDFS-6962.1.patch
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in 
> hdfs-site.xml, everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for 
> inhéritance.
> default:user:toto:rwx allow toto user with rwx access for inhéritance.
> default:mask::rwx inhéritance mask is rwx, so no mask
> 4/ Create a subdir to test inheritance of ACL
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs
> 5/ check ACLs /tmp/ACLS/hdfs
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs
> # file: /tmp/ACLS/hdfs
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:r-x
> group::r-x
> group:readwrite:rwx #effective:r-x
> mask::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> Here we can see that the readwrite group has rwx ACL bu only r-x is effective 
> because the 

[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-06-11 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323941#comment-15323941
 ] 

John Zhuge edited comment on HDFS-6962 at 6/11/16 6:39 AM:
---

Patch 002 (prototype):
* Create a new class {{FsPermissionDuo}} that extends {{FsPermission}} to store 
both masked and unmasked permissions. HDFS client uses it to sneak unmasked 
permission from FileContext/FileSystem -> AFS -> Hdfs -> DFSClient -> RPC. NN 
uses it to sneak unmasked permission from RPC -> NameNodeRpcServer.create 
(placed into PermissionStatus) -> FSNamesystem.startFile -> 
FSDirWriterFileOp.startFile/addFile -> INodeFile -> INodeWithAdditionalFields.
* Add field {{unmasked}} to protobuf message {{CreateRequestProto}} and 
{{MkdirsRequestProto}}
* Modify {{copyINodeDefaultAcl}} to switch between old and new ACL inheritance 
behavior.

Questions:
* How to query NN config in {{copyINodeDefaultAcl}}?
* Anyway to avoid adding field FsPermissionDuo to {{INodeWithAdditionalFields}}?
* {{PermissionStatus#applyUMask}} never used, remove it?
* {{DFSClient#mkdirs}} and DFSClient#primitiveMkdir}} use file default if 
permission is null. Should use dir default permission?

TODO:
* Investigate the use of permissions in FSDirMkdirOp.createAncestorDirectories 
call tree
* Add and run unit tests
* Run system compatibility tests
* Update {{HdfsPermissionsGuide.md}}



was (Author: jzhuge):
Patch 002 (prototype):
* Create a new class {{FsPermissionDuo}} that extends {{FsPermission}} to store 
both masked and unmasked permissions. HDFS client uses it to sneak unmasked 
permission from FileContext/FileSystem -> AFS -> Hdfs -> DFSClient -> RPC. NN 
uses it to sneak unmasked permission from RPC -> NameNodeRpcServer.create 
(placed into PermissionStatus) -> FSNamesystem.startFile -> 
FSDirWriterFileOp.startFile/addFile -> INodeFile -> INodeWithAdditionalFields.
* Add field {{unmasked}} to protobuf message {{CreateRequestProto}} and 
{{MkdirsRequestProto}}
* Modify {{copyINodeDefaultAcl}} to switch between old and new ACL inheritance 
behavior.

Questions:
* How to query NN config in {{copyINodeDefaultAcl}}?
* Anyway to avoid adding field FsPermissionDuo to {{INodeWithAdditionalFields}}?
* {{PermissionStatus#applyUMask}} never used, remove it?
* {{DFSClient#mkdirs}} and DFSClient#primitiveMkdir}} use file default if 
permission is null. Should use dir default permission?

TODO:
* Investigate the use of permissions in FSDirMkdirOp.createAncestorDirectories 
call tree
* Add and run unit tests
* Run system compatibility tests


> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.002.patch, 
> HDFS-6962.1.patch
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in 
> hdfs-site.xml, everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for 
> inhéritance.
> default:user:toto:rwx allow toto user with rwx access for inhéritance.
> default:mask::rwx inhéritance mask is rwx, so no mask
> 4/ Create a subdir to test inheritance of ACL
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs
> 5/ check ACLs /tmp/ACLS/hdfs
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs
> # file: /tmp/ACLS/hdfs
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:r-x
> group::r-x
> group:readwrite:rwx #effective:r-x
> mask::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> Here we can see that the readwrite group has rwx ACL bu only r-x is effective 
> because the mask is r-x (mask::r-x) in spite of default mask for inheritance 
> is set to default:mask::rwx on /tmp/ACLS/
> 6/ Modifiy hdfs-site.xml et restart namenode
> 
> dfs.umaskmode
> 010
> 
> 7/ Create a subdir to test 

[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-06-11 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314624#comment-15314624
 ] 

John Zhuge edited comment on HDFS-6962 at 6/11/16 6:37 AM:
---

Hi [~cnauroth],

In order to maintain backwards compatibility in Hadoop 2.x, could we add a new 
NameNode flag {{dfs.namenode.posix.acl.inheritance.enabled}}, default false in 
Hadoop 2.x, default true in Hadoop 3?

Changes to message {{CreateRequestProto}} and {{MkdirsRequestProto}}
* Add optional field {{FsPermissionProto unmasked}} to store unmasked mode 
parameter
* Preserve the existing meaning of field {{FsPermissionProto masked}}: mode + 
umask.
* Please note this approach is slightly different from what you suggested 
above. I am ok either way but find this a little less possibility for 
misunderstanding.

Wrap around {{AclStorage#copyINodeDefaultAcl}}:
{code:java}
// TODO: How to query NameNode config?
boolean isPosixAclInheritanceEnabled = true;
FsPermission masked = child.getMaskedPermission();
FsPermission unmasked = child.getUnmaskedPermission();

if (isPosixAclInheritanceEnabled && unmasked != null) {
  //
  // HDFS-6962: POSIX ACL inheritance
  //
  // Set permission to unmasked
  child.setPermission(unmasked);
  if (!copyINodeDefaultAclInternal(child)) {
// No default ACL in parent dir
// Set permission to masked
child.setPermission(masked);
  }
} else {
  //
  // Old behavior before HDFS-6962
  //
  assert child.getFsPermission().equals(masked);
  copyINodeDefaultAclInternal(child);
}
{code}
Here {{INode#getUnmaskedPermission}} returns the unmasked mode sent by 
DFSClient; {{INode#getMaskedPermission}} returns the masked mode.


was (Author: jzhuge):
Hi [~cnauroth],

In order to maintain backwards compatibility in Hadoop 2.x, could we add a new 
NameNode flag {{dfs.namenode.posix.acl.inheritance.enabled}}, default false in 
Hadoop 2.x, default true in Hadoop 3?

Changes to message {{CreateRequestProto}} and {{MkdirsRequestProto}}
* Add optional field {{FsPermissionProto unmasked}} to store unmasked mode 
parameter
* Preserve the existing meaning of field {{FsPermissionProto masked}}: mode + 
umask.
* Please note this approach is slightly different from what you suggested 
above. I am ok either way but find this a little less possibility for 
misunderstanding.

Wrap around {{AclStorage#copyINodeDefaultAcl}}:
{code:java}
if (child.isPosixAclInheritanceEnabled() == true) {
  //
  // HDFS-6962. POSIX ACL inheritance.
  //
  // Set permission to unmasked if necessary
  if (!child.getFsPermission().equals(child.getUnmaskedMode())) {
child.setPermission(child.getUnmaskedMode());
  }
  if (copyINodeDefaultAclInternal(child) == false) {
// No default ACL in parent dir
// Set permission to masked
child.setPermission(child.getMaskedMode());
  }
} else {
  //
  // Old behavior before HDFS-6962.
  //
  // Set permission to masked if necessary
  if (!child.getFsPermission().equals(child.getMaskedMode())) {
child.setPermission(child.getMaskedMode());
  }
 copyINodeDefaultAclInternal(child);
}
{code}
Here {{INode#getUnmaskedMode}} returns the unmasked mode sent by DFSClient; 
{{INode#getMaskedMode}} returns the masked mode; 
{{INode#isPosixAclInheritanceEnabled}} returns true when the create/mkdir 
request comes from a client with HDFS-6962 fix and 
{{dfs.namenode.posix.acl.inheritance.enabled}} is true.

> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.002.patch, 
> HDFS-6962.1.patch
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in 
> hdfs-site.xml, everything ok !
> 

[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-06-10 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323941#comment-15323941
 ] 

John Zhuge edited comment on HDFS-6962 at 6/10/16 6:20 AM:
---

Patch 002 (prototype):
* Create a new class {{FsPermissionDuo}} that extends {{FsPermission}} to store 
both masked and unmasked permissions. HDFS client uses it to sneak unmasked 
permission from FileContext/FileSystem -> AFS -> Hdfs -> DFSClient -> RPC. NN 
uses it to sneak unmasked permission from RPC -> NameNodeRpcServer.create 
(placed into PermissionStatus) -> FSNamesystem.startFile -> 
FSDirWriterFileOp.startFile/addFile -> INodeFile -> INodeWithAdditionalFields.
* Add field {{unmasked}} to protobuf message {{CreateRequestProto}} and 
{{MkdirsRequestProto}}
* Modify {{copyINodeDefaultAcl}} to switch between old and new ACL inheritance 
behavior.

Questions:
* How to query NN config in {{copyINodeDefaultAcl}}?
* Anyway to avoid adding field FsPermissionDuo to {{INodeWithAdditionalFields}}?
* {{PermissionStatus#applyUMask}} never used, remove it?
* {{DFSClient#mkdirs}} and DFSClient#primitiveMkdir}} use file default if 
permission is null. Should use dir default permission?

TODO:
* Investigate the use of permissions in FSDirMkdirOp.createAncestorDirectories 
call tree
* Add and run unit tests
* Run system compatibility tests



was (Author: jzhuge):
Patch 002 (prototype):
* Create a new class {{FsPermissionDuo}} that extends {{FsPermission}} to store 
both masked and unmasked permissions. HDFS client uses it to sneak unmasked 
permission from FileContext/FileSystem -> AFS -> Hdfs -> DFSClient -> RPC. NN 
uses it to sneak unmasked permission from RPC -> NameNodeRpcServer.create 
(placed into PermissionStatus) -> FSNamesystem.startFile -> 
FSDirWriterFileOp.startFile/addFile -> INodeFile -> INodeWithAdditionalFields.
* Add field {{unmasked}} to protobuf message {{CreateRequestProto}} and 
{{MkdirsRequestProto}}
* Modify {{copyINodeDefaultAcl}} to switch between old and new ACL inheritance 
behavior.

TODO:
* Add and run unit tests
* Run system compatibility tests
* Anyway to avoid adding field FsPermissionDuo to {{INodeWithAdditionalFields}}?
* Investigate the use of permissions in FSDirMkdirOp.createAncestorDirectories 
call tree
* {{PermissionStatus#applyUMask}} never used, remove it?
* {{DFSClient#mkdirs}} and DFSClient#primitiveMkdir}} use file default if 
permission is null. Should use dir default permission?

> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.002.patch, 
> HDFS-6962.1.patch
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in 
> hdfs-site.xml, everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for 
> inhéritance.
> default:user:toto:rwx allow toto user with rwx access for inhéritance.
> default:mask::rwx inhéritance mask is rwx, so no mask
> 4/ Create a subdir to test inheritance of ACL
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs
> 5/ check ACLs /tmp/ACLS/hdfs
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs
> # file: /tmp/ACLS/hdfs
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:r-x
> group::r-x
> group:readwrite:rwx #effective:r-x
> mask::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> Here we can see that the readwrite group has rwx ACL bu only r-x is effective 
> because the mask is r-x (mask::r-x) in spite of default mask for inheritance 
> is set to default:mask::rwx on /tmp/ACLS/
> 6/ Modifiy hdfs-site.xml et restart namenode
> 
> dfs.umaskmode
> 010
> 
> 7/ Create a subdir to test inheritance of ACL with new parameter umaskmode
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs2
> 8/ Check ACL 

[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-06-06 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314624#comment-15314624
 ] 

John Zhuge edited comment on HDFS-6962 at 6/7/16 5:07 AM:
--

Hi [~cnauroth],

In order to maintain backwards compatibility in Hadoop 2.x, could we add a new 
NameNode flag {{dfs.namenode.posix.acl.inheritance.enabled}}, default false in 
Hadoop 2.x, default true in Hadoop 3?

Changes to message {{CreateRequestProto}} and {{MkdirsRequestProto}}
* Add optional field {{FsPermissionProto unmasked}} to store unmasked mode 
parameter
* Preserve the existing meaning of field {{FsPermissionProto masked}}: mode + 
umask.
* Please note this approach is slightly different from what you suggested 
above. I am ok either way but find this a little less possibility for 
misunderstanding.

Wrap around {{AclStorage#copyINodeDefaultAcl}}:
{code:java}
if (child.isPosixAclInheritanceEnabled() == true) {
  //
  // HDFS-6962. POSIX ACL inheritance.
  //
  // Set permission to unmasked if necessary
  if (!child.getFsPermission().equals(child.getUnmaskedMode())) {
child.setPermission(child.getUnmaskedMode());
  }
  if (copyINodeDefaultAclInternal(child) == false) {
// No default ACL in parent dir
// Set permission to masked
child.setPermission(child.getMaskedMode());
  }
} else {
  //
  // Old behavior before HDFS-6962.
  //
  // Set permission to masked if necessary
  if (!child.getFsPermission().equals(child.getMaskedMode())) {
child.setPermission(child.getMaskedMode());
  }
 copyINodeDefaultAclInternal(child);
}
{code}
Here {{INode#getUnmaskedMode}} returns the unmasked mode sent by DFSClient; 
{{INode#getMaskedMode}} returns the masked mode; 
{{INode#isPosixAclInheritanceEnabled}} returns true when the create/mkdir 
request comes from a client with HDFS-6962 fix and 
{{dfs.namenode.posix.acl.inheritance.enabled}} is true.


was (Author: jzhuge):
Hi [~cnauroth],

In order to maintain backwards compatibility in Hadoop 2.x, could we add a new 
NameNode flag {{dfs.namenode.posix.acl.inheritance.enabled}}, default false in 
Hadoop 2.x, default true in Hadoop 3?

Changes to message {{CreateRequestProto}} and {{MkdirsRequestProto}}
* Add optional field {{FsPermissionProto unmasked}} to store unmasked mode 
parameter
* Preserve the existing meaning of field {{FsPermissionProto masked}}: mode + 
umask.
* Please note this approach is slightly different from what you suggested 
above. I am ok either way but find this a little less possibility for 
misunderstanding.

Wrap around {{AclStorage#copyINodeDefaultAcl}}:
{code:java}
if (child.isPosixAclInheritanceEnabled() == true) {
  /*
   * HDFS-6962. POSIX ACL inheritance.
   */
  // Set permission to unmasked if necessary
  if (!child.getFsPermission().equals(child.getUnmaskedMode())) {
child.setPermission(child.getUnmaskedMode());
  }
  if (AclStorage.copyINodeDefaultAclInternal(child) == false) {
// No default ACL in parent dir
// Set permission to masked
child.setPermission(child.getMaskedMode());
  }
} else {
  /*
   * Old behavior before HDFS-6962.
   */
  // Set permission to masked if necessary
  if (!child.getFsPermission().equals(child.getMaskedMode())) {
child.setPermission(child.getMaskedMode());
  }
  AclStorage.copyINodeDefaultAclInternal(child);
}
{code}
Here {{INode#getUnmaskedMode}} returns the unmasked mode sent by DFSClient; 
{{INode#getMaskedMode}} returns the masked mode; 
{{INode#isPosixAclInheritanceEnabled}} returns true when the create/mkdir 
request come from a client that supports POSIX ACL inheritance and 
{{dfs.namenode.posix.acl.inheritance.enabled}} is true.

> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.1.patch
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> 

[jira] [Comment Edited] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-06-06 Thread John Zhuge (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314624#comment-15314624
 ] 

John Zhuge edited comment on HDFS-6962 at 6/7/16 5:04 AM:
--

Hi [~cnauroth],

In order to maintain backwards compatibility in Hadoop 2.x, could we add a new 
NameNode flag {{dfs.namenode.posix.acl.inheritance.enabled}}, default false in 
Hadoop 2.x, default true in Hadoop 3?

Changes to message {{CreateRequestProto}} and {{MkdirsRequestProto}}
* Add optional field {{FsPermissionProto unmasked}} to store unmasked mode 
parameter
* Preserve the existing meaning of field {{FsPermissionProto masked}}: mode + 
umask.
* Please note this approach is slightly different from what you suggested 
above. I am ok either way but find this a little less possibility for 
misunderstanding.

Wrap around {{AclStorage#copyINodeDefaultAcl}}:
{code:java}
if (child.isPosixAclInheritanceEnabled() == true) {
  /*
   * HDFS-6962. POSIX ACL inheritance.
   */
  // Set permission to unmasked if necessary
  if (!child.getFsPermission().equals(child.getUnmaskedMode())) {
child.setPermission(child.getUnmaskedMode());
  }
  if (AclStorage.copyINodeDefaultAclInternal(child) == false) {
// No default ACL in parent dir
// Set permission to masked
child.setPermission(child.getMaskedMode());
  }
} else {
  /*
   * Old behavior before HDFS-6962.
   */
  // Set permission to masked if necessary
  if (!child.getFsPermission().equals(child.getMaskedMode())) {
child.setPermission(child.getMaskedMode());
  }
  AclStorage.copyINodeDefaultAclInternal(child);
}
{code}
Here {{INode#getUnmaskedMode}} returns the unmasked mode sent by DFSClient; 
{{INode#getMaskedMode}} returns the masked mode; 
{{INode#isPosixAclInheritanceEnabled}} returns true when the create/mkdir 
request come from a client that supports POSIX ACL inheritance and 
{{dfs.namenode.posix.acl.inheritance.enabled}} is true.


was (Author: jzhuge):
Hi [~cnauroth],

In order to maintain backwards compatibility in Hadoop 2.x, could we add a new 
NameNode flag {{dfs.namenode.posix.acl.inheritance}}, default false in Hadoop 
2.x, default true in Hadoop 3?

Changes to message {{CreateRequestProto}} and {{MkdirsRequestProto}}
* Add optional field {{FsPermissionProto unmasked}} to store unmasked mode 
parameter
* The meaning of the existing field {{FsPermissionProto masked}} stays the 
same: mode + umask, for both old and new client.
* Please note this approach is slightly different alternative what you 
suggested. I am ok either way.

Wrap around {{AclStorage#copyINodeDefaultAcl}}:
{code:java}
  public static void copyINodeDefaultAcl(INode child) {
// Current permission is masked
assert child.getFsPermission().equals(child.getMaskedMode());
if (child.getPosixAclInheritance()) {
  // Set permission to unmasked
  child.setPermission(child.getUnmaskedMode());
}
if (AclStorage.copyINodeDefaultAclInternal(child) == false) {
  // No default ACL in parent dir
  if (child.getPosixAclInheritance()) {
// Set permission to masked
child.setPermission(child.getMaskedMode());
  }
}
  }
{code}
Here {{INode#getUnmaskedMode}} returns the unmasked mode sent by DFSClient; 
{{INode#getMaskedMode}} returns the masked mode; 
{{INode#getPosixAclInheritance}} returns true when from a new client and the 
flag is on.

> ACLs inheritance conflict with umaskmode
> 
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
>Reporter: LINTE
>Assignee: John Zhuge
>Priority: Critical
>  Labels: hadoop, security
> Attachments: HDFS-6962.001.patch, HDFS-6962.1.patch
>
>
> In hdfs-site.xml 
> 
> dfs.umaskmode
> 027
> 
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user 
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in 
> hdfs-site.xml, everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for 
> inhéritance.
> default:user:toto:rwx allow toto user with rwx