[jira] [Commented] (HDFS-13668) FSPermissionChecker may throws AIOOE when check if inode has permission
[
https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576363#comment-16576363
]
Wei-Chiu Chuang commented on HDFS-13668:
+1
> FSPermissionChecker may throws AIOOE when check if inode has permission
> ---
>
> Key: HDFS-13668
> URL: https://issues.apache.org/jira/browse/HDFS-13668
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 3.1.0, 2.10.0, 2.7.7
>Reporter: He Xiaoqiao
>Assignee: He Xiaoqiao
>Priority: Major
> Attachments: HDFS-13668-trunk.001.patch, HDFS-13668-trunk.002.patch,
> HDFS-13668-trunk.003.patch
>
>
> {{FSPermissionChecker}} may throw {{ArrayIndexOutOfBoundsException:0}} when
> check if has permission, since it only check inode's {{aclFeature}} if null
> or not but not check it's entry size. When it meets {{aclFeature}} not null
> but it's entry size equal to 0, it will throw AIOOE.
> {code:java}
> private boolean hasPermission(INodeAttributes inode, FsAction access) {
> ..
> final AclFeature aclFeature = inode.getAclFeature();
> if (aclFeature != null) {
> // It's possible that the inode has a default ACL but no access ACL.
> int firstEntry = aclFeature.getEntryAt(0);
> if (AclEntryStatusFormat.getScope(firstEntry) == AclEntryScope.ACCESS) {
> return hasAclPermission(inode, access, mode, aclFeature);
> }
> }
> ..
> }
> {code}
> Actually if use default {{INodeAttributeProvider}}, it can ensure that when
> {{inode}}'s aclFeature is not null and it's entry size also will be greater
> than 0, but {{INodeAttributeProvider}} is a public interface, we could not
> ensure external implement (e.g. Apache Sentry, Apache Ranger) also has the
> similar constraint.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13668) FSPermissionChecker may throws AIOOE when check if inode has permission
[
https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576276#comment-16576276
]
genericqa commented on HDFS-13668:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 24s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 59s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 76m
19s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}138m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | HDFS-13668 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12935121/HDFS-13668-trunk.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux bf731e17e4f3 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0a71bf1 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/24747/testReport/ |
| Max. process+thread count | 3275 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/24747/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> FSPermissionChecker may throws AIOOE when check if inode has permission
> ---
>
>
[jira] [Commented] (HDFS-13668) FSPermissionChecker may throws AIOOE when check if inode has permission
[
https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576102#comment-16576102
]
He Xiaoqiao commented on HDFS-13668:
{quote}We usually call miniDFS.getFileSystem() instead. But this is entirely
valid.
{quote}
update about getting FileSystem instance in{{TestINodeAttributeProvider}}
following [~jojochuang]'s suggestion, and sorry for missing this ungraceful
usage yesterday.
> FSPermissionChecker may throws AIOOE when check if inode has permission
> ---
>
> Key: HDFS-13668
> URL: https://issues.apache.org/jira/browse/HDFS-13668
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 3.1.0, 2.10.0, 2.7.7
>Reporter: He Xiaoqiao
>Assignee: He Xiaoqiao
>Priority: Major
> Attachments: HDFS-13668-trunk.001.patch, HDFS-13668-trunk.002.patch,
> HDFS-13668-trunk.003.patch
>
>
> {{FSPermissionChecker}} may throw {{ArrayIndexOutOfBoundsException:0}} when
> check if has permission, since it only check inode's {{aclFeature}} if null
> or not but not check it's entry size. When it meets {{aclFeature}} not null
> but it's entry size equal to 0, it will throw AIOOE.
> {code:java}
> private boolean hasPermission(INodeAttributes inode, FsAction access) {
> ..
> final AclFeature aclFeature = inode.getAclFeature();
> if (aclFeature != null) {
> // It's possible that the inode has a default ACL but no access ACL.
> int firstEntry = aclFeature.getEntryAt(0);
> if (AclEntryStatusFormat.getScope(firstEntry) == AclEntryScope.ACCESS) {
> return hasAclPermission(inode, access, mode, aclFeature);
> }
> }
> ..
> }
> {code}
> Actually if use default {{INodeAttributeProvider}}, it can ensure that when
> {{inode}}'s aclFeature is not null and it's entry size also will be greater
> than 0, but {{INodeAttributeProvider}} is a public interface, we could not
> ensure external implement (e.g. Apache Sentry, Apache Ranger) also has the
> similar constraint.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13668) FSPermissionChecker may throws AIOOE when check if inode has permission
[
https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575813#comment-16575813
]
He Xiaoqiao commented on HDFS-13668:
ping [~drankye],[~jojochuang],[~shashikant], any further more suggestions?
> FSPermissionChecker may throws AIOOE when check if inode has permission
> ---
>
> Key: HDFS-13668
> URL: https://issues.apache.org/jira/browse/HDFS-13668
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 3.1.0, 2.10.0, 2.7.7
>Reporter: He Xiaoqiao
>Assignee: He Xiaoqiao
>Priority: Major
> Attachments: HDFS-13668-trunk.001.patch, HDFS-13668-trunk.002.patch
>
>
> {{FSPermissionChecker}} may throw {{ArrayIndexOutOfBoundsException:0}} when
> check if has permission, since it only check inode's {{aclFeature}} if null
> or not but not check it's entry size. When it meets {{aclFeature}} not null
> but it's entry size equal to 0, it will throw AIOOE.
> {code:java}
> private boolean hasPermission(INodeAttributes inode, FsAction access) {
> ..
> final AclFeature aclFeature = inode.getAclFeature();
> if (aclFeature != null) {
> // It's possible that the inode has a default ACL but no access ACL.
> int firstEntry = aclFeature.getEntryAt(0);
> if (AclEntryStatusFormat.getScope(firstEntry) == AclEntryScope.ACCESS) {
> return hasAclPermission(inode, access, mode, aclFeature);
> }
> }
> ..
> }
> {code}
> Actually if use default {{INodeAttributeProvider}}, it can ensure that when
> {{inode}}'s aclFeature is not null and it's entry size also will be greater
> than 0, but {{INodeAttributeProvider}} is a public interface, we could not
> ensure external implement (e.g. Apache Sentry, Apache Ranger) also has the
> similar constraint.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13668) FSPermissionChecker may throws AIOOE when check if inode has permission
[
https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574874#comment-16574874
]
He Xiaoqiao commented on HDFS-13668:
Thanks [~jojochuang]
{quote}We should probably revisit ACL checks in NameNode at some point and make
them more defensive.{quote}
Yes, that's a good point, actually I have checked {{AclStorage}},{{INode}} and
its subclass, also {{FSPermissionChecker}} and I do not find any probably AIOOE.
{quote}We usually call miniDFS.getFileSystem() instead. But this is entirely
valid.
{quote}
Sorry don't get the point.
> FSPermissionChecker may throws AIOOE when check if inode has permission
> ---
>
> Key: HDFS-13668
> URL: https://issues.apache.org/jira/browse/HDFS-13668
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 3.1.0, 2.10.0, 2.7.7
>Reporter: He Xiaoqiao
>Assignee: He Xiaoqiao
>Priority: Major
> Attachments: HDFS-13668-trunk.001.patch, HDFS-13668-trunk.002.patch
>
>
> {{FSPermissionChecker}} may throw {{ArrayIndexOutOfBoundsException:0}} when
> check if has permission, since it only check inode's {{aclFeature}} if null
> or not but not check it's entry size. When it meets {{aclFeature}} not null
> but it's entry size equal to 0, it will throw AIOOE.
> {code:java}
> private boolean hasPermission(INodeAttributes inode, FsAction access) {
> ..
> final AclFeature aclFeature = inode.getAclFeature();
> if (aclFeature != null) {
> // It's possible that the inode has a default ACL but no access ACL.
> int firstEntry = aclFeature.getEntryAt(0);
> if (AclEntryStatusFormat.getScope(firstEntry) == AclEntryScope.ACCESS) {
> return hasAclPermission(inode, access, mode, aclFeature);
> }
> }
> ..
> }
> {code}
> Actually if use default {{INodeAttributeProvider}}, it can ensure that when
> {{inode}}'s aclFeature is not null and it's entry size also will be greater
> than 0, but {{INodeAttributeProvider}} is a public interface, we could not
> ensure external implement (e.g. Apache Sentry, Apache Ranger) also has the
> similar constraint.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13668) FSPermissionChecker may throws AIOOE when check if inode has permission
[
https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574806#comment-16574806
]
Wei-Chiu Chuang commented on HDFS-13668:
+1
{quote}{{INodeAttributeProvider}} is a public interface, we could not ensure
external implement (e.g. Apache Sentry, Apache Ranger) also has the similar
constraint.
{quote}
We've seen bugs in Apache Sentry that rendered invalid ACLs in the past. So
this is entirely possible. We should probably revisit ACL checks in NameNode at
some point and make them more defensive.
{code:java}
FileSystem fs = FileSystem.get(miniDFS.getConfiguration(0));{code}
We usually call miniDFS.getFileSystem() instead. But this is entirely valid.
> FSPermissionChecker may throws AIOOE when check if inode has permission
> ---
>
> Key: HDFS-13668
> URL: https://issues.apache.org/jira/browse/HDFS-13668
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 3.1.0, 2.10.0, 2.7.7
>Reporter: He Xiaoqiao
>Assignee: He Xiaoqiao
>Priority: Major
> Attachments: HDFS-13668-trunk.001.patch, HDFS-13668-trunk.002.patch
>
>
> {{FSPermissionChecker}} may throw {{ArrayIndexOutOfBoundsException:0}} when
> check if has permission, since it only check inode's {{aclFeature}} if null
> or not but not check it's entry size. When it meets {{aclFeature}} not null
> but it's entry size equal to 0, it will throw AIOOE.
> {code:java}
> private boolean hasPermission(INodeAttributes inode, FsAction access) {
> ..
> final AclFeature aclFeature = inode.getAclFeature();
> if (aclFeature != null) {
> // It's possible that the inode has a default ACL but no access ACL.
> int firstEntry = aclFeature.getEntryAt(0);
> if (AclEntryStatusFormat.getScope(firstEntry) == AclEntryScope.ACCESS) {
> return hasAclPermission(inode, access, mode, aclFeature);
> }
> }
> ..
> }
> {code}
> Actually if use default {{INodeAttributeProvider}}, it can ensure that when
> {{inode}}'s aclFeature is not null and it's entry size also will be greater
> than 0, but {{INodeAttributeProvider}} is a public interface, we could not
> ensure external implement (e.g. Apache Sentry, Apache Ranger) also has the
> similar constraint.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13668) FSPermissionChecker may throws AIOOE when check if inode has permission
[
https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574498#comment-16574498
]
He Xiaoqiao commented on HDFS-13668:
[~drankye],[~shashikant],
check failed UT(TestSnapshotFileLength,
TestDFSStripedOutputStreamWithFailureWithRandomECPolicy) and test locally, it
seems to work fine and I am sure it is not relate to this patch.
> FSPermissionChecker may throws AIOOE when check if inode has permission
> ---
>
> Key: HDFS-13668
> URL: https://issues.apache.org/jira/browse/HDFS-13668
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 3.1.0, 2.10.0, 2.7.7
>Reporter: He Xiaoqiao
>Assignee: He Xiaoqiao
>Priority: Major
> Attachments: HDFS-13668-trunk.001.patch, HDFS-13668-trunk.002.patch
>
>
> {{FSPermissionChecker}} may throw {{ArrayIndexOutOfBoundsException:0}} when
> check if has permission, since it only check inode's {{aclFeature}} if null
> or not but not check it's entry size. When it meets {{aclFeature}} not null
> but it's entry size equal to 0, it will throw AIOOE.
> {code:java}
> private boolean hasPermission(INodeAttributes inode, FsAction access) {
> ..
> final AclFeature aclFeature = inode.getAclFeature();
> if (aclFeature != null) {
> // It's possible that the inode has a default ACL but no access ACL.
> int firstEntry = aclFeature.getEntryAt(0);
> if (AclEntryStatusFormat.getScope(firstEntry) == AclEntryScope.ACCESS) {
> return hasAclPermission(inode, access, mode, aclFeature);
> }
> }
> ..
> }
> {code}
> Actually if use default {{INodeAttributeProvider}}, it can ensure that when
> {{inode}}'s aclFeature is not null and it's entry size also will be greater
> than 0, but {{INodeAttributeProvider}} is a public interface, we could not
> ensure external implement (e.g. Apache Sentry, Apache Ranger) also has the
> similar constraint.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13668) FSPermissionChecker may throws AIOOE when check if inode has permission
[
https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574417#comment-16574417
]
genericqa commented on HDFS-13668:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 30s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 4s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 76m 59s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}140m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | HDFS-13668 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12934914/HDFS-13668-trunk.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 99fc13265050 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / d81cd36 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/24733/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/24733/testReport/ |
| Max. process+thread count | 3829 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/P
[jira] [Commented] (HDFS-13668) FSPermissionChecker may throws AIOOE when check if inode has permission
[
https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574300#comment-16574300
]
He Xiaoqiao commented on HDFS-13668:
Thanks [~drankye],[~shashikant] for your suggestions, submit v002 following
your advice and trigger jenkins.
> FSPermissionChecker may throws AIOOE when check if inode has permission
> ---
>
> Key: HDFS-13668
> URL: https://issues.apache.org/jira/browse/HDFS-13668
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 3.1.0, 2.10.0, 2.7.7
>Reporter: He Xiaoqiao
>Assignee: He Xiaoqiao
>Priority: Major
> Attachments: HDFS-13668-trunk.001.patch, HDFS-13668-trunk.002.patch
>
>
> {{FSPermissionChecker}} may throw {{ArrayIndexOutOfBoundsException:0}} when
> check if has permission, since it only check inode's {{aclFeature}} if null
> or not but not check it's entry size. When it meets {{aclFeature}} not null
> but it's entry size equal to 0, it will throw AIOOE.
> {code:java}
> private boolean hasPermission(INodeAttributes inode, FsAction access) {
> ..
> final AclFeature aclFeature = inode.getAclFeature();
> if (aclFeature != null) {
> // It's possible that the inode has a default ACL but no access ACL.
> int firstEntry = aclFeature.getEntryAt(0);
> if (AclEntryStatusFormat.getScope(firstEntry) == AclEntryScope.ACCESS) {
> return hasAclPermission(inode, access, mode, aclFeature);
> }
> }
> ..
> }
> {code}
> Actually if use default {{INodeAttributeProvider}}, it can ensure that when
> {{inode}}'s aclFeature is not null and it's entry size also will be greater
> than 0, but {{INodeAttributeProvider}} is a public interface, we could not
> ensure external implement (e.g. Apache Sentry, Apache Ranger) also has the
> similar constraint.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13668) FSPermissionChecker may throws AIOOE when check if inode has permission
[
https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574260#comment-16574260
]
Shashikant Banerjee commented on HDFS-13668:
Thanks [~hexiaoqiao], for reporting and working on this. The patch looks good
to me overall. Some minor comments:
# There are some whitespace issues reported. Please fix them.
# In TestINodeAttributeProvider#testAclFeature, we can remove this code as
this may not be required
{code:java}
fs.rename(aclChildDir, aclChildDirTarget{code}
I am + 1 after that.
> FSPermissionChecker may throws AIOOE when check if inode has permission
> ---
>
> Key: HDFS-13668
> URL: https://issues.apache.org/jira/browse/HDFS-13668
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 3.1.0, 2.10.0, 2.7.7
>Reporter: He Xiaoqiao
>Assignee: He Xiaoqiao
>Priority: Major
> Attachments: HDFS-13668-trunk.001.patch
>
>
> {{FSPermissionChecker}} may throw {{ArrayIndexOutOfBoundsException:0}} when
> check if has permission, since it only check inode's {{aclFeature}} if null
> or not but not check it's entry size. When it meets {{aclFeature}} not null
> but it's entry size equal to 0, it will throw AIOOE.
> {code:java}
> private boolean hasPermission(INodeAttributes inode, FsAction access) {
> ..
> final AclFeature aclFeature = inode.getAclFeature();
> if (aclFeature != null) {
> // It's possible that the inode has a default ACL but no access ACL.
> int firstEntry = aclFeature.getEntryAt(0);
> if (AclEntryStatusFormat.getScope(firstEntry) == AclEntryScope.ACCESS) {
> return hasAclPermission(inode, access, mode, aclFeature);
> }
> }
> ..
> }
> {code}
> Actually if use default {{INodeAttributeProvider}}, it can ensure that when
> {{inode}}'s aclFeature is not null and it's entry size also will be greater
> than 0, but {{INodeAttributeProvider}} is a public interface, we could not
> ensure external implement (e.g. Apache Sentry, Apache Ranger) also has the
> similar constraint.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13668) FSPermissionChecker may throws AIOOE when check if inode has permission
[
https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574231#comment-16574231
]
Kai Zheng commented on HDFS-13668:
--
This NPE looks like a good catch and fix. +1 from me. Thanks Xiaoqiao.
[~jojochuang], would you mind also giving it a look? Thanks.
> FSPermissionChecker may throws AIOOE when check if inode has permission
> ---
>
> Key: HDFS-13668
> URL: https://issues.apache.org/jira/browse/HDFS-13668
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 3.1.0, 2.10.0, 2.7.7
>Reporter: He Xiaoqiao
>Assignee: He Xiaoqiao
>Priority: Major
> Attachments: HDFS-13668-trunk.001.patch
>
>
> {{FSPermissionChecker}} may throw {{ArrayIndexOutOfBoundsException:0}} when
> check if has permission, since it only check inode's {{aclFeature}} if null
> or not but not check it's entry size. When it meets {{aclFeature}} not null
> but it's entry size equal to 0, it will throw AIOOE.
> {code:java}
> private boolean hasPermission(INodeAttributes inode, FsAction access) {
> ..
> final AclFeature aclFeature = inode.getAclFeature();
> if (aclFeature != null) {
> // It's possible that the inode has a default ACL but no access ACL.
> int firstEntry = aclFeature.getEntryAt(0);
> if (AclEntryStatusFormat.getScope(firstEntry) == AclEntryScope.ACCESS) {
> return hasAclPermission(inode, access, mode, aclFeature);
> }
> }
> ..
> }
> {code}
> Actually if use default {{INodeAttributeProvider}}, it can ensure that when
> {{inode}}'s aclFeature is not null and it's entry size also will be greater
> than 0, but {{INodeAttributeProvider}} is a public interface, we could not
> ensure external implement (e.g. Apache Sentry, Apache Ranger) also has the
> similar constraint.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13668) FSPermissionChecker may throws AIOOE when check if inode has permission
[
https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507323#comment-16507323
]
genericqa commented on HDFS-13668:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 18s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 10s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 96m 3s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}159m 36s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency |
| | hadoop.hdfs.server.namenode.TestReencryptionWithKMS |
| | hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints |
| | hadoop.hdfs.client.impl.TestBlockReaderLocal |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HDFS-13668 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12927198/HDFS-13668-trunk.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 209f9ab0f0f2 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ccfb816 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC1 |
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/24416/artifact/out/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/24416/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
