[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846864#comment-16846864 ] Prabhu Joseph commented on HADOOP-16287: Thanks [~eyang]. > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: Sub-task > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch, > HADOOP-16287-004.patch, HADOOP-16287-005.patch, HADOOP-16287-006.patch, > HADOOP-16287-007.patch, HADOOP-16827-003.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846817#comment-16846817 ] Hudson commented on HADOOP-16287: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16591 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/16591/]) HADOOP-16287. Implement ProxyUserAuthenticationFilter for web protocol (eyang: rev ea0b1d8fba57f56e2a75e9a70d4768ba75952823) * (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authentication/server/ProxyUserAuthenticationFilter.java * (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authentication/server/ProxyUserAuthenticationFilterInitializer.java * (add) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authentication/server/TestProxyUserAuthenticationFilter.java * (edit) hadoop-common-project/hadoop-common/src/site/markdown/HttpAuthentication.md * (edit) hadoop-common-project/hadoop-common/pom.xml * (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authentication/server/package-info.java > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: Sub-task > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch, > HADOOP-16287-004.patch, HADOOP-16287-005.patch, HADOOP-16287-006.patch, > HADOOP-16287-007.patch, HADOOP-16827-003.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16841753#comment-16841753 ] Larry McCay commented on HADOOP-16287: -- [~daryn] - I see no difference between what is provided here and existing hadoop.auth cookies for things like webhdfs. I am assuming that admins are only allowing the use of proxyusers that can be trusted. Here is my +1. > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: Sub-task > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch, > HADOOP-16287-004.patch, HADOOP-16287-005.patch, HADOOP-16287-006.patch, > HADOOP-16287-007.patch, HADOOP-16827-003.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16840528#comment-16840528 ] Eric Yang commented on HADOOP-16287: +1 on patch 007 to include the new ProxyUserAuthenticationFilter. For this to work globally in a web application such as YARN UI or HDFS UI, we must ensure that the same filter mechanism applies to all endpoints. I opened HADOOP-16314 to track the required changes in application code. > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: New Feature > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch, > HADOOP-16287-004.patch, HADOOP-16287-005.patch, HADOOP-16287-006.patch, > HADOOP-16287-007.patch, HADOOP-16827-003.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16840110#comment-16840110 ] Prabhu Joseph commented on HADOOP-16287: Fixed checkstyle issues in [^HADOOP-16287-007.patch]. > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: New Feature > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch, > HADOOP-16287-004.patch, HADOOP-16287-005.patch, HADOOP-16287-006.patch, > HADOOP-16287-007.patch, HADOOP-16827-003.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[
https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16840105#comment-16840105
]
Hadoop QA commented on HADOOP-16287:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 50s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 47s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
6s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 99m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16287 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12968750/HADOOP-16287-007.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 7037ec65c246 4.4.0-144-generic #170~14.04.1-Ubuntu SMP Mon Mar
18 15:02:05 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 2d8282b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16251/testReport/ |
| Max. process+thread count | 1717 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16251/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was autom
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[
https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16839213#comment-16839213
]
Hadoop QA commented on HADOOP-16287:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 9s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
20s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 42s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 31s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
44s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 95m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16287 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12968640/HADOOP-16287-006.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 0a4529d7b45d 4.4.0-141-generic #167~14.04.1-Ubuntu SMP Mon Dec
10 13:20:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 02c9efc |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16250/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16250/testReport/ |
| Max. process+thread count | 1502 (vs. ulimit of 1) |
| modules | C: hadoop-common
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[
https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16839156#comment-16839156
]
Prabhu Joseph commented on HADOOP-16287:
[~eyang] Have added new filter initializer
{{ProxyUserAuthenticationFilterInitializer}} which adds
{{ProxyUserAuthenticationFilter}} in [^HADOOP-16287-006.patch] and below
documentation.
{code}
Trusted Proxy
--
Trusted Proxy adds support to perform operations using end user instead of
proxy user. It fetches the end user from doAs query parameter. To enable
Trusted Proxy, please set the following configuration parameter:
Add
org.apache.hadoop.security.authentication.server.ProxyUserAuthenticationFilterInitializer
to hadoop.http.filter.initializers at the end in core-site.xml.
{code}
> KerberosAuthenticationHandler Trusted Proxy Support for Knox
>
>
> Key: HADOOP-16287
> URL: https://issues.apache.org/jira/browse/HADOOP-16287
> Project: Hadoop Common
> Issue Type: New Feature
> Components: auth
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch,
> HADOOP-16287-004.patch, HADOOP-16287-005.patch, HADOOP-16287-006.patch,
> HADOOP-16827-003.patch
>
>
> Knox passes doAs with end user while accessing RM, WebHdfs Rest Api.
> Currently KerberosAuthenticationHandler sets the remote user to Knox. Need
> Trusted Proxy Support by reading doAs query parameter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1683#comment-1683 ] Eric Yang commented on HADOOP-16287: [~Prabhu Joseph] Thank you for the patch. I think we also need documentation on how to configure this in core-site.xml. Can you add some document in hadoop-common-project/hadoop-common/src/site/markdown/HttpAuthentication.md? Thanks > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: New Feature > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch, > HADOOP-16287-004.patch, HADOOP-16287-005.patch, HADOOP-16827-003.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836936#comment-16836936 ] Prabhu Joseph commented on HADOOP-16287: Thanks [~sunilg] and [~eyang] for review. The latest patch is [^HADOOP-16287-005.patch], some how it is shown in middle. > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: New Feature > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch, > HADOOP-16287-004.patch, HADOOP-16287-005.patch, HADOOP-16827-003.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836930#comment-16836930 ] Sunil Govindan commented on HADOOP-16287: - Thanks [~Prabhu Joseph] and [~eyang] In Summary, we ensure that in a secure cluster (where authentication is kerberos), we will be adding this new filter (by using from an initialiser for this filter which will be done in another Jira ) at the end of the filter chain. With this new filter, incoming authenticated user will be also checked for 2 things. # Whether incoming authenticated user (say for eg: knox) is a valid proxy user or not (from proxy user configurations) # Whether any doAs param is present in the same requests. In such case, doAs user will be considered as the requested user for Hadoop. Since this is very much configuration driven and not coming in default path, it looks like a cleaner solution. Appreciate your thoughts [~lmccay] [~vinodkv] [~daryn] [~leftnoteasy] > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: New Feature > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch, > HADOOP-16287-004.patch, HADOOP-16287-005.patch, HADOOP-16827-003.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836147#comment-16836147 ] Prabhu Joseph commented on HADOOP-16287: Test case failures are not related. Have reported HADOOP-16303 and HADOOP-16304 to fix the intermittent test case failures. > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: New Feature > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch, > HADOOP-16287-004.patch, HADOOP-16287-005.patch, HADOOP-16827-003.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[
https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836127#comment-16836127
]
Hadoop QA commented on HADOOP-16287:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 31s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 58s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 13s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 92m 4s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.util.TestDiskCheckerWithDiskIo |
| | hadoop.util.TestReadWriteDiskValidator |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16287 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12968258/HADOOP-16287-005.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 5bf44b359b48 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0c5fa2e |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16239/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16239/testReport/ |
| Max. process+thread count | 1598 (vs. ulimit of 1) |
| modules | C
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836081#comment-16836081 ] Prabhu Joseph commented on HADOOP-16287: Patch 5 fixes checkstyle issues. > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: New Feature > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch, > HADOOP-16287-004.patch, HADOOP-16287-005.patch, HADOOP-16827-003.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835961#comment-16835961 ] Eric Yang commented on HADOOP-16287: [~daryn] Any concern with patch 4? If not, I will give +1 on this patch and commit. > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: New Feature > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch, > HADOOP-16287-004.patch, HADOOP-16827-003.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[
https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835531#comment-16835531
]
Hadoop QA commented on HADOOP-16287:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 44s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
22s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 41s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 38s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
5s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 97m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16287 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12968179/HADOOP-16287-004.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux a2952578474d 4.4.0-144-generic #170~14.04.1-Ubuntu SMP Mon Mar
18 15:02:05 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 96dc5ce |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16236/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16236/testReport/ |
| Max. process+thread count | 1465 (vs. ulimit of 1) |
| modules | C: hadoop-common
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835487#comment-16835487 ] Prabhu Joseph commented on HADOOP-16287: [~eyang] Yes, using request attribute to set and get doAsUser is not the right way. Have wrapped the request overriding getRemoteUser to return doAsUser. > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: New Feature > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch, > HADOOP-16827-003.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834908#comment-16834908 ] Eric Yang commented on HADOOP-16287: [~Prabhu Joseph] thank you for the patch. Patch 003 has two problems. It adds request attribute for doAsUser. If downstream logic does not look at request attribute for doAsUser, the request has the full privileges of authenticated user. If down stream logic set another request attribute doAsUser, the request can switch to any user. This is not secure. Second problem, in catching AuthorizationException, it does not return immediately after produce a response for FORBIDDEN. The request will continue to process the filter chain. This will leak more data to caller, if the caller continue to listen for packets after getting FORBIDDEN message without disconnect. > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: New Feature > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch, > HADOOP-16827-003.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[
https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834787#comment-16834787
]
Hadoop QA commented on HADOOP-16287:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 36s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 17m
0s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 41s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 22s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
27s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}101m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16287 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12968047/HADOOP-16827-003.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 8ed5ff0e 4.4.0-141-generic #167~14.04.1-Ubuntu SMP Mon Dec
10 13:20:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 49e1292 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16231/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16231/testReport/ |
| Max. process+thread count | 1497 (vs. ulimit of 1) |
| modules | C: hadoop-common
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[
https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834645#comment-16834645
]
Hadoop QA commented on HADOOP-16287:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 8m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 21s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
41s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 44s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 9 new + 0 unchanged - 0 fixed = 9 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 26s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
15s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}100m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16287 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12968039/HADOOP-16287-002.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux f2e1c6df72af 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 49e1292 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16229/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16229/testReport/ |
| Max. process+thread count | 1382 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[
https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834607#comment-16834607
]
Prabhu Joseph commented on HADOOP-16287:
[~daryn] [~eyang] Thanks for the review. When testing the patch-001 with YARN
RM UI and Knox Gateway using multiple browsers in parallel, observed AuthToken
for user1 is used for all the subsequent operations.
Have set the impersonated user in http request attribute "proxyUser". YARN RM
UI code will define a filter initializer which adds the
{{ProxyUserAuthenticationFIlter}}. RM UI code can lists apps based on the end
user instead of proxy user.
> KerberosAuthenticationHandler Trusted Proxy Support for Knox
>
>
> Key: HADOOP-16287
> URL: https://issues.apache.org/jira/browse/HADOOP-16287
> Project: Hadoop Common
> Issue Type: New Feature
> Components: auth
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: HADOOP-16287-001.patch, HADOOP-16287-002.patch
>
>
> Knox passes doAs with end user while accessing RM, WebHdfs Rest Api.
> Currently KerberosAuthenticationHandler sets the remote user to Knox. Need
> Trusted Proxy Support by reading doAs query parameter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[ https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832604#comment-16832604 ] Eric Yang commented on HADOOP-16287: It would be best to ensure that respond of servers are identical to client with or without proxy. > KerberosAuthenticationHandler Trusted Proxy Support for Knox > > > Key: HADOOP-16287 > URL: https://issues.apache.org/jira/browse/HADOOP-16287 > Project: Hadoop Common > Issue Type: New Feature > Components: auth >Affects Versions: 3.2.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: HADOOP-16287-001.patch > > > Knox passes doAs with end user while accessing RM, WebHdfs Rest Api. > Currently KerberosAuthenticationHandler sets the remote user to Knox. Need > Trusted Proxy Support by reading doAs query parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[
https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832601#comment-16832601
]
Eric Yang commented on HADOOP-16287:
[~daryn] {quote}-1 on returning a new auth cookie as the impersonated user.
It's insanely dangerous and will create bugs and/or security holes. The auth
cookie must be the authenticated user. Let's explore the unintended side
effects.{quote}
If the auth cookie is forwarded from proxy to end user, then end user got token
for proxy user (authenticated user). That does not sound right. It's either
no auth cookie returned, or the cookie is token for end user credential. This
prevents accidental leaks of impersonation power. The three points listed are
implementation mistake that can happen, if proxyuser or server code is not
written properly. Knox does shield hadoop.auth cookie from leaking. The
handling of hadoop.auth cookie between Hadoop and Knox should be private
conversation. If the operation between servers are multi-calls, the cached
token can reduce hitting KDC server for each call.
> KerberosAuthenticationHandler Trusted Proxy Support for Knox
>
>
> Key: HADOOP-16287
> URL: https://issues.apache.org/jira/browse/HADOOP-16287
> Project: Hadoop Common
> Issue Type: New Feature
> Components: auth
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: HADOOP-16287-001.patch
>
>
> Knox passes doAs with end user while accessing RM, WebHdfs Rest Api.
> Currently KerberosAuthenticationHandler sets the remote user to Knox. Need
> Trusted Proxy Support by reading doAs query parameter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[
https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832580#comment-16832580
]
Daryn Sharp commented on HADOOP-16287:
--
We need this, but -1 on one very seemingly simple change noted below this list.
Cursory review:
# I'd like to see this be a unique filter, not a subclass of
{{KerberosAuthenticationHandler}}, since there's no reason for it to be
specific to any given authentication type. Ideally it should be an enforced
filter installed at the tail of the filter chain.
# Should be much cheaper to use {{request.getParameter("doAs")}} versus
manually re-parsing the query string.
# Returning no meaningful status message with the forbidden response isn't
useful. Might not have to catch the exception from {{ProxyUsers.authorize}}.
I think the exception mappings will convert it to a forbidden response and also
afford apps to ability encode the remote exception in the response payload.
Test it out though.
-1 on returning a new auth cookie as the impersonated user. It's insanely
dangerous and will create bugs and/or security holes. The auth cookie must be
the authenticated user. Let's explore the unintended side effects.
The auth cookie is equivalent to hard authentication (typically kerberos unless
there's a custom auth filter). That's very important because impersonation and
management operations (ie. token ops) can only be performed via hard
authentication or auth cookie. The impersonated user did not authenticate so
they _must not_ be granted an auth cookie.
Cookie aware clients including but not limited to {{AuthenticatedURL}} and by
extension the rest-based kms client, and perhaps things like Hue, may be
unexpectedly impacted. Consider if "proxyuser" impersonates "user1". This patch
will cause a cookie for user1 to be used for all subsequent operations. Now a
few bad things can happen:
# "proxyuser" attempts another operation as user1. Instead of passing an auth
cookie for proxyuser, it passes one for user1 and fails because user1 is likely
not a proxy user and thus cannot proxy to itself.
# "proxyuser" now attempts to impersonate user2. Uses the auth cookie for
user1 which hopefully fails because user1 likely isn't a proxy user too.
# "proxyuser" attempts to perform an operation as itself but instead will do
it as user1.
Best case, all these should fail. Worst case, the potential for very creative
abuse can elevate privileges esp. with sloppy proxy user configurations which
apparently are more common than I thought.
> KerberosAuthenticationHandler Trusted Proxy Support for Knox
>
>
> Key: HADOOP-16287
> URL: https://issues.apache.org/jira/browse/HADOOP-16287
> Project: Hadoop Common
> Issue Type: New Feature
> Components: auth
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: HADOOP-16287-001.patch
>
>
> Knox passes doAs with end user while accessing RM, WebHdfs Rest Api.
> Currently KerberosAuthenticationHandler sets the remote user to Knox. Need
> Trusted Proxy Support by reading doAs query parameter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16287) KerberosAuthenticationHandler Trusted Proxy Support for Knox
[
https://issues.apache.org/jira/browse/HADOOP-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832478#comment-16832478
]
Hadoop QA commented on HADOOP-16287:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 22m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 47s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 19m
33s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 41s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 58s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
18s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
50s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}115m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16287 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12967770/HADOOP-16287-001.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 6d2f3a6ff82a 4.4.0-144-generic #170~14.04.1-Ubuntu SMP Mon Mar
18 15:02:05 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / f1875b2 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16218/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16218/testReport/ |
| Max. process+thread count | 1555 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
h
