[jira] [Commented] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734315#comment-16734315
]
Hudson commented on HADOOP-15997:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15703 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/15703/])
HADOOP-15997. KMS client uses wrong UGI after HADOOP-14445. Contributed
(sunilg: rev 51427cbdfb39cb6f5774b7b70009d7ee4388edfc)
* (edit)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/kms/TestLoadBalancingKMSClientProvider.java
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java
> KMS client uses wrong UGI after HADOOP-14445
>
>
> Key: HADOOP-15997
> URL: https://issues.apache.org/jira/browse/HADOOP-15997
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.2.0, 3.0.4, 3.1.2
> Environment: Hadoop 3.0.x (CDH6.x), Kerberized, HDFS at-rest
> encryption, multiple KMS
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Blocker
> Fix For: 3.2.0, 3.3.0, 3.1.2
>
> Attachments: HADOOP-15997.001.patch, HADOOP-15997.02.patch
>
>
> After HADOOP-14445, KMS client always authenticates itself using the
> credentials from login user, rather than current user.
> {noformat}
> 2018-12-07 15:58:30,663 DEBUG [main]
> org.apache.hadoop.crypto.key.kms.KMSClientProvider: Using loginUser when
> Kerberos is enabled but the actual user does not have either KMS Delegation
> Token or Kerberos Credentials
> {noformat}
> The log message {{"Using loginUser when Kerberos is enabled but the actual
> user does not have either KMS Delegation Token or Kerberos Credentials"}} is
> printed because {{KMSClientProvider#containsKmsDt()}} is null when it
> definitely has the kms delegation token.
> In fact, {{KMSClientProvider#containsKmsDt()}} should select delegation token
> using {{clientTokenProvider.selectDelegationToken(creds)}} rather than
> checking if its dtService is in the user credentials.
> This is done correctly in {{KMSClientProvider#createAuthenticatedURL}} though.
> We found this bug when it broke Cloudera's Backup and Disaster Recovery tool.
>
> [~daryn] [~xiaochen] mind taking a look? HADOOP-14445 is a huge patch but it
> is almost perfect except for this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734301#comment-16734301
]
Sunil Govindan commented on HADOOP-15997:
-
Thanks. Committing this shortly
> KMS client uses wrong UGI after HADOOP-14445
>
>
> Key: HADOOP-15997
> URL: https://issues.apache.org/jira/browse/HADOOP-15997
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.2.0, 3.0.4, 3.1.2
> Environment: Hadoop 3.0.x (CDH6.x), Kerberized, HDFS at-rest
> encryption, multiple KMS
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Blocker
> Attachments: HADOOP-15997.001.patch, HADOOP-15997.02.patch
>
>
> After HADOOP-14445, KMS client always authenticates itself using the
> credentials from login user, rather than current user.
> {noformat}
> 2018-12-07 15:58:30,663 DEBUG [main]
> org.apache.hadoop.crypto.key.kms.KMSClientProvider: Using loginUser when
> Kerberos is enabled but the actual user does not have either KMS Delegation
> Token or Kerberos Credentials
> {noformat}
> The log message {{"Using loginUser when Kerberos is enabled but the actual
> user does not have either KMS Delegation Token or Kerberos Credentials"}} is
> printed because {{KMSClientProvider#containsKmsDt()}} is null when it
> definitely has the kms delegation token.
> In fact, {{KMSClientProvider#containsKmsDt()}} should select delegation token
> using {{clientTokenProvider.selectDelegationToken(creds)}} rather than
> checking if its dtService is in the user credentials.
> This is done correctly in {{KMSClientProvider#createAuthenticatedURL}} though.
> We found this bug when it broke Cloudera's Backup and Disaster Recovery tool.
>
> [~daryn] [~xiaochen] mind taking a look? HADOOP-14445 is a huge patch but it
> is almost perfect except for this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733278#comment-16733278
]
Wei-Chiu Chuang commented on HADOOP-15997:
--
The test failure is due to HADOOP-16016, unrelated to this patch. It's
triggered by a change in the latest JDK8.
> KMS client uses wrong UGI after HADOOP-14445
>
>
> Key: HADOOP-15997
> URL: https://issues.apache.org/jira/browse/HADOOP-15997
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.2.0, 3.0.4, 3.1.2
> Environment: Hadoop 3.0.x (CDH6.x), Kerberized, HDFS at-rest
> encryption, multiple KMS
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Blocker
> Attachments: HADOOP-15997.001.patch, HADOOP-15997.02.patch
>
>
> After HADOOP-14445, KMS client always authenticates itself using the
> credentials from login user, rather than current user.
> {noformat}
> 2018-12-07 15:58:30,663 DEBUG [main]
> org.apache.hadoop.crypto.key.kms.KMSClientProvider: Using loginUser when
> Kerberos is enabled but the actual user does not have either KMS Delegation
> Token or Kerberos Credentials
> {noformat}
> The log message {{"Using loginUser when Kerberos is enabled but the actual
> user does not have either KMS Delegation Token or Kerberos Credentials"}} is
> printed because {{KMSClientProvider#containsKmsDt()}} is null when it
> definitely has the kms delegation token.
> In fact, {{KMSClientProvider#containsKmsDt()}} should select delegation token
> using {{clientTokenProvider.selectDelegationToken(creds)}} rather than
> checking if its dtService is in the user credentials.
> This is done correctly in {{KMSClientProvider#createAuthenticatedURL}} though.
> We found this bug when it broke Cloudera's Backup and Disaster Recovery tool.
>
> [~daryn] [~xiaochen] mind taking a look? HADOOP-14445 is a huge patch but it
> is almost perfect except for this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731878#comment-16731878
]
Sunil Govindan commented on HADOOP-15997:
-
Jenkins seems fine with the patch apart from test issue. [~jojochuang] cud u
pls check test failure. if no objections, i ll help to commit this patch by
tomorrow.
cc [~templedf] [~daryn] [~xiaochen]
> KMS client uses wrong UGI after HADOOP-14445
>
>
> Key: HADOOP-15997
> URL: https://issues.apache.org/jira/browse/HADOOP-15997
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.2.0, 3.0.4, 3.1.2
> Environment: Hadoop 3.0.x (CDH6.x), Kerberized, HDFS at-rest
> encryption, multiple KMS
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Blocker
> Attachments: HADOOP-15997.001.patch, HADOOP-15997.02.patch
>
>
> After HADOOP-14445, KMS client always authenticates itself using the
> credentials from login user, rather than current user.
> {noformat}
> 2018-12-07 15:58:30,663 DEBUG [main]
> org.apache.hadoop.crypto.key.kms.KMSClientProvider: Using loginUser when
> Kerberos is enabled but the actual user does not have either KMS Delegation
> Token or Kerberos Credentials
> {noformat}
> The log message {{"Using loginUser when Kerberos is enabled but the actual
> user does not have either KMS Delegation Token or Kerberos Credentials"}} is
> printed because {{KMSClientProvider#containsKmsDt()}} is null when it
> definitely has the kms delegation token.
> In fact, {{KMSClientProvider#containsKmsDt()}} should select delegation token
> using {{clientTokenProvider.selectDelegationToken(creds)}} rather than
> checking if its dtService is in the user credentials.
> This is done correctly in {{KMSClientProvider#createAuthenticatedURL}} though.
> We found this bug when it broke Cloudera's Backup and Disaster Recovery tool.
>
> [~daryn] [~xiaochen] mind taking a look? HADOOP-14445 is a huge patch but it
> is almost perfect except for this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729946#comment-16729946
]
Hadoop QA commented on HADOOP-15997:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 28s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 11s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 19s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 96m 57s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.ssl.TestSSLFactory |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-15997 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12952282/HADOOP-15997.02.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f32fd4e72a21 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / d8f670f |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15703/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15703/testReport/ |
| Max. process+thread count | 1359 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15703/console |
| Powered by | Apache Yetus
[jira] [Commented] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729900#comment-16729900
]
Wei-Chiu Chuang commented on HADOOP-15997:
--
Thanks [~adam.antal] the patch name seems fine with me. I'm triggering the
build manually again.
> KMS client uses wrong UGI after HADOOP-14445
>
>
> Key: HADOOP-15997
> URL: https://issues.apache.org/jira/browse/HADOOP-15997
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.2.0, 3.0.4, 3.1.2
> Environment: Hadoop 3.0.x (CDH6.x), Kerberized, HDFS at-rest
> encryption, multiple KMS
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Blocker
> Attachments: HADOOP-15997.001.patch, HADOOP-15997.02.patch
>
>
> After HADOOP-14445, KMS client always authenticates itself using the
> credentials from login user, rather than current user.
> {noformat}
> 2018-12-07 15:58:30,663 DEBUG [main]
> org.apache.hadoop.crypto.key.kms.KMSClientProvider: Using loginUser when
> Kerberos is enabled but the actual user does not have either KMS Delegation
> Token or Kerberos Credentials
> {noformat}
> The log message {{"Using loginUser when Kerberos is enabled but the actual
> user does not have either KMS Delegation Token or Kerberos Credentials"}} is
> printed because {{KMSClientProvider#containsKmsDt()}} is null when it
> definitely has the kms delegation token.
> In fact, {{KMSClientProvider#containsKmsDt()}} should select delegation token
> using {{clientTokenProvider.selectDelegationToken(creds)}} rather than
> checking if its dtService is in the user credentials.
> This is done correctly in {{KMSClientProvider#createAuthenticatedURL}} though.
> We found this bug when it broke Cloudera's Backup and Disaster Recovery tool.
>
> [~daryn] [~xiaochen] mind taking a look? HADOOP-14445 is a huge patch but it
> is almost perfect except for this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729618#comment-16729618
]
Adam Antal commented on HADOOP-15997:
-
Thanks for the patch [~jojochuang]. I'm not an expert in this area, so
meaningfully I only checked the patch syntactically, and it looks good to me.
The newly added test passes and the other 2 unrelated tests from jenkins pass
as well. It seems the patch v2 did not trigger jenkins - maybe wrong name
(.002.patch instead of .02.patch)?
> KMS client uses wrong UGI after HADOOP-14445
>
>
> Key: HADOOP-15997
> URL: https://issues.apache.org/jira/browse/HADOOP-15997
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.2.0, 3.0.4, 3.1.2
> Environment: Hadoop 3.0.x (CDH6.x), Kerberized, HDFS at-rest
> encryption, multiple KMS
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Blocker
> Attachments: HADOOP-15997.001.patch, HADOOP-15997.02.patch
>
>
> After HADOOP-14445, KMS client always authenticates itself using the
> credentials from login user, rather than current user.
> {noformat}
> 2018-12-07 15:58:30,663 DEBUG [main]
> org.apache.hadoop.crypto.key.kms.KMSClientProvider: Using loginUser when
> Kerberos is enabled but the actual user does not have either KMS Delegation
> Token or Kerberos Credentials
> {noformat}
> The log message {{"Using loginUser when Kerberos is enabled but the actual
> user does not have either KMS Delegation Token or Kerberos Credentials"}} is
> printed because {{KMSClientProvider#containsKmsDt()}} is null when it
> definitely has the kms delegation token.
> In fact, {{KMSClientProvider#containsKmsDt()}} should select delegation token
> using {{clientTokenProvider.selectDelegationToken(creds)}} rather than
> checking if its dtService is in the user credentials.
> This is done correctly in {{KMSClientProvider#createAuthenticatedURL}} though.
> We found this bug when it broke Cloudera's Backup and Disaster Recovery tool.
>
> [~daryn] [~xiaochen] mind taking a look? HADOOP-14445 is a huge patch but it
> is almost perfect except for this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16724664#comment-16724664
]
Wei-Chiu Chuang commented on HADOOP-15997:
--
Thanks [~templedf]. Rev002 to address Daniel's review comments.
> KMS client uses wrong UGI after HADOOP-14445
>
>
> Key: HADOOP-15997
> URL: https://issues.apache.org/jira/browse/HADOOP-15997
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.2.0, 3.0.4, 3.1.2
> Environment: Hadoop 3.0.x (CDH6.x), Kerberized, HDFS at-rest
> encryption, multiple KMS
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Blocker
> Attachments: HADOOP-15997.001.patch, HADOOP-15997.02.patch
>
>
> After HADOOP-14445, KMS client always authenticates itself using the
> credentials from login user, rather than current user.
> {noformat}
> 2018-12-07 15:58:30,663 DEBUG [main]
> org.apache.hadoop.crypto.key.kms.KMSClientProvider: Using loginUser when
> Kerberos is enabled but the actual user does not have either KMS Delegation
> Token or Kerberos Credentials
> {noformat}
> The log message {{"Using loginUser when Kerberos is enabled but the actual
> user does not have either KMS Delegation Token or Kerberos Credentials"}} is
> printed because {{KMSClientProvider#containsKmsDt()}} is null when it
> definitely has the kms delegation token.
> In fact, {{KMSClientProvider#containsKmsDt()}} should select delegation token
> using {{clientTokenProvider.selectDelegationToken(creds)}} rather than
> checking if its dtService is in the user credentials.
> This is done correctly in {{KMSClientProvider#createAuthenticatedURL}} though.
> We found this bug when it broke Cloudera's Backup and Disaster Recovery tool.
>
> [~daryn] [~xiaochen] mind taking a look? HADOOP-14445 is a huge patch but it
> is almost perfect except for this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16723927#comment-16723927
]
Sunil Govindan commented on HADOOP-15997:
-
ping [~templedf] [~jojochuang] [~xiaochen] [~daryn]
> KMS client uses wrong UGI after HADOOP-14445
>
>
> Key: HADOOP-15997
> URL: https://issues.apache.org/jira/browse/HADOOP-15997
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.2.0, 3.0.4, 3.1.2
> Environment: Hadoop 3.0.x (CDH6.x), Kerberized, HDFS at-rest
> encryption, multiple KMS
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Blocker
> Attachments: HADOOP-15997.001.patch
>
>
> After HADOOP-14445, KMS client always authenticates itself using the
> credentials from login user, rather than current user.
> {noformat}
> 2018-12-07 15:58:30,663 DEBUG [main]
> org.apache.hadoop.crypto.key.kms.KMSClientProvider: Using loginUser when
> Kerberos is enabled but the actual user does not have either KMS Delegation
> Token or Kerberos Credentials
> {noformat}
> The log message {{"Using loginUser when Kerberos is enabled but the actual
> user does not have either KMS Delegation Token or Kerberos Credentials"}} is
> printed because {{KMSClientProvider#containsKmsDt()}} is null when it
> definitely has the kms delegation token.
> In fact, {{KMSClientProvider#containsKmsDt()}} should select delegation token
> using {{clientTokenProvider.selectDelegationToken(creds)}} rather than
> checking if its dtService is in the user credentials.
> This is done correctly in {{KMSClientProvider#createAuthenticatedURL}} though.
> We found this bug when it broke Cloudera's Backup and Disaster Recovery tool.
>
> [~daryn] [~xiaochen] mind taking a look? HADOOP-14445 is a huge patch but it
> is almost perfect except for this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721899#comment-16721899
]
Sunil Govindan commented on HADOOP-15997:
-
[~templedf] [~xiaochen] [~daryn] pls help to review.
I marked this as a blocker for 3.2.0 and 3.1.2
> KMS client uses wrong UGI after HADOOP-14445
>
>
> Key: HADOOP-15997
> URL: https://issues.apache.org/jira/browse/HADOOP-15997
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.2.0, 3.0.4, 3.1.2
> Environment: Hadoop 3.0.x (CDH6.x), Kerberized, HDFS at-rest
> encryption, multiple KMS
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Blocker
> Attachments: HADOOP-15997.001.patch
>
>
> After HADOOP-14445, KMS client always authenticates itself using the
> credentials from login user, rather than current user.
> {noformat}
> 2018-12-07 15:58:30,663 DEBUG [main]
> org.apache.hadoop.crypto.key.kms.KMSClientProvider: Using loginUser when
> Kerberos is enabled but the actual user does not have either KMS Delegation
> Token or Kerberos Credentials
> {noformat}
> The log message {{"Using loginUser when Kerberos is enabled but the actual
> user does not have either KMS Delegation Token or Kerberos Credentials"}} is
> printed because {{KMSClientProvider#containsKmsDt()}} is null when it
> definitely has the kms delegation token.
> In fact, {{KMSClientProvider#containsKmsDt()}} should select delegation token
> using {{clientTokenProvider.selectDelegationToken(creds)}} rather than
> checking if its dtService is in the user credentials.
> This is done correctly in {{KMSClientProvider#createAuthenticatedURL}} though.
> We found this bug when it broke Cloudera's Backup and Disaster Recovery tool.
>
> [~daryn] [~xiaochen] mind taking a look? HADOOP-14445 is a huge patch but it
> is almost perfect except for this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721478#comment-16721478
]
Hadoop QA commented on HADOOP-15997:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 43s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 27s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 0s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 90m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.util.TestDiskCheckerWithDiskIo |
| | hadoop.util.TestReadWriteDiskValidator |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-15997 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951368/HADOOP-15997.001.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 1ad95b99bcfc 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / aa12859 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15654/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15654/testReport/ |
| Max. process+thread count | 1687 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/1
[jira] [Commented] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721382#comment-16721382
]
Daniel Templeton commented on HADOOP-15997:
---
The change looks good to me. Two minor quibbles. First, I always appreciate
messages in assert statements so that I don't have to read code to know how the
test failed. Second, we seem to have lost some context in the debug logs with
the new patch. There's still enough debug logging to show that something is
happening, but it's no longer clear why.
I would love to get a nod from [~daryn] or [~xiaochen] before we move forward.
> KMS client uses wrong UGI after HADOOP-14445
>
>
> Key: HADOOP-15997
> URL: https://issues.apache.org/jira/browse/HADOOP-15997
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.2.0, 3.0.4, 3.1.2
> Environment: Hadoop 3.0.x (CDH6.x), Kerberized, HDFS at-rest
> encryption, multiple KMS
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HADOOP-15997.001.patch
>
>
> After HADOOP-14445, KMS client always authenticates itself using the
> credentials from login user, rather than current user.
> {noformat}
> 2018-12-07 15:58:30,663 DEBUG [main]
> org.apache.hadoop.crypto.key.kms.KMSClientProvider: Using loginUser when
> Kerberos is enabled but the actual user does not have either KMS Delegation
> Token or Kerberos Credentials
> {noformat}
> The log message {{"Using loginUser when Kerberos is enabled but the actual
> user does not have either KMS Delegation Token or Kerberos Credentials"}} is
> printed because {{KMSClientProvider#containsKmsDt()}} is null when it
> definitely has the kms delegation token.
> In fact, {{KMSClientProvider#containsKmsDt()}} should select delegation token
> using {{clientTokenProvider.selectDelegationToken(creds)}} rather than
> checking if its dtService is in the user credentials.
> This is done correctly in {{KMSClientProvider#createAuthenticatedURL}} though.
> We found this bug when it broke Cloudera's Backup and Disaster Recovery tool.
>
> [~daryn] [~xiaochen] mind taking a look? HADOOP-14445 is a huge patch but it
> is almost perfect except for this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
