[
https://issues.apache.org/jira/browse/HADOOP-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16210832#comment-16210832
]
Wei-Chiu Chuang commented on HADOOP-14030:
------------------------------------------
I started with this Jenkins precommit build:
https://builds.apache.org/job/PreCommit-HADOOP-Build/13543
If you look at any of the TestKDiag output, like this one:
https://builds.apache.org/job/PreCommit-HADOOP-Build/13543/testReport/org.apache.hadoop.security/TestKDiag/testKeytabAndPrincipal/
You will find 12 principals in the keytab: client, server, server1 through
server10, which suggests the keytab is generated by someone else, maybe a
leftover or a race condition.
{noformat}
== Examining keytab
/testptch/hadoop/hadoop-common-project/hadoop-common/target/keytab ==
keytab principal count: 12
server/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=DES3_CBC_SHA1
server/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=AES128_CTS_HMAC_SHA1_96
server9/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=DES3_CBC_SHA1
server9/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=AES128_CTS_HMAC_SHA1_96
server7/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=DES3_CBC_SHA1
server7/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=AES128_CTS_HMAC_SHA1_96
server8/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=DES3_CBC_SHA1
server8/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=AES128_CTS_HMAC_SHA1_96
server2/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=DES3_CBC_SHA1
server2/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=AES128_CTS_HMAC_SHA1_96
server3/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=DES3_CBC_SHA1
server3/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=AES128_CTS_HMAC_SHA1_96
cli...@example.com: version=1 expires=generalized time [tag=0x18, len=2+15]
Wed Oct 18 11:27:16 UTC 2017 encryption=DES3_CBC_SHA1
cli...@example.com: version=1 expires=generalized time [tag=0x18, len=2+15]
Wed Oct 18 11:27:16 UTC 2017 encryption=AES128_CTS_HMAC_SHA1_96
server1/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=DES3_CBC_SHA1
server1/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=AES128_CTS_HMAC_SHA1_96
server4/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=DES3_CBC_SHA1
server4/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=AES128_CTS_HMAC_SHA1_96
server0/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=DES3_CBC_SHA1
server0/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=AES128_CTS_HMAC_SHA1_96
server6/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=DES3_CBC_SHA1
server6/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=AES128_CTS_HMAC_SHA1_96
server5/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=DES3_CBC_SHA1
server5/localh...@example.com: version=1 expires=generalized time [tag=0x18,
len=2+15] Wed Oct 18 11:27:16 UTC 2017 encryption=AES128_CTS_HMAC_SHA1_96
keytab entry count: 24
{noformat}
The only test that generates that many keytab principals is TestRaceWhenRelogin.
Further, this set of TestKDiag tests started at Wed Oct 18 11:27:16, and
TestRaceWhenRelogin also started around the same time:
https://builds.apache.org/job/PreCommit-HADOOP-Build/13543/testReport/org.apache.hadoop.security/TestRaceWhenRelogin/test/
{noformat}
2017-10-18 11:27:16,715 INFO minikdc.MiniKdc (MiniKdc.java:<init>(225)) -
Configuration:
{noformat}
If you look at keytab location of both tests, they actually generate keytabs at
the same directory, same file name. So it looks like a race condition between
parallel tests for me.
I suggest we use a randomized file name or directory for keytabs. Other tests
are likely prone to this bug as well.
> PreCommit TestKDiag failure
> ---------------------------
>
> Key: HADOOP-14030
> URL: https://issues.apache.org/jira/browse/HADOOP-14030
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 3.0.0-alpha4
> Reporter: John Zhuge
> Assignee: Wei-Chiu Chuang
>
> https://builds.apache.org/job/PreCommit-HADOOP-Build/11523/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
> {noformat}
> Tests run: 13, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 2.175 sec
> <<< FAILURE! - in org.apache.hadoop.security.TestKDiag
> testKeytabAndPrincipal(org.apache.hadoop.security.TestKDiag) Time elapsed:
> 0.05 sec <<< ERROR!
> org.apache.hadoop.security.KerberosAuthException: Login failure for user:
> f...@example.com from keytab
> /testptch/hadoop/hadoop-common-project/hadoop-common/target/keytab
> javax.security.auth.login.LoginException: Unable to obtain password from user
> at
> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:897)
> at
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760)
> at
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
> at
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1355)
> at org.apache.hadoop.security.KDiag.loginFromKeytab(KDiag.java:630)
> at org.apache.hadoop.security.KDiag.execute(KDiag.java:396)
> at org.apache.hadoop.security.KDiag.run(KDiag.java:236)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.security.KDiag.exec(KDiag.java:1047)
> at org.apache.hadoop.security.TestKDiag.kdiag(TestKDiag.java:119)
> at
> org.apache.hadoop.security.TestKDiag.testKeytabAndPrincipal(TestKDiag.java:162)
> testFileOutput(org.apache.hadoop.security.TestKDiag) Time elapsed: 0.033 sec
> <<< ERROR!
> org.apache.hadoop.security.KerberosAuthException: Login failure for user:
> f...@example.com from keytab
> /testptch/hadoop/hadoop-common-project/hadoop-common/target/keytab
> javax.security.auth.login.LoginException: Unable to obtain password from user
> at
> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:897)
> at
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760)
> at
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
> at
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1355)
> at org.apache.hadoop.security.KDiag.loginFromKeytab(KDiag.java:630)
> at org.apache.hadoop.security.KDiag.execute(KDiag.java:396)
> at org.apache.hadoop.security.KDiag.run(KDiag.java:236)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.security.KDiag.exec(KDiag.java:1047)
> at org.apache.hadoop.security.TestKDiag.kdiag(TestKDiag.java:119)
> at
> org.apache.hadoop.security.TestKDiag.testFileOutput(TestKDiag.java:186)
> testLoadResource(org.apache.hadoop.security.TestKDiag) Time elapsed: 0.031
> sec <<< ERROR!
> org.apache.hadoop.security.KerberosAuthException: Login failure for user:
> f...@example.com from keytab
> /testptch/hadoop/hadoop-common-project/hadoop-common/target/keytab
> javax.security.auth.login.LoginException: Unable to obtain password from user
> at
> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:897)
> at
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760)
> at
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
> at
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1355)
> at org.apache.hadoop.security.KDiag.loginFromKeytab(KDiag.java:630)
> at org.apache.hadoop.security.KDiag.execute(KDiag.java:396)
> at org.apache.hadoop.security.KDiag.run(KDiag.java:236)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.security.KDiag.exec(KDiag.java:1047)
> at org.apache.hadoop.security.TestKDiag.kdiag(TestKDiag.java:119)
> at
> org.apache.hadoop.security.TestKDiag.testLoadResource(TestKDiag.java:196)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
