[jira] Commented: (HADOOP-6632) Support for using different Kerberos keys for different instances of Hadoop services
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12926125#action_12926125 ] Hudson commented on HADOOP-6632: Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/]) > Support for using different Kerberos keys for different instances of Hadoop > services > > > Key: HADOOP-6632 > URL: https://issues.apache.org/jira/browse/HADOOP-6632 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Kan Zhang >Assignee: Kan Zhang > Fix For: 0.22.0 > > Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, > HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch > > > We tested using the same Kerberos key for all datanodes in a HDFS cluster or > the same Kerberos key for all TaskTarckers in a MapRed cluster. But it > doesn't work. The reason is that when datanodes try to authenticate to the > namenode all at once, the Kerberos authenticators they send to the namenode > may have the same timestamp and will be rejected as replay requests. This > JIRA makes it possible to use a unique key for each service instance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different Kerberos keys for different instances of Hadoop services
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894192#action_12894192 ] Todd Lipcon commented on HADOOP-6632: - Thanks, Deveraj. That makes sense. > Support for using different Kerberos keys for different instances of Hadoop > services > > > Key: HADOOP-6632 > URL: https://issues.apache.org/jira/browse/HADOOP-6632 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Kan Zhang >Assignee: Kan Zhang > Fix For: 0.22.0 > > Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, > HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch > > > We tested using the same Kerberos key for all datanodes in a HDFS cluster or > the same Kerberos key for all TaskTarckers in a MapRed cluster. But it > doesn't work. The reason is that when datanodes try to authenticate to the > namenode all at once, the Kerberos authenticators they send to the namenode > may have the same timestamp and will be rejected as replay requests. This > JIRA makes it possible to use a unique key for each service instance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different Kerberos keys for different instances of Hadoop services
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894170#action_12894170 ] Devaraj Das commented on HADOOP-6632: - Yes this was intentional. The mr patch seemed like a hack and that's why we didn't commit it to trunk, and instead raised MAPREDUCE-1824 to discuss that... BTW, the problem which the mr patch attempted to address would be significantly less once we have HADOOP-6706 committed that does retries in case of failures due to the false replay attack detection by the rpc servers. MAPREDUCE-1824 takes a low priority.. > Support for using different Kerberos keys for different instances of Hadoop > services > > > Key: HADOOP-6632 > URL: https://issues.apache.org/jira/browse/HADOOP-6632 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Kan Zhang >Assignee: Kan Zhang > Fix For: 0.22.0 > > Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, > HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch > > > We tested using the same Kerberos key for all datanodes in a HDFS cluster or > the same Kerberos key for all TaskTarckers in a MapRed cluster. But it > doesn't work. The reason is that when datanodes try to authenticate to the > namenode all at once, the Kerberos authenticators they send to the namenode > may have the same timestamp and will be rejected as replay requests. This > JIRA makes it possible to use a unique key for each service instance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different Kerberos keys for different instances of Hadoop services
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894158#action_12894158 ] Todd Lipcon commented on HADOOP-6632: - It looks like the 6632.mr.patch portion was committed to ydist but not trunk - was this intentional? > Support for using different Kerberos keys for different instances of Hadoop > services > > > Key: HADOOP-6632 > URL: https://issues.apache.org/jira/browse/HADOOP-6632 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Kan Zhang >Assignee: Kan Zhang > Fix For: 0.22.0 > > Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, > HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch > > > We tested using the same Kerberos key for all datanodes in a HDFS cluster or > the same Kerberos key for all TaskTarckers in a MapRed cluster. But it > doesn't work. The reason is that when datanodes try to authenticate to the > namenode all at once, the Kerberos authenticators they send to the namenode > may have the same timestamp and will be rejected as replay requests. This > JIRA makes it possible to use a unique key for each service instance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different Kerberos keys for different instances of Hadoop services
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890240#action_12890240 ] Hudson commented on HADOOP-6632: Integrated in Hadoop-Common-trunk #398 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/398/]) HADOOP-6632. Adds support for using different keytabs for different servers in a Hadoop cluster. In the earier implementation, all servers of a certain type \(like TaskTracker\), would have the same keytab and the same principal. Now the principal name is a pattern that has _HOST in it. Contributed by Kan Zhang & Jitendra Pandey. > Support for using different Kerberos keys for different instances of Hadoop > services > > > Key: HADOOP-6632 > URL: https://issues.apache.org/jira/browse/HADOOP-6632 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Kan Zhang >Assignee: Kan Zhang > Fix For: 0.22.0 > > Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, > HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch > > > We tested using the same Kerberos key for all datanodes in a HDFS cluster or > the same Kerberos key for all TaskTarckers in a MapRed cluster. But it > doesn't work. The reason is that when datanodes try to authenticate to the > namenode all at once, the Kerberos authenticators they send to the namenode > may have the same timestamp and will be rejected as replay requests. This > JIRA makes it possible to use a unique key for each service instance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different Kerberos keys for different instances of Hadoop services
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890132#action_12890132 ] Hudson commented on HADOOP-6632: Integrated in Hadoop-Hdfs-trunk-Commit #346 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/346/]) HDFS-1201. The HDFS component for HADOOP-6632. Contributed by Kan Zhang & Jitendra Pandey. > Support for using different Kerberos keys for different instances of Hadoop > services > > > Key: HADOOP-6632 > URL: https://issues.apache.org/jira/browse/HADOOP-6632 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Kan Zhang >Assignee: Kan Zhang > Fix For: 0.22.0 > > Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, > HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch > > > We tested using the same Kerberos key for all datanodes in a HDFS cluster or > the same Kerberos key for all TaskTarckers in a MapRed cluster. But it > doesn't work. The reason is that when datanodes try to authenticate to the > namenode all at once, the Kerberos authenticators they send to the namenode > may have the same timestamp and will be rejected as replay requests. This > JIRA makes it possible to use a unique key for each service instance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different Kerberos keys for different instances of Hadoop services
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890112#action_12890112 ] Hudson commented on HADOOP-6632: Integrated in Hadoop-Common-trunk-Commit #331 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/331/]) HADOOP-6632. Adds support for using different keytabs for different servers in a Hadoop cluster. In the earier implementation, all servers of a certain type \(like TaskTracker\), would have the same keytab and the same principal. Now the principal name is a pattern that has _HOST in it. Contributed by Kan Zhang & Jitendra Pandey. > Support for using different Kerberos keys for different instances of Hadoop > services > > > Key: HADOOP-6632 > URL: https://issues.apache.org/jira/browse/HADOOP-6632 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Kan Zhang >Assignee: Kan Zhang > Fix For: 0.22.0 > > Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, > HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch > > > We tested using the same Kerberos key for all datanodes in a HDFS cluster or > the same Kerberos key for all TaskTarckers in a MapRed cluster. But it > doesn't work. The reason is that when datanodes try to authenticate to the > namenode all at once, the Kerberos authenticators they send to the namenode > may have the same timestamp and will be rejected as replay requests. This > JIRA makes it possible to use a unique key for each service instance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different Kerberos keys for different instances of Hadoop services
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890104#action_12890104 ] Hadoop QA commented on HADOOP-6632: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449900/c6632-07.patch against trunk revision 965556. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/624/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/624/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/624/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/624/console This message is automatically generated. > Support for using different Kerberos keys for different instances of Hadoop > services > > > Key: HADOOP-6632 > URL: https://issues.apache.org/jira/browse/HADOOP-6632 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Kan Zhang >Assignee: Kan Zhang > Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, > HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch > > > We tested using the same Kerberos key for all datanodes in a HDFS cluster or > the same Kerberos key for all TaskTarckers in a MapRed cluster. But it > doesn't work. The reason is that when datanodes try to authenticate to the > namenode all at once, the Kerberos authenticators they send to the namenode > may have the same timestamp and will be rejected as replay requests. This > JIRA makes it possible to use a unique key for each service instance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different Kerberos keys for different instances of Hadoop services
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888982#action_12888982 ] Kan Zhang commented on HADOOP-6632: --- The javadoc warnings are unrelated to this patch. Manually verified the feature on a single node cluster. > Support for using different Kerberos keys for different instances of Hadoop > services > > > Key: HADOOP-6632 > URL: https://issues.apache.org/jira/browse/HADOOP-6632 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Kan Zhang >Assignee: Kan Zhang > Attachments: 6632.mr.patch, c6632-05.patch, > HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch > > > We tested using the same Kerberos key for all datanodes in a HDFS cluster or > the same Kerberos key for all TaskTarckers in a MapRed cluster. But it > doesn't work. The reason is that when datanodes try to authenticate to the > namenode all at once, the Kerberos authenticators they send to the namenode > may have the same timestamp and will be rejected as replay requests. This > JIRA makes it possible to use a unique key for each service instance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different Kerberos keys for different instances of Hadoop services
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1273#action_1273 ] Hadoop QA commented on HADOOP-6632: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449571/c6632-05.patch against trunk revision 964134. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/615/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/615/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/615/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/615/console This message is automatically generated. > Support for using different Kerberos keys for different instances of Hadoop > services > > > Key: HADOOP-6632 > URL: https://issues.apache.org/jira/browse/HADOOP-6632 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Kan Zhang >Assignee: Kan Zhang > Attachments: 6632.mr.patch, c6632-05.patch, > HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch > > > We tested using the same Kerberos key for all datanodes in a HDFS cluster or > the same Kerberos key for all TaskTarckers in a MapRed cluster. But it > doesn't work. The reason is that when datanodes try to authenticate to the > namenode all at once, the Kerberos authenticators they send to the namenode > may have the same timestamp and will be rejected as replay requests. This > JIRA makes it possible to use a unique key for each service instance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different Kerberos keys for different instances of Hadoop services
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12845021#action_12845021 ] Kan Zhang commented on HADOOP-6632: --- One error message we observed. 2010-03-03 07:33:50,542 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8020: readAndProcess threw exception javax.security.sasl.SaslException: GSS initia te failed [Caused by GSSException: Failure unspecified at GSS-API level (Mechanism level: Request is a replay (34))]. Count of bytes read: 0 javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: Failure unspecified at GSS-API level (Mechanism level: Request is a replay (34))] at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:159) at org.apache.hadoop.ipc.Server$Connection.saslReadAndProcess(Server.java:913) at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1071) at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:459) at org.apache.hadoop.ipc.Server$Listener.run(Server.java:368) Caused by: GSSException: Failure unspecified at GSS-API level (Mechanism level: Request is a replay (34)) at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:741) at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:323) at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:267) at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:137) ... 4 more Caused by: KrbException: Request is a replay (34) at sun.security.krb5.KrbApReq.authenticate(KrbApReq.java:299) at sun.security.krb5.KrbApReq.(KrbApReq.java:134) at sun.security.jgss.krb5.InitSecContextToken.(InitSecContextToken.java:79) at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:724) ... 7 more > Support for using different Kerberos keys for different instances of Hadoop > services > > > Key: HADOOP-6632 > URL: https://issues.apache.org/jira/browse/HADOOP-6632 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Kan Zhang >Assignee: Kan Zhang > > We tested using the same Kerberos key for all datanodes in a HDFS cluster or > the same Kerberos key for all TaskTarckers in a MapRed cluster. But it > doesn't work. The reason is that when datanodes try to authenticate to the > namenode all at once, the Kerberos authenticators they send to the namenode > may have the same timestamp and will be rejected as replay requests. This > JIRA makes it possible to use a unique key for each service instance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
