[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to renew delegation token over HTTP
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289428#comment-13289428 ] Hudson commented on MAPREDUCE-2420: --- Integrated in Hadoop-Mapreduce-22-branch #104 (See [https://builds.apache.org/job/Hadoop-Mapreduce-22-branch/104/]) MAPREDUCE-2420. JobTracker should be able to renew delegation token over HTTP. Contributed by Boris Shkolnik and Benoy Antony. (Revision 1346233) Result = SUCCESS shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346233 Files : * /hadoop/common/branches/branch-0.22/mapreduce/CHANGES.txt * /hadoop/common/branches/branch-0.22/mapreduce/src/java/org/apache/hadoop/mapreduce/security/token/DelegationTokenRenewal.java > JobTracker should be able to renew delegation token over HTTP > - > > Key: MAPREDUCE-2420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Boris Shkolnik >Assignee: Boris Shkolnik > Fix For: 0.22.1 > > Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, > MR-2420.22-3.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch > > > in case JobTracker has to talk to a NameNode running a different version (RPC > version mismatch), Jobtracker should be able to fall back to HTTP renewal. > Example of the case - running distcp between different versions using hfpt. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to renew delegation token over HTTP
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280538#comment-13280538 ] Benoy Antony commented on MAPREDUCE-2420: - I removed my patch for 22. The patch -MR-2420.22-3.patch is good. The patch for 22 in HDFS-1860 has to be applied first. > JobTracker should be able to renew delegation token over HTTP > - > > Key: MAPREDUCE-2420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Boris Shkolnik >Assignee: Boris Shkolnik > Fix For: 0.23.0 > > Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, > MR-2420.22-3.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch > > > in case JobTracker has to talk to a NameNode running a different version (RPC > version mismatch), Jobtracker should be able to fall back to HTTP renewal. > Example of the case - running distcp between different versions using hfpt. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to renew delegation token over HTTP
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13028773#comment-13028773 ] Hudson commented on MAPREDUCE-2420: --- Integrated in Hadoop-Mapreduce-trunk #669 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk/669/]) > JobTracker should be able to renew delegation token over HTTP > - > > Key: MAPREDUCE-2420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.20.4 >Reporter: Boris Shkolnik >Assignee: Boris Shkolnik > Fix For: 0.23.0 > > Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, > MR-2420.22-3.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch > > > in case JobTracker has to talk to a NameNode running a different version (RPC > version mismatch), Jobtracker should be able to fall back to HTTP renewal. > Example of the case - running distcp between different versions using hfpt. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to renew delegation token over HTTP
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026771#comment-13026771 ] Hudson commented on MAPREDUCE-2420: --- Integrated in Hadoop-Mapreduce-trunk-Commit #651 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/651/]) MAPREDUCE-2420. JobTracker should be able to renew delegation token over HTTP. Contributed by Boris Shkolnik. > JobTracker should be able to renew delegation token over HTTP > - > > Key: MAPREDUCE-2420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.20.4 >Reporter: Boris Shkolnik >Assignee: Boris Shkolnik > Fix For: 0.23.0 > > Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, > MR-2420.patch, MR2420-2.patch, MR2420-3.patch > > > in case JobTracker has to talk to a NameNode running a different version (RPC > version mismatch), Jobtracker should be able to fall back to HTTP renewal. > Example of the case - running distcp between different versions using hfpt. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to renew delegation token over HTTP
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026765#comment-13026765 ] Jitendra Nath Pandey commented on MAPREDUCE-2420: - I have committed this. Thanks to Boris! > JobTracker should be able to renew delegation token over HTTP > - > > Key: MAPREDUCE-2420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.20.4 >Reporter: Boris Shkolnik >Assignee: Boris Shkolnik > Fix For: 0.23.0 > > Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, > MR-2420.patch, MR2420-2.patch, MR2420-3.patch > > > in case JobTracker has to talk to a NameNode running a different version (RPC > version mismatch), Jobtracker should be able to fall back to HTTP renewal. > Example of the case - running distcp between different versions using hfpt. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to renew delegation token over HTTP
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026749#comment-13026749 ] Jitendra Nath Pandey commented on MAPREDUCE-2420: - +1 for the patch. > JobTracker should be able to renew delegation token over HTTP > - > > Key: MAPREDUCE-2420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.20.4 >Reporter: Boris Shkolnik >Assignee: Boris Shkolnik > Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, > MR-2420.patch, MR2420-2.patch, MR2420-3.patch > > > in case JobTracker has to talk to a NameNode running a different version (RPC > version mismatch), Jobtracker should be able to fall back to HTTP renewal. > Example of the case - running distcp between different versions using hfpt. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to renew delegation token over HTTP
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026693#comment-13026693 ] Jitendra Nath Pandey commented on MAPREDUCE-2420: - Please use constant defined in DFSConfigKeys for dfs.https.port and for the default as well. > JobTracker should be able to renew delegation token over HTTP > - > > Key: MAPREDUCE-2420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.20.4 >Reporter: Boris Shkolnik >Assignee: Boris Shkolnik > Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, > MR-2420.patch, MR2420-2.patch > > > in case JobTracker has to talk to a NameNode running a different version (RPC > version mismatch), Jobtracker should be able to fall back to HTTP renewal. > Example of the case - running distcp between different versions using hfpt. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to renew delegation token over HTTP
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026690#comment-13026690 ] Boris Shkolnik commented on MAPREDUCE-2420: --- ran the ant test and it passed. > JobTracker should be able to renew delegation token over HTTP > - > > Key: MAPREDUCE-2420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.20.4 >Reporter: Boris Shkolnik >Assignee: Boris Shkolnik > Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, > MR-2420.patch, MR2420-2.patch > > > in case JobTracker has to talk to a NameNode running a different version (RPC > version mismatch), Jobtracker should be able to fall back to HTTP renewal. > Example of the case - running distcp between different versions using hfpt. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to renew delegation token over HTTP
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026636#comment-13026636 ] Boris Shkolnik commented on MAPREDUCE-2420: --- +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] No unit tests, but I did manual testing using discp copying from the same cluster or from 0.20 one > JobTracker should be able to renew delegation token over HTTP > - > > Key: MAPREDUCE-2420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.20.4 >Reporter: Boris Shkolnik >Assignee: Boris Shkolnik > Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, > MR-2420.patch, MR2420-2.patch > > > in case JobTracker has to talk to a NameNode running a different version (RPC > version mismatch), Jobtracker should be able to fall back to HTTP renewal. > Example of the case - running distcp between different versions using hfpt. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to renew delegation token over HTTP
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016076#comment-13016076 ] Boris Shkolnik commented on MAPREDUCE-2420: --- committed to branch-20-security. > JobTracker should be able to renew delegation token over HTTP > - > > Key: MAPREDUCE-2420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.20.4 >Reporter: Boris Shkolnik >Assignee: Boris Shkolnik > Attachments: MR-2420-1.patch, MR-2420.patch > > > in case JobTracker has to talk to a NameNode running a different version (RPC > version mismatch), Jobtracker should be able to fall back to HTTP renewal. > Example of the case - running distcp between different versions using hfpt. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to renew delegation token over HTTP
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016060#comment-13016060 ] Devaraj Das commented on MAPREDUCE-2420: This patch handles the case of HDFS token renewals, when the JobTracker's HDFS is running a different version of HDFS than the one the job is trying to use. An example of such a job is distcp (where it uses hftp to talk to a different source cluster to pull data to the cluster where distcp is running). When the job is submitted, the client requests a delegation token over hftp and stuffs it in the job. Today, the NameNode doesn't distinguish between hftp and hdfs accesses, and issues HDFS tokens for both (and the token-kind field in the token has the value as 'HDFS'). Ideally, that should be fixed to have the token-kind as HFTP for hftp accesses. We should have the JobTracker handle all sorts of token renewals, and have a way in which it can look at a token and decide which protocol to use to talk to the server in question. This includes HDFS, HFTP, and also HIVE (where the protocol is thrift). I think this patch is okay for the short term - the JobTracker falls back to hftp if it couldn't renew a token over hdfs. In the patch, there are a bunch of white space changes that aren't required. The string comparisons for exception messages and then instantiating a concrete exception could probably be replaced with a forname() on the string exception. When we fix this issue in trunk, please make it more generic on lines similar to above. > JobTracker should be able to renew delegation token over HTTP > - > > Key: MAPREDUCE-2420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.20.4 >Reporter: Boris Shkolnik >Assignee: Boris Shkolnik > Attachments: MR-2420.patch > > > in case JobTracker has to talk to a NameNode running a different version (RPC > version mismatch), Jobtracker should be able to fall back to HTTP renewal. > Example of the case - running distcp between different versions using hfpt. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
