[ https://issues.apache.org/jira/browse/MAPREDUCE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101298#comment-13101298 ]
Daryn Sharp commented on MAPREDUCE-2764: ---------------------------------------- I have other comments/concerns, but the most important is that setting of the token service in {{HftpFileSystem}} was removed. It must be added back or it will break the ip->host change for hftp tokens which is where it is most needed. Please use {{SecurityUtils.setTokenService}}. We'll also have to add a {{SecurityUtil.getTokenServiceAddress}} that decodes the token's addresses and performs another lookup via {{NetUtil.createSocketAddr}}. All the places directly accessing the service should be updated. > Fix renewal of dfs delegation tokens > ------------------------------------ > > Key: MAPREDUCE-2764 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2764 > Project: Hadoop Map/Reduce > Issue Type: Bug > Reporter: Daryn Sharp > Assignee: Owen O'Malley > Fix For: 0.20.205.0 > > Attachments: MAPREDUCE-2764-2.patch, MAPREDUCE-2764-3.patch, > MAPREDUCE-2764-4.patch, MAPREDUCE-2764.patch, delegation.patch, > token-renew.patch, token-renew.patch > > > The JT may have issues renewing hftp tokens which disrupt long distcp jobs. > The problem is the JT's delegation token renewal code is built on brittle > assumptions. The token's service field contains only the "ip:port" pair. > The renewal process assumes that the scheme must be hdfs. If that fails due > to a {{VersionMismatchException}}, it tries https based on another assumption > that it must be hftp if it's not hdfs. A number of other exceptions, most > commonly {{IOExceptions}}, can be generated which fouls up the renewal since > it won't fallback to https. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira