[ https://issues.apache.org/jira/browse/MAPREDUCE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398726#comment-13398726 ]
Sanjay Radia commented on MAPREDUCE-3825: ----------------------------------------- Summary: # Solution 4 with following changes ** FileSystem#addDelegationTokens returns the newly added tokens ** non-static method and also not-final ** add similar method to AbstractFileSystem # in trunk, and 2.0 remove addDelegationTokens - it was added in 0.23. Some customers are testing 0.23 - and hence we could remove this later from 0.23 # Add convenience method - FileUtil:AddTokens(renewer, path[] ps, credentials) - this can use filesystem or filecontext in its impl. > MR should not be getting duplicate tokens for a MR Job. > ------------------------------------------------------- > > Key: MAPREDUCE-3825 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3825 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: security > Affects Versions: 0.23.1, 0.24.0 > Reporter: Daryn Sharp > Assignee: Daryn Sharp > Attachments: MAPREDUCE-3825.patch, TokenCache.pdf, solution4.patch > > > This is the counterpart to HADOOP-7967. > MR gets tokens for all input, output and the default filesystem when a MR job > is submitted. > The APIs in FileSystem make it challenging to avoid duplicate tokens when > there are file systems that have embedded > filesystems. > Here is the original description that Daryn wrote: > The token cache currently tries to assume a filesystem's token service key. > The assumption generally worked while there was a one to one mapping of > filesystem to token. With the advent of multi-token filesystems like viewfs, > the token cache will try to use a service key (ie. for viewfs) that will > never exist (because it really gets the mounted fs tokens). > The descriop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira