[jira] [Updated] (MAPREDUCE-3825) MR should not be getting duplicate tokens for a MR Job.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-3825: --- Target Version/s: 0.23.3, 2.0.0, 3.0.0 (was: 0.24.0, 0.23.1) MR should not be getting duplicate tokens for a MR Job. --- Key: MAPREDUCE-3825 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3825 Project: Hadoop Map/Reduce Issue Type: Bug Components: security Affects Versions: 0.23.1, 0.24.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Attachments: MAPREDUCE-3825.patch, TokenCache.pdf, solution4.patch This is the counterpart to HADOOP-7967. MR gets tokens for all input, output and the default filesystem when a MR job is submitted. The APIs in FileSystem make it challenging to avoid duplicate tokens when there are file systems that have embedded filesystems. Here is the original description that Daryn wrote: The token cache currently tries to assume a filesystem's token service key. The assumption generally worked while there was a one to one mapping of filesystem to token. With the advent of multi-token filesystems like viewfs, the token cache will try to use a service key (ie. for viewfs) that will never exist (because it really gets the mounted fs tokens). The descriop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3825) MR should not be getting duplicate tokens for a MR Job.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated MAPREDUCE-3825: --- Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Status: Open (was: Patch Available) About to post a suggested implementation of solution 4. The code isn't in MR so canceling patch to prevent a failed build. MR should not be getting duplicate tokens for a MR Job. --- Key: MAPREDUCE-3825 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3825 Project: Hadoop Map/Reduce Issue Type: Bug Components: security Affects Versions: 0.23.1, 0.24.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Attachments: MAPREDUCE-3825.patch, TokenCache.pdf This is the counterpart to HADOOP-7967. MR gets tokens for all input, output and the default filesystem when a MR job is submitted. The APIs in FileSystem make it challenging to avoid duplicate tokens when there are file systems that have embedded filesystems. Here is the original description that Daryn wrote: The token cache currently tries to assume a filesystem's token service key. The assumption generally worked while there was a one to one mapping of filesystem to token. With the advent of multi-token filesystems like viewfs, the token cache will try to use a service key (ie. for viewfs) that will never exist (because it really gets the mounted fs tokens). The descriop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3825) MR should not be getting duplicate tokens for a MR Job.
[
https://issues.apache.org/jira/browse/MAPREDUCE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated MAPREDUCE-3825:
---
Attachment: solution4.patch
Posting solution 4 with a small twist to satisfy this shared dislike:
bq. (Sanjay) I dislike the notion of passing in the credentials ... which is
then updated by the method - a weird APi. But i can live with it.
I don't want anyone to live in discomfort, so I'm proposing a
{{getDelegationTokens(renewer, creds)}} that _does not modify_ the given creds.
Instead, it returns a {{Credentials}} that contains only the new creds not
present in the given creds.
I added a static flavor of {{FileSystem.getDelegationTokens}} since it seemed
like a more natural location than an external class, but I'm open to
alternatives.
No tests included, pending approval of the approach.
MR should not be getting duplicate tokens for a MR Job.
---
Key: MAPREDUCE-3825
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3825
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: security
Affects Versions: 0.23.1, 0.24.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: MAPREDUCE-3825.patch, TokenCache.pdf, solution4.patch
This is the counterpart to HADOOP-7967.
MR gets tokens for all input, output and the default filesystem when a MR job
is submitted.
The APIs in FileSystem make it challenging to avoid duplicate tokens when
there are file systems that have embedded
filesystems.
Here is the original description that Daryn wrote:
The token cache currently tries to assume a filesystem's token service key.
The assumption generally worked while there was a one to one mapping of
filesystem to token. With the advent of multi-token filesystems like viewfs,
the token cache will try to use a service key (ie. for viewfs) that will
never exist (because it really gets the mounted fs tokens).
The descriop
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
