[
https://issues.apache.org/jira/browse/MAPREDUCE-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144330#comment-13144330
]
Daryn Sharp commented on MAPREDUCE-3359:
----------------------------------------
Although I helped contrib to this jira, I have concerns regarding its safety
and hope it's a temporary fix.
I feel the config is of questionable value since a misbehaving client may
"forget" to cancel its tokens. The NN is holding tokens in memory so it could
lead to a potential, and perhaps unintentional, denial of service attack.
When tokens are shared between jobs, it's ambiguous as to when the tokens can
be safely cancelled. How does a client know that other running or queued jobs
are using the tokens? If the client intends to launch multiple jobs, but the
client errors out, the tokens can't be cancelled or "very bad" things will
happen to the jobs already submitted. Tasks will pound on the NN every second
with the bad token, and yarn tasks appear to run "forever" if rpc connections
fail. In a test env, orphaned tasks had pounded on the NN every second for a
*month*.
Allowing the RM to cancel tokens when the job completes, which implies tokens
are good for one and only one job submission, removes the ambiguity of when
it's safe to cancel the tokens. This reduces the chance of a dos attack on the
NN, and from a security perspective closes the window of exposure vs. allowing
tokens to linger until their lifetime expires.
> Yarn clients / AM should be able to provide config options to the RM / NM
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3359
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3359
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Attachments: MR-3359.txt
>
>
> The RM and NM do not read a job's configuration. Clients / AMs should however
> be able to configure certain parameters for the RM/NM on a per app basis -
> like the Log Retention policy, token cancellation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
