[
https://issues.apache.org/jira/browse/MAPREDUCE-4427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414014#comment-13414014
]
Thomas Graves commented on MAPREDUCE-4427:
------------------------------------------
note - I agree this is good stuff!
Sorry I meant add it to the client side of the app - which you've answered,
thanks.
I was thinking of any multi-tenant cluster, where people could start abusing
the option and running things on gateway machines and overloading those
gateways or perhaps just causing traffic between outside machines that SEs
don't want or expect. But thinking about this more, there are plenty of other
ways to cause issues like that so I'm good with leaving this off. If a use case
ever comes up we can revisit.
Another question or atleast something to think about - it appears it still goes
through all the queue checks when submitting the application. I'm wondering if
some of those checks might not apply in this case - for instance max am
resources (maxActiveApplications) doesn't really apply because external AM's
aren't using queue capacity for the AM itself. That might not be a big issue
right now if people use this just for debug, but if this is used say by AM's to
launch other AM's in arbitrary containers it might be more of an issue.
What happens when you kill one of these applications? The RM can't really
force kill it - so does it just kill all containers its requested and "block"
the AM from communicating.
minor nitpicky comments to consider:
Can we just remove the commented out code in the container executor: +
//lfs.mkdir(containerDir, null, false);
Could you also clarify what the comment there means "+ // Without this app
with no cache files cannot launch tasks"? Is it supposed to be "without this,
app with" And then is "this" passing true into lfs.mkdir as last parameter?
> Enable the RM to work with AM's that are not managed by it
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4427
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4427
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 3.0.0
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Labels: mrv2
> Attachments: MAPREDUCE-4427-1.patch, MAPREDUCE-4427-2.patch
>
>
> Currently, the RM itself manages the AM by allocating a container for it and
> negotiating the launch on the NodeManager and manages the AM lifecycle.
> Thereafter, the AM negotiates resources with the RM and launches tasks to do
> the real work.
> It would be a useful improvement to enhance this model by allowing the AM to
> be launched independently by the client without requiring the RM. These AM's
> would be launched on a gateway machine that can talk to the cluster. This
> would open up new use cases such as the following
> 1) Easy debugging of AM, specially during initial development. Having the AM
> launched on an arbitrary cluster node makes it hard to looks at logs or
> attach a debugger to the AM. If it can be launched locally then these tasks
> would be easier.
> 2) Running AM's that need special privileges that may not be available on
> machines managed by the NodeManager
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
