[
https://issues.apache.org/jira/browse/TEZ-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13918657#comment-13918657
]
Siddharth Seth commented on TEZ-708:
------------------------------------
LocalTezAMRMClientAsync has it's advantages in the sense that the book-keeping,
assignment logic (priorities) etc do not need to be re-created in a
LocalTaskScheduler. However it has the downside that there's a bunch of
functionality there related to locality which becomes irrelevant when running
in local mode.
The decision to run a container within the AM or remotely could be taken during
the DAG schedule phase (explicit), or during Task scheduling - since this is
where available resources is actually known. It may just make sense to modify
the TaskScheduler to deal with allocations from multiple source (RM /
LocalContainers). Explicit local asks must wait for Local containers (specific
resource ask, avoid asking for remote containers in this case).
The task scheduler is also where logic like container re-use (affinity based
scheduling) etc goes in. While that's not required for local mode - it would be
useful to have in a mixed uber-remote execution model.
For just a local mode - I'd go with LocalTezAMRMClientAsync for now, since that
continues to allow TaskScheduler logic to be invoked. Eventually, we can look
at changing TaskScheduler to be aware of "local" resource requirements.
> Avoid asking for new containers in uber-local mode
> --------------------------------------------------
>
> Key: TEZ-708
> URL: https://issues.apache.org/jira/browse/TEZ-708
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Chen He
> Assignee: Jonathan Eagles
>
> Once we finish this task, the TaskSchedulerEventHandler should not ask for
> new container in the Uber-mode.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
