In Yarn, we can easily have multiple containers allocated in the same node.
On Fri, Sep 26, 2014 at 6:05 PM, Nan Zhu <zhunanmcg...@gmail.com> wrote:
> just a quick reply, we cannot start two executors in the same host for a
> single application in the standard deployment (one worker per machine)
>
> I’m not sure if it will create an issue when you have multiple workers in
> the same host, as submitWaitingStages is called everywhere and I never
> try such a deployment mode
>
> Best,
>
> --
> Nan Zhu
>
> On Friday, September 26, 2014 at 8:02 AM, praveen seluka wrote:
>
> Can someone explain the motivation behind passing executorAdded event to
> DAGScheduler ? *DAGScheduler *does *submitWaitingStages *when *executorAdded
> *method is called by *TaskSchedulerImpl*. I see some issue in the below
> code,
>
> *TaskSchedulerImpl.scala code*
> if (!executorsByHost.contains(o.host)) {
> executorsByHost(o.host) = new HashSet[String]()
> executorAdded(o.executorId, o.host)
> newExecAvail = true
> }
>
> Note that executorAdded is called only when there is a new host and not
> for every new executor. For instance, there can be two executors in the
> same host and in this case. (But DAGScheduler executorAdded is notified
> only for new host - so only once in this case). If this is indeed an issue,
> I would like to submit a patch for this quickly. [cc Andrew Or]
>
> - Praveen
>
>
>
>
