Some corrections.
On Fri, Sep 26, 2014 at 5:32 PM, praveen seluka <praveen.sel...@gmail.com>
wrote:
> Can someone explain the motivation behind passing executorAdded event to
> DAGScheduler ? *DAGScheduler *does *submitWaitingStages *when *executorAdded
> *method is called by *TaskSchedulerImpl*. I see some issue in the below
> code,
>
> *TaskSchedulerImpl.scala code*
> if (!executorsByHost.contains(o.host)) {
> executorsByHost(o.host) = new HashSet[String]()
> executorAdded(o.executorId, o.host)
> newExecAvail = true
> }
>
> Note that executorAdded is called only when there is a new host and not
> for every new executor. For instance, there can be two executors in the
> same host and in this case the DAGscheduler is notified only once. If this
> is indeed an issue, I would like to submit a patch for this quickly. [cc
> Andrew Or]
>
> - Praveen
>
>
>
