Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21758#discussion_r204316098
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -291,6 +292,11 @@ private[spark] class TaskSchedulerImpl(
executorIdToRunningTaskIds(execId).add(tid)
availableCpus(i) -= CPUS_PER_TASK
assert(availableCpus(i) >= 0)
+ // Only update hosts for a barrier task.
+ if (taskSet.isBarrier) {
+ // The executor address is expected to be non empty.
+ addressesWithDescs += Tuple2(shuffledOffers(i).address.get,
task)
--- End diff --
nit: `Tuple2(shuffledOffers(i).address.get, task)` can be
`(shuffledOffers(i).address.get -> task)`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
