Jakub Dubovsky created SPARK-23283:
--------------------------------------
Summary: Probable mistake in hasLaunchedTask condition
Key: SPARK-23283
URL: https://issues.apache.org/jira/browse/SPARK-23283
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.2.1
Reporter: Jakub Dubovsky
In class {code}TaskSchedulerImpl{code} there is a
[variable|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala#L97]
tracking whether some task was already launched. It is then set to true
[here|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala#L377]
whenever some task is launched.
I believe the
[condition|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala#L376]
should be {code}tasks.map(_.size).sum > 0{code} instead of current
{code}tasks.size > 0{code}
Current condition asks whether there are some resource offers while it should
ask whether there were some launched tasks. In some cases it might happen that
resources are offered but no tasks launched. In such a case no warning is
issued.
Can you confirm this is valid?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
