Repository: spark
Updated Branches:
  refs/heads/branch-1.4 1d49d8c3f -> aee046dfa


[SPARK-7932] Fix misleading scheduler delay visualization

The existing code rounds down to the nearest percent when computing the 
proportion
of a task's time that was spent on each phase of execution, and then computes
the scheduler delay proportion as 100 - sum(all other proportions).  As a 
result,
a few extra percent can end up in the scheduler delay. This commit eliminates
the rounding so that the time visualizations correspond properly to the real 
times.

sarutak If you could take a look at this, that would be great! Not sure if 
there's a good
reason to round here that I missed.

cc shivaram

Author: Kay Ousterhout <kayousterh...@gmail.com>

Closes #6484 from kayousterhout/SPARK-7932 and squashes the following commits:

1723cc4 [Kay Ousterhout] [SPARK-7932] Fix misleading scheduler delay 
visualization

(cherry picked from commit 04ddcd4db7801abefa9c9effe5d88413b29d713b)
Signed-off-by: Kay Ousterhout <kayousterh...@gmail.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/aee046df
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/aee046df
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/aee046df

Branch: refs/heads/branch-1.4
Commit: aee046dfa111b4323edd5f4ccb36075449492952
Parents: 1d49d8c
Author: Kay Ousterhout <kayousterh...@gmail.com>
Authored: Thu May 28 22:09:49 2015 -0700
Committer: Kay Ousterhout <kayousterh...@gmail.com>
Committed: Thu May 28 22:09:59 2015 -0700

----------------------------------------------------------------------
 core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/aee046df/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala 
b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
index 31e2e7f..b83a49f 100644
--- a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
+++ b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
@@ -527,7 +527,7 @@ private[ui] class StagePage(parent: StagesTab) extends 
WebUIPage("stage") {
         minLaunchTime = launchTime.min(minLaunchTime)
         maxFinishTime = finishTime.max(maxFinishTime)
 
-        def toProportion(time: Long) = (time.toDouble / totalExecutionTime * 
100).toLong
+        def toProportion(time: Long) = time.toDouble / totalExecutionTime * 100
 
         val metricsOpt = taskUIData.taskMetrics
         val shuffleReadTime =


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org

Reply via email to