Repository: spark Updated Branches: refs/heads/branch-1.5 402c0ca9a -> d9d4bdea2
[SPARK-9829] [WEBUI] Display the update value for peak execution memory The peak execution memory is not correct because it shows the sum of finished tasks' values when a task finishes. This PR fixes it by using the update value rather than the accumulator value. Author: zsxwing <zsxw...@gmail.com> Closes #8121 from zsxwing/SPARK-9829. (cherry picked from commit 4e3f4b934f74e8c7c06f4940d6381343f9fd4918) Signed-off-by: Andrew Or <and...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d9d4bdea Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d9d4bdea Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d9d4bdea Branch: refs/heads/branch-1.5 Commit: d9d4bdea2d711c50273012206832c6ee1a8d90d6 Parents: 402c0ca Author: zsxwing <zsxw...@gmail.com> Authored: Tue Aug 11 23:23:17 2015 -0700 Committer: Andrew Or <and...@databricks.com> Committed: Tue Aug 11 23:23:31 2015 -0700 ---------------------------------------------------------------------- core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/d9d4bdea/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---------------------------------------------------------------------- diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala index 0c94204..fb4556b 100644 --- a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala +++ b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala @@ -860,7 +860,7 @@ private[ui] class TaskDataSource( } val peakExecutionMemoryUsed = taskInternalAccumulables .find { acc => acc.name == InternalAccumulator.PEAK_EXECUTION_MEMORY } - .map { acc => acc.value.toLong } + .map { acc => acc.update.getOrElse("0").toLong } .getOrElse(0L) val maybeInput = metrics.flatMap(_.inputMetrics) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
