Repository: spark Updated Branches: refs/heads/branch-2.3 c7a0dea46 -> a1ee6f1fc
[SPARK-23470][UI] Use first attempt of last stage to define job description. This is much faster than finding out what the last attempt is, and the data should be the same. There's room for improvement in this page (like only loading data for the jobs being shown, instead of loading all available jobs and sorting them), but this should bring performance on par with the 2.2 version. Author: Marcelo Vanzin <van...@cloudera.com> Closes #20644 from vanzin/SPARK-23470. (cherry picked from commit 2ba77ed9e51922303e3c3533e368b95788bd7de5) Signed-off-by: Sameer Agarwal <samee...@apache.org> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a1ee6f1f Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a1ee6f1f Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a1ee6f1f Branch: refs/heads/branch-2.3 Commit: a1ee6f1fc543120763f1b373bb31bc6d84004318 Parents: c7a0dea Author: Marcelo Vanzin <van...@cloudera.com> Authored: Tue Feb 20 17:54:06 2018 -0800 Committer: Sameer Agarwal <samee...@apache.org> Committed: Tue Feb 20 17:54:17 2018 -0800 ---------------------------------------------------------------------- core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/a1ee6f1f/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---------------------------------------------------------------------- diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala index a4710f6..08a927a 100644 --- a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala +++ b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala @@ -1040,7 +1040,7 @@ private[ui] object ApiHelper { } def lastStageNameAndDescription(store: AppStatusStore, job: JobData): (String, String) = { - val stage = store.asOption(store.lastStageAttempt(job.stageIds.max)) + val stage = store.asOption(store.stageAttempt(job.stageIds.max, 0)) (stage.map(_.name).getOrElse(""), stage.flatMap(_.description).getOrElse(job.name)) } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
