Github user smurakozi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20251#discussion_r161276636
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala ---
@@ -429,20 +429,40 @@ private[ui] class JobDataSource(
val formattedDuration = duration.map(d =>
UIUtils.formatDuration(d)).getOrElse("Unknown")
val submissionTime = jobData.submissionTime
val formattedSubmissionTime =
submissionTime.map(UIUtils.formatDate).getOrElse("Unknown")
- val jobDescription =
UIUtils.makeDescription(jobData.description.getOrElse(""),
- basePath, plainText = false)
+
+ val lastStage = {
+ val stageAttempts = jobData.stageIds.flatMap(store.stageData(_))
+ if (!stageAttempts.isEmpty) {
+ val lastId = stageAttempts.map(_.stageId).max
+ stageAttempts.find(_.stageId == lastId)
+ } else {
+ None
+ }}
+
+ val jobDescription = jobData.description
+ .getOrElse(lastStage.flatMap(_.description).
+ getOrElse(jobData.name))
+
+ val lastStageName = lastStage.map(_.name).getOrElse(jobData.name)
+
+ val lastStageDescription = lastStage.flatMap(_.description)
+ .getOrElse(jobData.description
+ .getOrElse(jobData.name))
--- End diff --
No need to break, but for me, it shows the structure and logic of the code
better this way.
I've formatted the calculation of the two descriptions symmetric and I'm
also happy to remove unnecessary line breaks if you recommend it.
Thanks for the review and your comments @srowen
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
