[GitHub] deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI
deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI URL: https://github.com/apache/spark/pull/23637#issuecomment-458857165 Retest please. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI
deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI URL: https://github.com/apache/spark/pull/23637#issuecomment-458856363 Jenkins retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI
deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI URL: https://github.com/apache/spark/pull/23637#issuecomment-458826733 I get it. Thank you @gengliangwang! I think `Unknown Job without stages` will be better. I will change it. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI
deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI URL: https://github.com/apache/spark/pull/23637#issuecomment-458793308 @srowen Yes, I argee with you. @gengliangwang Could you give me some suggestions? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI
deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI URL: https://github.com/apache/spark/pull/23637#issuecomment-458788215 Here is the lastest PR screenshot:  This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI
deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0
partition job in WebUI
URL: https://github.com/apache/spark/pull/23637#issuecomment-458787898
@gengliangwang @srowen Sorry, maybe I didn't make it clear. The job.name
will be ` (Unknown Stage Name)` in this case whatever changing PR or not. I
change the original PR is to fit with the code logic.
We can see the default job.name will be sent in:
```
// AppStatusListener.scala
val lastStageInfo = event.stageInfos.sortBy(_.stageId).lastOption
val lastStageName = lastStageInfo.map(_.name).getOrElse("(Unknown Stage
Name)")
val jobGroup = Option(event.properties)
.flatMap { p => Option(p.getProperty(SparkContext.SPARK_JOB_GROUP_ID))
}
val sqlExecutionId = Option(event.properties)
.flatMap(p =>
Option(p.getProperty(SQL_EXECUTION_ID_KEY)).map(_.toLong))
val job = new LiveJob(
event.jobId,
lastStageName,
if (event.time > 0) Some(new Date(event.time)) else None,
event.stageIds,
jobGroup,
numTasks,
sqlExecutionId)
liveJobs.put(event.jobId, job)
liveUpdate(job, now)
```
```
private class LiveJob(
val jobId: Int,
name: String,
val submissionTime: Option[Date],
val stageIds: Seq[Int],
jobGroup: Option[String],
numTasks: Int,
sqlExecutionId: Option[Long]) extends LiveEntity {
```
So, if necessary， we can set job.name in two methods:
> 1.Add more info in SparkListenerJobStart
> 2.Constructing a empty stage contains a callsite in StageInfo?
But it will be more complex and get some compatibility problem.
In the end, I change it to original. Mybe the UI is kind of confusing. This
is acceptable because the job description in here places the last stage info
originally. In this case, we have no stage. So showing "Unknown Stage Name" is
acceptable I think.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI
deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI URL: https://github.com/apache/spark/pull/23637#issuecomment-458421732 Jenkins help me test. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI
deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI URL: https://github.com/apache/spark/pull/23637#issuecomment-458003956 Yes, it takes no time at all and It always succeeds. I think in this case, we could not check it twich. Event not creating a JobWaiter if necessary This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI
deshanxiao commented on issue #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI URL: https://github.com/apache/spark/pull/23637#issuecomment-457133975 It looks like:  This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
