This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new df52c80cc27 [SPARK-42358][CORE] Send ExecutorUpdated with the message
argument in Master.removeWorker
df52c80cc27 is described below
commit df52c80cc27be85b5a07a83c243d82c6bbee0309
Author: Bo Zhang <bo.zh...@databricks.com>
AuthorDate: Tue Feb 7 21:09:00 2023 -0800
[SPARK-42358][CORE] Send ExecutorUpdated with the message argument in
Master.removeWorker
### What changes were proposed in this pull request?
Currently field `message` in `ExecutorUpdated` sent in Master.removeWorker
is always `Some("worker lost")`. This PR change to use the method argument
`message` instead.
### Why are the changes needed?
This is to provide more information in the message to better differentiate
the causes of worker removals.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing tests.
Closes #39903 from bozhang2820/spark-42358.
Lead-authored-by: Bo Zhang <bo.zh...@databricks.com>
Co-authored-by: Bo Zhang <bozhang2...@gmail.com>
Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
---
core/src/main/scala/org/apache/spark/deploy/master/Master.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/core/src/main/scala/org/apache/spark/deploy/master/Master.scala
b/core/src/main/scala/org/apache/spark/deploy/master/Master.scala
index 6085a41963e..9b11440b7e4 100644
--- a/core/src/main/scala/org/apache/spark/deploy/master/Master.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/master/Master.scala
@@ -962,7 +962,7 @@ private[deploy] class Master(
for (exec <- worker.executors.values) {
logInfo("Telling app of lost executor: " + exec.id)
exec.application.driver.send(ExecutorUpdated(
- exec.id, ExecutorState.LOST, Some("worker lost"), None,
Some(worker.host)))
+ exec.id, ExecutorState.LOST, Some(s"worker lost: $msg"), None,
Some(worker.host)))
exec.state = ExecutorState.LOST
exec.application.removeExecutor(exec)
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
