[
https://issues.apache.org/jira/browse/SPARK-51918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-51918:
-----------------------------------
Labels: pull-request-available (was: )
> logging cutoff at executor exit
> -------------------------------
>
> Key: SPARK-51918
> URL: https://issues.apache.org/jira/browse/SPARK-51918
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 4.0.0, 3.5.5
> Reporter: Andrei Leibovski
> Priority: Minor
> Labels: pull-request-available
>
> Spark app/job logging is cutoff when executors exit.
> {{[ExecutorRunner.killProcess()|https://github.com/apache/spark/blob/676cac092540755e04a5ba7eaaf1e5aea8138930/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala#L103-L108]}}
> does not wait for {{stdoutAppender/stderrAppender}} to stop and does not
> [flush|https://github.com/apache/spark/blob/676cac092540755e04a5ba7eaaf1e5aea8138930/core/src/main/scala/org/apache/spark/util/logging/FileAppender.scala#L85-L87]
> remaining buffer data.
> Noticed this while working on a {{SparkPlugin}} to shutdown per-executor
> resources (connection pools), put some logging statements into
> {{ExecutorPlugin.shutdown()}} but they weren't showing up; confirmed the
> {{shutdown()}} was actually working correctly, the issues was just the
> logging.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
