[ https://issues.apache.org/jira/browse/SPARK-31769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113874#comment-17113874 ]
Wenchen Fan commented on SPARK-31769: ------------------------------------- SGTM. We can include driver side information like app id. > Add support of MDC in spark driver logs. > ---------------------------------------- > > Key: SPARK-31769 > URL: https://issues.apache.org/jira/browse/SPARK-31769 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Affects Versions: 3.0.0 > Reporter: Izek Greenfield > Priority: Major > > In order to align driver logs with spark executors logs, add support to log > MDC state for properties that are shared between the driver the server > executors (e.g. application name. task id) > The use case is applicable in 2 cases: > # streaming > # running spark under job server (in this case you have a long-running spark > context that handles many tasks from different sources). > See also [SPARK-8981|https://issues.apache.org/jira/browse/SPARK-8981] that > handles the MDC in executor context. > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org