Gengliang Wang created SPARK-47574: -------------------------------------- Summary: Introduce Structured Logging Framework Key: SPARK-47574 URL: https://issues.apache.org/jira/browse/SPARK-47574 Project: Spark Issue Type: Sub-task Components: Project Infra Affects Versions: 4.0.0 Reporter: Gengliang Wang Assignee: Gengliang Wang
Introduce Structured Logging Framework as per [https://docs.google.com/document/d/1rATVGmFLNVLmtxSpWrEceYm7d-ocgu8ofhryVs4g3XU/edit?usp=sharing] . * The default logging output format will be json lines. For example {code:java} { "ts":"2023-03-12T12:02:46.661-0700", "level":"ERROR", "msg":"Cannot determine whether executor 289 is alive or not", "context":{ "executor_id":"289" }, "exception":{ "class":"org.apache.spark.SparkException", "msg":"Exception thrown in awaitResult", "stackTrace":"..." }, "source":"BlockManagerMasterEndpoint" } {code} * Introduce a new configuration `spark.log.structuredLogging.enabled` to control the default log4j configuration. Users can set it as false to get plain text log outputs * The change will start with logError method. The Logging API will be changed from `logError(s"Cannot determine whether executor $executorId is alive or not.", e)` to `logError(log"Cannot determine whether executor ${MDC(EXECUTOR_ID, executorId)} is alive or not.", e)` -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org