dimtiris kanoute created SPARK-39580: ----------------------------------------
Summary: Write event logs in a continuous manner Key: SPARK-39580 URL: https://issues.apache.org/jira/browse/SPARK-39580 Project: Spark Issue Type: New Feature Components: Spark Core Affects Versions: 3.3.0 Reporter: dimtiris kanoute We are using Spark Thrift Server as a service to run Spark SQL queries. Currently Spark writes *eventlogs* once the job is done/ gracefully killed. This means that in case of Spark thrift Server when it runs as one job in case of unexpected error ( i.e. OOM ) we cannot access the logs after, as it doesn't write them due to the unexpected shut down. We would like to suggest as an improvement the functionality for sparks *to write the event logs to a cloud storage in a continuous manner.* -- This message was sent by Atlassian Jira (v8.20.7#820007) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org