Anish Shrigondekar created SPARK-42668: ------------------------------------------
Summary: Catch exception while trying to close compressed stream in HDFSStateStoreProvider abort Key: SPARK-42668 URL: https://issues.apache.org/jira/browse/SPARK-42668 Project: Spark Issue Type: Task Components: Structured Streaming Affects Versions: 3.4.0 Reporter: Anish Shrigondekar Catch exception while trying to close compressed stream in HDFSStateStoreProvider abort We have seen some cases where the task exits as cancelled/failed which triggers the abort in the task completion listener for HDFSStateStoreProvider. As part of this, we cancel the backing stream and close the compressed stream. However, different stores such as Azure blob store could throw exceptions which are not caught in the current path, leading to job failures. This change proposes to fix this issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org