[ https://issues.apache.org/jira/browse/SPARK-38605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-38605: ------------------------------------ Assignee: Apache Spark > Retrying on file manager operation in HDFSMetadataLog > ----------------------------------------------------- > > Key: SPARK-38605 > URL: https://issues.apache.org/jira/browse/SPARK-38605 > Project: Spark > Issue Type: Improvement > Components: Structured Streaming > Affects Versions: 3.4.0 > Reporter: L. C. Hsieh > Assignee: Apache Spark > Priority: Major > > Currently HDFSMetadataLog uses CheckpointFileManager to do some file > operation like opening metadata file. It is very easy to be affected by > network blips and causes the streaming query failed. Although we can restart > the streaming query, but it takes more time to recover. > Such file operations should be resilient with such situation by retrying. -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org