Hu Ziqian created SPARK-42744: --------------------------------- Summary: delete uploaded file when job finish Key: SPARK-42744 URL: https://issues.apache.org/jira/browse/SPARK-42744 Project: Spark Issue Type: Improvement Components: Kubernetes Affects Versions: 3.1.2 Reporter: Hu Ziqian
On kubernetes, spark use spark.kubernetes.file.upload.path to upload local files to Hadoop compatible file system. But spark do not delete those files at all. In this issue, we add a configuration spark.kubernetes.uploaded.file.delete.on.termination to delete those files by driver when job finishs. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org