[spark] branch master updated: [MINOR][TESTS] Clearing residual files after SparkSubmitSuite

[gurwls223]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new c93b410c81e [MINOR][TESTS] Clearing residual files after 
SparkSubmitSuite
c93b410c81e is described below

commit c93b410c81e93cd81bed080aea2c8095d3acb956
Author: panbingkun <pbk1...@gmail.com>
AuthorDate: Mon Jul 31 10:40:22 2023 +0900

    [MINOR][TESTS] Clearing residual files after SparkSubmitSuite
    
    ### What changes were proposed in this pull request?
    The pr aims to clear residual files after SparkSubmitSuite ("SPARK-35084: 
include jars of the --packages in k8s client mode & driver runs inside a POD")
    
    ### Why are the changes needed?
    Clear residual files after UT.
    
    ### Does this PR introduce _any_ user-facing change?
    No.
    
    ### How was this patch tested?
    Manually Test.
    ./build/sbt "core/testOnly *SparkSubmitSuite -- -z \"SPARK-35084: include 
jars of the --packages in k8s client mode & driver runs inside a POD\""
    
    Closes #42201 from panbingkun/minor_SparkSubmitSuite_clean.
    
    Authored-by: panbingkun <pbk1...@gmail.com>
    Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
---
 .../scala/org/apache/spark/deploy/SparkSubmitSuite.scala  | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala 
b/core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala
index 8e2d6e6cf5f..7c787f83c6a 100644
--- a/core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala
+++ b/core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala
@@ -510,8 +510,19 @@ class SparkSubmitSuite
         "my.great.lib.MyLib", "my.great.dep.MyLib")
 
       val appArgs = new SparkSubmitArguments(clArgs)
-      val (_, _, sparkConf, _) = submit.prepareSubmitEnvironment(appArgs)
-      sparkConf.get("spark.jars").contains("mylib") shouldBe true
+      try {
+        val (_, _, sparkConf, _) = submit.prepareSubmitEnvironment(appArgs)
+        sparkConf.get("spark.jars").contains("mylib") shouldBe true
+      } finally {
+        val mainJarPath = Paths.get("my.great.dep_mylib-0.1.jar")
+        val depJarPath = Paths.get("my.great.lib_mylib-0.1.jar")
+        if (Files.exists(mainJarPath)) {
+          Files.delete(mainJarPath)
+        }
+        if (Files.exists(depJarPath)) {
+          Files.delete(depJarPath)
+        }
+      }
     }
   }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org