This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch branch-3.4 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.4 by this push: new 58a4a49389a5 [SPARK-47236][CORE] Fix `deleteRecursivelyUsingJavaIO` to skip non-existing file input 58a4a49389a5 is described below commit 58a4a49389a5f9979f7dabc5320116a212eb4bdb Author: Dongjoon Hyun <dh...@apple.com> AuthorDate: Thu Feb 29 19:08:15 2024 -0800 [SPARK-47236][CORE] Fix `deleteRecursivelyUsingJavaIO` to skip non-existing file input ### What changes were proposed in this pull request? This PR aims to fix `deleteRecursivelyUsingJavaIO` to skip non-existing file input. ### Why are the changes needed? `deleteRecursivelyUsingJavaIO` is a fallback of `deleteRecursivelyUsingUnixNative`. We should have identical capability. Currently, it fails. ``` [info] java.nio.file.NoSuchFileException: /Users/dongjoon/APACHE/spark-merge/target/tmp/spark-e264d853-42c0-44a2-9a30-22049522b04f [info] at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) [info] at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) [info] at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) [info] at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) [info] at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:148) [info] at java.base/java.nio.file.Files.readAttributes(Files.java:1851) [info] at org.apache.spark.network.util.JavaUtils.deleteRecursivelyUsingJavaIO(JavaUtils.java:126) ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? This is difficult to test this `private static` Java method. I tested this with #45344 . ### Was this patch authored or co-authored using generative AI tooling? No. Closes #45346 from dongjoon-hyun/SPARK-47236. Authored-by: Dongjoon Hyun <dh...@apple.com> Signed-off-by: Dongjoon Hyun <dh...@apple.com> (cherry picked from commit 1cd7bab5c5c2bd8d595b131c88e6576486dbf123) Signed-off-by: Dongjoon Hyun <dh...@apple.com> --- .../src/main/java/org/apache/spark/network/util/JavaUtils.java | 1 + 1 file changed, 1 insertion(+) diff --git a/common/network-common/src/main/java/org/apache/spark/network/util/JavaUtils.java b/common/network-common/src/main/java/org/apache/spark/network/util/JavaUtils.java index 7e410e9eab22..59744ec5748a 100644 --- a/common/network-common/src/main/java/org/apache/spark/network/util/JavaUtils.java +++ b/common/network-common/src/main/java/org/apache/spark/network/util/JavaUtils.java @@ -124,6 +124,7 @@ public class JavaUtils { private static void deleteRecursivelyUsingJavaIO( File file, FilenameFilter filter) throws IOException { + if (!file.exists()) return; BasicFileAttributes fileAttributes = Files.readAttributes(file.toPath(), BasicFileAttributes.class); if (fileAttributes.isDirectory() && !isSymlink(file)) { --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org