[
https://issues.apache.org/jira/browse/FLINK-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094297#comment-17094297
]
frank wang commented on FLINK-16198:
------------------------------------
I test the example,
1. i delete the some code, the test is ok
@Test
public void testCompressionOnRelativePath() throws IOException {
final java.nio.file.Path compressDir =
tmp.newFolder("compressDir").toPath();
delete this code
/*final java.nio.file.Path relativeCompressDir =
Paths.get(new
File("").getAbsolutePath()).relativize(compressDir);
*/
verifyDirectoryCompression(compressDir);
}
2. i think the problems is duing to concurrency,
private static void guardIfWindows(ThrowingConsumer<File, IOException> toRun,
File file) throws IOException {
// for windows, we synchronize on a global lock, to prevent
concurrent delete issues
// >
// in the future, we may want to find either a good way of
working around file visibility
// in Windows under concurrent operations (the behavior seems
completely unpredictable)
// or make this locking more fine grained, for example on
directory path prefixes
synchronized (WINDOWS_DELETE_LOCK) {
for (int attempt = 1; attempt <= 10; attempt++) {
try {
toRun.accept(file);
break;
}
catch (AccessDeniedException e) {
// ah, windows...
}
// briefly wait and fall through the loop
try {
Thread.sleep(1);
} catch (InterruptedException e) {
// restore the interruption flag and
error out of the method
Thread.currentThread().interrupt();
throw new IOException("operation
interrupted");
}
}
}
}
because it hasnot concurrency control on mac or linux, may be we can modify
some code to follow
> FileUtilsTest fails on Mac OS
> -----------------------------
>
> Key: FLINK-16198
> URL: https://issues.apache.org/jira/browse/FLINK-16198
> Project: Flink
> Issue Type: Bug
> Components: FileSystems, Tests
> Affects Versions: 1.11.0
> Reporter: Andrey Zagrebin
> Priority: Blocker
> Labels: starter
> Fix For: 1.11.0
>
>
> The following tests fail if run on Mac OS (IDE/maven).
>
> FileUtilsTest.testCompressionOnRelativePath:
> {code:java}
> java.nio.file.NoSuchFileException:
> ../../../../../var/folders/67/v4yp_42d21j6_n8k1h556h0c0000gn/T/junit6496651678375117676/compressDir/rootDirjava.nio.file.NoSuchFileException:
>
> ../../../../../var/folders/67/v4yp_42d21j6_n8k1h556h0c0000gn/T/junit6496651678375117676/compressDir/rootDir
> at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at
> sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384)
> at java.nio.file.Files.createDirectory(Files.java:674) at
> org.apache.flink.util.FileUtilsTest.verifyDirectoryCompression(FileUtilsTest.java:440)
> at
> org.apache.flink.util.FileUtilsTest.testCompressionOnRelativePath(FileUtilsTest.java:261)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
> org.junit.rules.RunRules.evaluate(RunRules.java:20) at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
> org.junit.runner.JUnitCore.run(JUnitCore.java:137) at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> {code}
>
> FileUtilsTest.testDeleteDirectoryConcurrently:
> {code:java}
> java.nio.file.FileSystemException:
> /var/folders/67/v4yp_42d21j6_n8k1h556h0c0000gn/T/junit7558825557740784886/junit3566161583262218465/ab1fa0bde8b22cad58b717508c7a7300/121fdf5f7b057183843ed2e1298f9b66/6598025f390d3084d69c98b36e542fe2/8db7cd9c063396a19a86f5b63ce53f66:
> Invalid argument at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)
> at
> sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:108)
> at java.nio.file.Files.deleteIfExists(Files.java:1165)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectoryInternal(FileUtils.java:324)
> at org.apache.flink.util.FileUtils.guardIfWindows(FileUtils.java:391)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectory(FileUtils.java:258)
> at
> org.apache.flink.util.FileUtils.cleanDirectoryInternal(FileUtils.java:376)
> at
> org.apache.flink.util.FileUtils.deleteDirectoryInternal(FileUtils.java:335)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectoryInternal(FileUtils.java:320)
> at org.apache.flink.util.FileUtils.guardIfWindows(FileUtils.java:391)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectory(FileUtils.java:258)
> at
> org.apache.flink.util.FileUtils.cleanDirectoryInternal(FileUtils.java:376)
> at
> org.apache.flink.util.FileUtils.deleteDirectoryInternal(FileUtils.java:335)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectoryInternal(FileUtils.java:320)
> at org.apache.flink.util.FileUtils.guardIfWindows(FileUtils.java:391)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectory(FileUtils.java:258)
> at
> org.apache.flink.util.FileUtils.cleanDirectoryInternal(FileUtils.java:376)
> at
> org.apache.flink.util.FileUtils.deleteDirectoryInternal(FileUtils.java:335)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectoryInternal(FileUtils.java:320)
> at org.apache.flink.util.FileUtils.guardIfWindows(FileUtils.java:391)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectory(FileUtils.java:258)
> at
> org.apache.flink.util.FileUtils.cleanDirectoryInternal(FileUtils.java:376)
> at
> org.apache.flink.util.FileUtils.deleteDirectoryInternal(FileUtils.java:335)
> at org.apache.flink.util.FileUtils.guardIfWindows(FileUtils.java:391)
> at org.apache.flink.util.FileUtils.deleteDirectory(FileUtils.java:276)
> at
> org.apache.flink.util.FileUtilsTest$Deleter.go(FileUtilsTest.java:515)
> at
> org.apache.flink.core.testutils.CheckedThread.run(CheckedThread.java:74)java.nio.file.FileSystemException:
>
> /var/folders/67/v4yp_42d21j6_n8k1h556h0c0000gn/T/junit7558825557740784886/junit3566161583262218465/ab1fa0bde8b22cad58b717508c7a7300/121fdf5f7b057183843ed2e1298f9b66/6598025f390d3084d69c98b36e542fe2/8db7cd9c063396a19a86f5b63ce53f66:
> Invalid argument
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)
> at
> sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:108)
> at java.nio.file.Files.deleteIfExists(Files.java:1165)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectoryInternal(FileUtils.java:324)
> at org.apache.flink.util.FileUtils.guardIfWindows(FileUtils.java:391)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectory(FileUtils.java:258)
> at
> org.apache.flink.util.FileUtils.cleanDirectoryInternal(FileUtils.java:376)
> at
> org.apache.flink.util.FileUtils.deleteDirectoryInternal(FileUtils.java:335)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectoryInternal(FileUtils.java:320)
> at org.apache.flink.util.FileUtils.guardIfWindows(FileUtils.java:391)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectory(FileUtils.java:258)
> at
> org.apache.flink.util.FileUtils.cleanDirectoryInternal(FileUtils.java:376)
> at
> org.apache.flink.util.FileUtils.deleteDirectoryInternal(FileUtils.java:335)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectoryInternal(FileUtils.java:320)
> at org.apache.flink.util.FileUtils.guardIfWindows(FileUtils.java:391)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectory(FileUtils.java:258)
> at
> org.apache.flink.util.FileUtils.cleanDirectoryInternal(FileUtils.java:376)
> at
> org.apache.flink.util.FileUtils.deleteDirectoryInternal(FileUtils.java:335)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectoryInternal(FileUtils.java:320)
> at org.apache.flink.util.FileUtils.guardIfWindows(FileUtils.java:391)
> at
> org.apache.flink.util.FileUtils.deleteFileOrDirectory(FileUtils.java:258)
> at
> org.apache.flink.util.FileUtils.cleanDirectoryInternal(FileUtils.java:376)
> at
> org.apache.flink.util.FileUtils.deleteDirectoryInternal(FileUtils.java:335)
> at org.apache.flink.util.FileUtils.guardIfWindows(FileUtils.java:391)
> at org.apache.flink.util.FileUtils.deleteDirectory(FileUtils.java:276)
> at
> org.apache.flink.util.FileUtilsTest$Deleter.go(FileUtilsTest.java:515)
> at
> org.apache.flink.core.testutils.CheckedThread.run(CheckedThread.java:74)%MCEPASTEBIN%{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
