[ https://issues.apache.org/jira/browse/FLINK-5659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15839808#comment-15839808 ]
ASF GitHub Bot commented on FLINK-5659: --------------------------------------- GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/3219 [FLINK-5659] Harden FileBaseUtils#deleteFileOrDirectory on WIndows This PR hardens `FileBaseUtils#deleteFileOrDirectory` when the method is called concurrently on Windows. `AccessDeniedException`s can be thrown when attempting to delete a file while a deletion is in progress. In order to delete a file it has to be opened, and as we all know you can't (easily) delete open files on Windows. For this case we catch the exception, double-check whether the file is deleted by now, and throw the exception if it isn't, assuming a lack of permission prevented the deletion. `DirectoryNotEmptyException`s can be thrown for directories for which a delete call for every file has succeeded, since the OS may be slow in updating the actual FileSystem. For this case we wait for a bit (50 milliseconds, yeah it's that slow) and check whether the directory is empty by now. If it isn't we throw the exception assuming a concurrent creation of a new file in the given directory. Note that neither changes are perfect; they are not guaranteed to prevent the issue. However, it no longer categorically fails for me 🎉, but only once every few hundred runs. You can merge this pull request into a Git repository by running: $ git pull https://github.com/zentol/flink 5659_fileutils_win Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3219.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3219 ---- commit f372899c7cd8f5102421b986be1f1b87e32cf15f Author: zentol <ches...@apache.org> Date: 2017-01-26T14:47:11Z [FLINK-5659] Harden FileBaseUtils#deleteFileOrDirectory on WIndows ---- > FileBaseUtils#deleteFileOrDirectory not thread-safe on Windows > -------------------------------------------------------------- > > Key: FLINK-5659 > URL: https://issues.apache.org/jira/browse/FLINK-5659 > Project: Flink > Issue Type: Bug > Affects Versions: 1.2.0, 1.3.0 > Reporter: Chesnay Schepler > Assignee: Chesnay Schepler > Priority: Trivial > > The {code}FileBaseUtils#deleteFileOrDirectory{code} is not thread-safe on > Windows. > First you will run into AccessDeniedExceptions since one thread tried to > delete a file while another thread was already doing that, for which the file > has to be opened. > Once you resolve those exceptions (by catching them double checking whether > the file still exists), you run into DirectoryNotEmptyExceptions since there > is some wacky timing/visibility issue when deleting files concurrently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)