[ https://issues.apache.org/jira/browse/IO-755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17436000#comment-17436000 ]
Gary D. Gregory edited comment on IO-755 at 10/29/21, 6:11 PM: --------------------------------------------------------------- How about using {{PathUtils.walk(Path, PathFilter, int, boolean, FileVisitOption...)}} and passing {{false}} to {{readAttributes}} which is what {{FileUtils.streamFiles(File, boolean, String...)}} and consequently {{FileUtils.iterateFiles(File, String[], boolean)}} and {{FileUtils.listFiles(File, String[], boolean)}} do. It would be nice if {{java.nio.file.FileVisitOption}} had a {{DONT_READ_FILE_ATTRIBUTES}} value I suppose as it looks like attribute reading in baked-in file walking. was (Author: garydgregory): How about using {{PathUtils.walk(Path, PathFilter, int, boolean, FileVisitOption...)}} and passing {{false}} to {{readAttributes}} which is what {{FileUtils.streamFiles(File, boolean, String...)}} and consequently {{FileUtils.iterateFiles(File, String[], boolean)}} and {{FileUtils.listFiles(File, String[], boolean)}} do. It would be nice if {{java.nio.file.FileVisitOption}} had a {{DONT_READ_FILE_ATTRIBUTES}} value I suppose. > Using FileUtils.listFiles with background changes fails on Linux > ---------------------------------------------------------------- > > Key: IO-755 > URL: https://issues.apache.org/jira/browse/IO-755 > Project: Commons IO > Issue Type: Bug > Components: Utilities > Affects Versions: 2.9.0, 2.10.0, 2.11.0 > Reporter: Alexis Suter > Priority: Major > Labels: listFiles > > When calling {{FileUtils.listFiles}} and a file is being removed from this or > a subdirectory during the execution the following Error occurs on linux. > h3. Stacktrace > {code:trace} > Caused by: java.nio.file.NoSuchFileException: /** > at > java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) > at > java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) > at > java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) > at > java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149) > at > java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.base/java.nio.file.Files.readAttributes(Files.java:1764) > at > java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219) > at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276) > at java.base/java.nio.file.FileTreeWalker.next(FileTreeWalker.java:373) > at java.base/java.nio.file.Files.walkFileTree(Files.java:2761) > at org.apache.commons.io.FileUtils.listAccumulate(FileUtils.java:2076) > at org.apache.commons.io.FileUtils.listFiles(FileUtils.java:2132) > {code} > h3. Upgrade of apache commons-io > The version of the dependency was upgraded from 2.8 to 2.11. In 2.9 the > library started using java.nio instead of java.io which might behave > differently on unix, when a file is not existent during processing of > directories. > h3. Causing functionality in our implementation > In some part we have multithreaded access to the filesystem. A folder is > accessed from multiple threads which access the folder with > {{org.apache.commons.io.FileUtils#listFiles(java.io.File, > org.apache.commons.io.filefilter.IOFileFilter, > org.apache.commons.io.filefilter.IOFileFilter)}}. These threads also *move*, > *delete* and *rename* files from this directory. While this seems to be > working on a windows machine, it is not the case on a Linux machine. In 2.8 > apparently there was no issue with this. > *In short*: While a directory is being listed with {{FileUtils.listFiles()}}, > a file is being deleted. When trying to read its file-attributes it stops > because of a {{NoSuchFileException}}. > h3. Other commons-io functionalities > We tried using {{PathUtils.walk(directory filters, 5, > false).map(Path::toFile).collect(Collectors.toList())}}. This method was > introduced with the same change. > This produced a similar stacktrace: > {code:trace} > Caused by: java.io.UncheckedIOException: java.nio.file.NoSuchFileException: /* > at > java.base/java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:87) > at > java.base/java.nio.file.FileTreeIterator.hasNext(FileTreeIterator.java:103) > at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132) > at > java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) > at > java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) > at > java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) > at > java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) > at > java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) > at > ch.elca.ole.filenet.FilenetAccessorImpl.listFilesDocumentsStorageDir(FilenetAccessorImpl.java:439) > at > ch.elca.ole.filenet.FilenetAccessorImpl.buildStub(FilenetAccessorImpl.java:401) > at > ch.elca.ole.filenet.FilenetAccessorImpl.createStub(FilenetAccessorImpl.java:310) > ... 47 more > Caused by: java.nio.file.NoSuchFileException: /* > at > java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) > at > java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) > at > java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) > at > java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149) > at > java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.base/java.nio.file.Files.readAttributes(Files.java:1764) > at > java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219) > at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276) > at java.base/java.nio.file.FileTreeWalker.next(FileTreeWalker.java:373) > at > java.base/java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:83) > ... 59 more{code} > h3. git commit > https://github.com/apache/commons-io/commit/4a514d3306b55b3667d1449ebd4cbe5f19dd7af0 > h3. Other Questions > * What was the reason to wrap all {{IOException}} into > {{UncheckedIOException}} in those methods? -- This message was sent by Atlassian Jira (v8.3.4#803005)