----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49025/#review138910 -----------------------------------------------------------
Since Flume 1.7+ will now only support Java 7+, why don't we consider using Java 7's new DirectoryStream API, rather than the much more expensive listFiles API? This probably will result in a huge performance boost plus far simpler code structure. It will require some rewrite of the code, but since we have tests and we expect current behavior, I would suggest doing that. - Hari Shreedharan On June 21, 2016, 3:32 p.m., Balázs Donát Bessenyei wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49025/ > ----------------------------------------------------------- > > (Updated June 21, 2016, 3:32 p.m.) > > > Review request for Flume, Denes Arvay, Mike Percy, and Attila Simon. > > > Repository: flume-git > > > Description > ------- > > SpoolrDir currently monitors a directory and can not handle sub-directories. > This JIRA is to make SpoolDir able to walk down a source directory and > monitor new files. > > > Diffs > ----- > > > flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java > d54f415 > > flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java > 3fe947d > > flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java > 5053697 > > flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java > fe530ff > flume-ng-doc/sphinx/FlumeUserGuide.rst 74d2887 > > Diff: https://reviews.apache.org/r/49025/diff/ > > > Testing > ------- > > Ran tests before the patch: > # mvn clean install -DskipTests -Drat.skip=true; mvn -pl flume-ng-core > -Drat.skip=true test > Tests run: 378, Failures: 0, Errors: 0, Skipped: 2 > > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 07:55 min > [INFO] Finished at: 2016-06-21T16:13:46+02:00 > [INFO] Final Memory: 35M/510M > [INFO] > ------------------------------------------------------------------------ > > After patch: > # mvn clean install -DskipTests -Drat.skip=true; mvn -pl flume-ng-core > -Drat.skip=true test > Tests run: 380, Failures: 0, Errors: 0, Skipped: 2 > > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 06:18 min > [INFO] Finished at: 2016-06-21T17:04:17+02:00 > [INFO] Final Memory: 35M/511M > [INFO] > ------------------------------------------------------------------------ > > Patch also includes docs > > > Thanks, > > Balázs Donát Bessenyei > >
