> On June 14, 2016, 2:21 a.m., Mike Percy wrote: > > flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/TaildirMatcher.java, > > line 103 > > <https://reviews.apache.org/r/48161/diff/1-2/?file=1404557#file1404557line103> > > > > s/consisting/consists/
you requested consisting in the first batch ;) > On June 14, 2016, 2:21 a.m., Mike Percy wrote: > > flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/TaildirMatcher.java, > > line 182 > > <https://reviews.apache.org/r/48161/diff/1-2/?file=1404557#file1404557line182> > > > > Would you mind renaming actualMTime to currentParentDirMTime? For > > consistency with lastSeenParentDirMTime renamed > On June 14, 2016, 2:21 a.m., Mike Percy wrote: > > flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/TaildirMatcher.java, > > line 245 > > <https://reviews.apache.org/r/48161/diff/1-2/?file=1404557#file1404557line245> > > > > why remove parentDir? Just wondering. I want to store all properties from configuration required for constructing these objects as it helps debugging a lot so filePattern had to remain a member variable. ParentDir and regex was mainly needed for the original logic which used java.io and has been removed with the previous iteration. As a positive outcome toString now truly represents the information required for reconstructing the object (these are the constructor arguments as well). Also parentdir + regex == filePattern so it was a simplification. > On June 14, 2016, 2:21 a.m., Mike Percy wrote: > > flume-ng-sources/flume-taildir-source/src/test/java/org/apache/flume/source/taildir/TestTaildirMatcher.java, > > line 51 > > <https://reviews.apache.org/r/48161/diff/1-2/?file=1404560#file1404560line51> > > > > I didn't mean to force you to add full Javadoc. You can add it if you > > want, but all I wanted was at least a quick comment to summarize what this > > helper function did. revert - Attila ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48161/#review137440 ----------------------------------------------------------- On June 13, 2016, 2:14 p.m., Attila Simon wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48161/ > ----------------------------------------------------------- > > (Updated June 13, 2016, 2:14 p.m.) > > > Review request for Flume. > > > Bugs: FLUME-2918 > https://issues.apache.org/jira/browse/FLUME-2918 > > > Repository: flume-git > > > Description > ------- > > The way TailDir source checks which files should be tracked was improved. > Existing implementation caused unneccessary high CPU usage for huge (+50K > files) directories. This fix allows users to eliminate continous listing of > parent directory (on each Source.process invocation) and introduce a more > performant method for listing&matching files. > > used java.nio.file.DirectoryStream to filter files > made pattern match calculation optionally cached > added junit tests > added javadoc > added license > > > Diffs > ----- > > > flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/ReliableTaildirEventReader.java > 5b6d465 > > flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/TaildirMatcher.java > PRE-CREATION > > flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/TaildirSource.java > 8816327 > > flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/TaildirSourceConfigurationConstants.java > 6165276 > > flume-ng-sources/flume-taildir-source/src/test/java/org/apache/flume/source/taildir/TestTaildirMatcher.java > PRE-CREATION > > flume-ng-sources/flume-taildir-source/src/test/java/org/apache/flume/source/taildir/TestTaildirSource.java > f9e614c > > Diff: https://reviews.apache.org/r/48161/diff/ > > > Testing > ------- > > mvn clean install -DskipTests -> built > junit tests for flume-taildir-source module -> passed > > > Thanks, > > Attila Simon > >
