----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3988/#review6311 -----------------------------------------------------------
Ship it! +1 - Arvind On 2012-03-23 20:55:21, Brock Noland wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/3988/ > ----------------------------------------------------------- > > (Updated 2012-03-23 20:55:21) > > > Review request for Flume. > > > Summary > ------- > > 1) All HDFS actions are now done in async mode > 2) If an HDFS append timesout, the file is closed and reopened. > 3) Batching is now handled by BucketWriter which was always aware of the > batch size. > > > This addresses bug FLUME-985. > https://issues.apache.org/jira/browse/FLUME-985 > > > Diffs > ----- > > flume-ng-sinks/flume-hdfs-sink/pom.xml bef2ca7 > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java > 45769f6 > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java > 1fdaddd > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java > 19b2559 > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/HDFSBadSeqWriter.java > 8a6740f > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/HDFSBadWriterFactory.java > b067c00 > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java > 8fa72a1 > > Diff: https://reviews.apache.org/r/3988/diff > > > Testing > ------- > > 1) Unit tests were added for close/reopen scenario. > 2) All unit tests pass > 3) I manually verified this patch improved FlumeNG's behavior when the > datanode it's writing to is restarted. In the past FlumeNG had to be > restarted, now Flume moves on and starts writing to a new file. > > > Thanks, > > Brock > >
