----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3988/#review6220 -----------------------------------------------------------
Ship it! sorry I didn't look at this earlier. Looks fine to me. Please see if the code needs to be rebased. - Prasad On 2012-02-21 21:51:32, Brock Noland wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/3988/ > ----------------------------------------------------------- > > (Updated 2012-02-21 21:51:32) > > > Review request for Flume. > > > Summary > ------- > > 1) All HDFS actions are now done in async mode > 2) If an HDFS append timesout, the file is closed and reopened. > 3) Batching is now handled by BucketWriter which was always aware of the > batch size. > > > This addresses bug FLUME-985. > https://issues.apache.org/jira/browse/FLUME-985 > > > Diffs > ----- > > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java > 19b2559 > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/HDFSBadSeqWriter.java > 8a6740f > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java > 7d8ee8a > flume-ng-sinks/flume-hdfs-sink/pom.xml f27851e > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java > 45769f6 > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java > 3da90a5 > > Diff: https://reviews.apache.org/r/3988/diff > > > Testing > ------- > > 1) Unit tests were added for close/reopen scenario. > 2) All unit tests pass > 3) I manually verified this patch improved FlumeNG's behavior when the > datanode it's writing to is restarted. In the past FlumeNG had to be > restarted, now Flume moves on and starts writing to a new file. > > > Thanks, > > Brock > >
