-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4662/
-----------------------------------------------------------
Review request for Flume.
Summary
-------
The sink process() keep tracks of the buckets opened during the transaction. At
the end of transaction, we need to flush all the buckets that has pending data.
This is required in order to ensure that the data removed from channel should
be safely in HDFS during commit.
Currently the files are tracked only when they are created and also getting
closed during the cleanup instead of flush.
The fix is to track buckets every time they are written to in the current
transaction. Also buckets with pending data should be flushed instead of close.
This addresses bug FLUME-1104.
https://issues.apache.org/jira/browse/FLUME-1104
Diffs
-----
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java
114682a
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java
6ff3737
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java
7a94f97
Diff: https://reviews.apache.org/r/4662/diff
Testing
-------
Thanks,
Prasad
