[
https://issues.apache.org/jira/browse/FLUME-985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237125#comment-13237125
]
[email protected] commented on FLUME-985:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3988/#review6311
-----------------------------------------------------------
Ship it!
+1
- Arvind
On 2012-03-23 20:55:21, Brock Noland wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3988/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-23 20:55:21)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. 1) All HDFS actions are now done in async mode
bq. 2) If an HDFS append timesout, the file is closed and reopened.
bq. 3) Batching is now handled by BucketWriter which was always aware of the
batch size.
bq.
bq.
bq. This addresses bug FLUME-985.
bq. https://issues.apache.org/jira/browse/FLUME-985
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-sinks/flume-hdfs-sink/pom.xml bef2ca7
bq.
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java
45769f6
bq.
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java
1fdaddd
bq.
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java
19b2559
bq.
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/HDFSBadSeqWriter.java
8a6740f
bq.
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/HDFSBadWriterFactory.java
b067c00
bq.
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java
8fa72a1
bq.
bq. Diff: https://reviews.apache.org/r/3988/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. 1) Unit tests were added for close/reopen scenario.
bq. 2) All unit tests pass
bq. 3) I manually verified this patch improved FlumeNG's behavior when the
datanode it's writing to is restarted. In the past FlumeNG had to be restarted,
now Flume moves on and starts writing to a new file.
bq.
bq.
bq. Thanks,
bq.
bq. Brock
bq.
bq.
> All HDFS Operations in HDFSEventSink should have a timeout
> ----------------------------------------------------------
>
> Key: FLUME-985
> URL: https://issues.apache.org/jira/browse/FLUME-985
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.0.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Fix For: v1.2.0
>
> Attachments: FLUME-985-0.patch, FLUME-985-1.patch
>
>
> In FLUME-871 appends were made asynchronous so we could time them out. All
> HDFS Operations should be done this same way.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
