[
https://issues.apache.org/jira/browse/FLUME-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248007#comment-13248007
]
[email protected] commented on FLUME-1104:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4662/#review6731
-----------------------------------------------------------
Ship it!
+1
- Arvind
On 2012-04-06 03:06:43, Prasad Mujumdar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4662/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-06 03:06:43)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The sink process() keep tracks of the buckets opened during the
transaction. At the end of transaction, we need to flush all the buckets that
has pending data. This is required in order to ensure that the data removed
from channel should be safely in HDFS during commit.
bq. Currently the files are tracked only when they are created and also
getting closed during the cleanup instead of flush.
bq.
bq. The fix is to track buckets every time they are written to in the current
transaction. Also buckets with pending data should be flushed instead of close.
bq.
bq.
bq. This addresses bug FLUME-1104.
bq. https://issues.apache.org/jira/browse/FLUME-1104
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java
7a94f97
bq.
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java
114682a
bq.
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java
6ff3737
bq.
bq. Diff: https://reviews.apache.org/r/4662/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Prasad
bq.
bq.
> HDFS rolls the first file incorrectly
> -------------------------------------
>
> Key: FLUME-1104
> URL: https://issues.apache.org/jira/browse/FLUME-1104
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Prasad Mujumdar
> Assignee: Prasad Mujumdar
> Fix For: v1.2.0
>
> Attachments: FLUME-1104-2.patch
>
>
> The sink process() keep tracks of the buckets opened during the transaction.
> At the end of transaction, we need to flush all the buckets that has pending
> data. This is required in order to ensure that the data removed from channel
> should be safely in HDFS during commit.
> Currently the files are tracked only when they are created and also getting
> closed during the cleanup instead of flush.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
