[
https://issues.apache.org/jira/browse/FLUME-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13693825#comment-13693825
]
Alexander Alten-Lorenz commented on FLUME-2113:
-----------------------------------------------
I dropped this into question. Please note, Flume is NOT designed to transfer
large files. Flume is a log collection aggregator, and can handle multiple
sources at once. To transfer large files please use HDFS related tools like
hadoop fs -put, or use the WebHDFS API
(http://hadoop.apache.org/docs/r1.0.4/webhdfs.html).
> Flume: No errors thrown but flume fails to transfer the file completely
> -----------------------------------------------------------------------
>
> Key: FLUME-2113
> URL: https://issues.apache.org/jira/browse/FLUME-2113
> Project: Flume
> Issue Type: Question
> Reporter: kasa v varun
> Priority: Minor
> Labels: hdfs, sink
>
> Flow: using a basic flow, spoolDir -> fileChannel -> HDFS
> No extra parameters set in the .conf file
> File size that i'm trying to transfer: 1.4GB
> Situation: the agents starts fine, the file transfer starts fine, the file in
> source gets renamed to .COMPLETED, the complete file is not getting
> transfered to HDFS, no errors/exceptions are being thrown. I ran the same
> adhoc several times i found that out of the 1.4 gigs only ~169Mb is getting
> transfered. Seems weird !
> Any suggestions? Any solutions? any hypotheses?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
