[
https://issues.apache.org/jira/browse/FLUME-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265075#comment-13265075
]
Mike Percy commented on FLUME-1117:
-----------------------------------
Hi Mubarak,
What build of FlumeNG are you using? Also, is any data showing up in HDFS? What
format is it in? Are you getting any error messages in your flume.log?
Here is an example config file I am using. This should work for you, in FlumeNG
on trunk. It's using a Syslog source so you may want to change the source.
--
agent.sources = svc_0_src
agent.channels = svc_0_chan
agent.sinks = svc_0_sink
agent.channels.svc_0_chan.type = memory
agent.channels.svc_0_chan.capacity = 100000
agent.channels.svc_0_chan.transactionCapacity = 1000
agent.sources.svc_0_src.type = org.apache.flume.source.SyslogTcpSource
agent.sources.svc_0_src.port = 10001
agent.sources.svc_0_src.channels = svc_0_chan
agent.sinks.svc_0_sink.type = hdfs
agent.sinks.svc_0_sink.hdfs.path =
hdfs://xxxxxx.cloudera.com/flume/20120427/flow0
agent.sinks.svc_0_sink.hdfs.fileType = DataStream
agent.sinks.svc_0_sink.hdfs.rollInterval = 300
agent.sinks.svc_0_sink.hdfs.rollSize = 0
agent.sinks.svc_0_sink.hdfs.rollCount = 0
agent.sinks.svc_0_sink.hdfs.batchSize = 1000
agent.sinks.svc_0_sink.hdfs.txnEventMax = 1000
agent.sinks.svc_0_sink.serializer = avro_event
agent.sinks.svc_0_sink.serializer.compressionCodec = snappy
agent.sinks.svc_0_sink.serializer.syncIntervalBytes = 2048000
agent.sinks.svc_0_sink.channel = svc_0_chan
--
Please let me know whether this works for you.
Thanks,
Mike
> Support output to files in Avro container format
> ------------------------------------------------
>
> Key: FLUME-1117
> URL: https://issues.apache.org/jira/browse/FLUME-1117
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.2.0
> Reporter: Mike Percy
> Assignee: Mike Percy
> Fix For: v1.2.0
>
> Attachments: FLUME-1117-4.patch
>
>
> Support output of events to a file in Avro container format.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
