Kettler Karl created FLUME-2824:
-----------------------------------
Summary: How does Flume write Json
Key: FLUME-2824
URL: https://issues.apache.org/jira/browse/FLUME-2824
Project: Flume
Issue Type: Blog - New Blog Request
Components: Sinks+Sources
Affects Versions: v1.5.2
Environment: HDP2.3
Reporter: Kettler Karl
Fix For: v1.5.2
How does Flume write in Jason?
We get always Twitter data in Avro format.
TwitterAgent.sources = Twitter
TwitterAgent.channels = MemChannel
TwitterAgent.sinks = HDFS
TwitterAgent.sources.Twitter.type =
org.apache.flume.source.twitter.TwitterSource
TwitterAgent.sources.Twitter.channels = MemChannel
TwitterAgent.sources.Twitter.consumerKey = xxx
TwitterAgent.sources.Twitter.consumerSecret = xxx
TwitterAgent.sources.Twitter.accessToken = xxx
TwitterAgent.sources.Twitter.accessTokenSecret = xxx
TwitterAgent.sources.Twitter.maxBatchSize = 10
TwitterAgent.sources.Twitter.maxBatchDurationMillis = 200
TwitterAgent.sources.Twitter.keywords = United Nations
TwitterAgent.sources.Twitter.deserializer.schemaType = LITERAL
TwitterAgent.sinks.HDFS.channel = MemChannel
TwitterAgent.sinks.HDFS.type = hdfs
TwitterAgent.sinks.HDFS.hdfs.path = /demo/tweets/stream/%y-%m-%d/%H%M%S
TwitterAgent.sinks.HDFS.hdfs.filePrefix = events
TwitterAgent.sinks.HDFS.hdfs.round = true
TwitterAgent.sinks.HDFS.hdfs.roundValue = 5
TwitterAgent.sinks.HDFS.hdfs.roundUnit = minute
TwitterAgent.sinks.HDFS.hdfs.useLocalTimeStamp = true
TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream
TwitterAgent.channels.MemChannel.type = memory
TwitterAgent.channels.MemChannel.capacity = 1000
TwitterAgent.channels.MemChannel.transactionCapacity = 100
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
