Kettler Karl created FLUME-2824: ----------------------------------- Summary: How does Flume write Json Key: FLUME-2824 URL: https://issues.apache.org/jira/browse/FLUME-2824 Project: Flume Issue Type: Blog - New Blog Request Components: Sinks+Sources Affects Versions: v1.5.2 Environment: HDP2.3 Reporter: Kettler Karl Fix For: v1.5.2
How does Flume write in Jason? We get always Twitter data in Avro format. TwitterAgent.sources = Twitter TwitterAgent.channels = MemChannel TwitterAgent.sinks = HDFS TwitterAgent.sources.Twitter.type = org.apache.flume.source.twitter.TwitterSource TwitterAgent.sources.Twitter.channels = MemChannel TwitterAgent.sources.Twitter.consumerKey = xxx TwitterAgent.sources.Twitter.consumerSecret = xxx TwitterAgent.sources.Twitter.accessToken = xxx TwitterAgent.sources.Twitter.accessTokenSecret = xxx TwitterAgent.sources.Twitter.maxBatchSize = 10 TwitterAgent.sources.Twitter.maxBatchDurationMillis = 200 TwitterAgent.sources.Twitter.keywords = United Nations TwitterAgent.sources.Twitter.deserializer.schemaType = LITERAL TwitterAgent.sinks.HDFS.channel = MemChannel TwitterAgent.sinks.HDFS.type = hdfs TwitterAgent.sinks.HDFS.hdfs.path = /demo/tweets/stream/%y-%m-%d/%H%M%S TwitterAgent.sinks.HDFS.hdfs.filePrefix = events TwitterAgent.sinks.HDFS.hdfs.round = true TwitterAgent.sinks.HDFS.hdfs.roundValue = 5 TwitterAgent.sinks.HDFS.hdfs.roundUnit = minute TwitterAgent.sinks.HDFS.hdfs.useLocalTimeStamp = true TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream TwitterAgent.channels.MemChannel.type = memory TwitterAgent.channels.MemChannel.capacity = 1000 TwitterAgent.channels.MemChannel.transactionCapacity = 100 -- This message was sent by Atlassian JIRA (v6.3.4#6332)