Hi all,
I want use spark-streaming with flume ,now i am in truble, I don't
know how to configure the flume ,I use I configure flume like this :
a1.sources = r1
a1.channels = c1 c2
a1.sources.r1.type = avro
a1.sources.r1.channels = c1 c2
a1.sources.r1.bind = 0.0.0.0
a1.sources.r1.port = 44444
a1.sources.r1.selector.type=replicating
a1.channels.c1.type = memory
a1.channels.c1.capacity = 10000
a1.channels.c1.transactionCapacity = 10000
a1.channels.c1.byteCapacityBufferPercentage = 20
a1.channels.c1.byteCapacity = 800000
a1.channels.c2.type = memory
a1.channels.c2.capacity = 10000
a1.channels.c2.transactionCapacity = 10000
a1.channels.c2.byteCapacityBufferPercentage = 20
a1.channels.c2.byteCapacity = 800000
a1.sinks = k1
a1.sinks.k1.type = hdfs
a1.sinks.k1.channel = c1
a1.sinks.k1.hdfs.path = /user/hxf/flume
a1.sinks.k1.hdfs.filePrefix = events-
a1.sinks.k1.hdfs.rollCount=0
a1.sinks.k1.hdfs.rollInterval=10
a1.sinks.k1.hdfs.rollSize=0
a1.channels = c1
but when I send data to the 44444 port I get an error like this :
org.apache.avro.AvroRuntimeException: Excessively large list allocation request
detected: 154218761 items! Connection closed.
dose anybody can help me? thanks!
