You have a memory channel with capacity of 10M events. Your VM might be going OOM if there is events are buffered and not enough heap is allocated.
On Fri, Nov 7, 2014 at 3:03 PM, mahendran m <[email protected]> wrote: > Hi All, > > I am new to Apache flume . I have configured using thrift source to send log > to HDFS my Config as below > > # list sources, sinks and channels in the agent > a1.sources = r1 > a1.sinks = k1 > a1.channels = c1 > > # avro sink properties > a1.sources.r1.type = thrift > a1.sources.r1.bind = localhost > a1.sources.r1.port = 44444 > a1.sources.r1.interceptors = i1 > a1.sources.r1.interceptors.i1.type = > DecoderInterceptor.CustomInterceptor$Builder > a1.sources.r1.interceptors.i1.referalUrl=referalUrl > a1.sources.r1.interceptors.i1.referalHost=referalHost > > #HDFS sink > a1.sinks.k1.type = hdfs > a1.sinks.k1.hdfs.fileType = DataStream > a1.sinks.k1.hdfs.fileSuffix= .txt > a1.sinks.k1.hdfs.rollSize = 1048576 > a1.sinks.k1.hdfs.rollCount = 0 > a1.sinks.k1.hdfs.rollInterval = 0 > a1.sinks.k1.hdfs.batchSize = 1000 > a1.sinks.k1.hdfs.minBlockReplicas = 1 > a1.sinks.k1.hdfs.callTimeout = 60000 > a1.sinks.k1.hdfs.path = hdfs://localhost:9000/flumeChannel100/Thrift > > # Use a channel which buffers events in memory > a1.channels.c1.type = memory > a1.channels.c1.capacity = 10000000 > a1.channels.c1.transactionCapacity = 1000 > a1.channels.c1.byteCapacityBufferPercentage = 10 > a1.channels.c1.byteCapacity = 5368709120 > > # define the flow > a1.sources.r1.channels = c1 > a1.sinks.k1.channel = c1 > > when i stated HDFS, flume service and generated the logs from c# application > . my logs are moved to HDFS everything OK still now. but when stopped HDFS > service . flume agent itself get stopped. Is this default behavior ? . or > any wen wrong . > > Regards, > Mahendran > > -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
