Re: Memory Channel Error on Load Testing

2013-10-18 Thread Roshan Naik
the error indicates that source is pumping in data faster than the sink is draining the memory channel. causing the channel to fill up. it does not appear to be a MemCh issue. On Fri, Oct 18, 2013 at 8:47 AM, Deepak Subhramanian < deepak.subhraman...@gmail.com> wrote: > Hi > I am getting error w

Memory Channel Error on Load Testing

2013-10-18 Thread Deepak Subhramanian
Hi I am getting error while load testing a flume agent with HTTSrc and Avro Sink. Is there an optimum configuration for memory channel for get flume working on high load. Thanks in advance for any inputs. tier1.sources = httpsrc1 tier1.channels = c1 c2 tier1.sinks= avrosink1 tier1.source

Re: Roll based on date

2013-10-18 Thread David Sinclair
The SyslogTcpSource will put a header on the flume event named 'timestamp'. This timestamp will be from the syslog entry. You could then set the filePrefix in the sink to grab this out. For example tier1.sinks.hdfsSink.hdfs.filePrefix = FlumeData.%{timestamp} dave On Thu, Oct 17, 2013 at 10:23