ExecSource always spits out one flume event per line, no matter what command is executed.
On Jan 22, 2014, at 9:09 AM, Chhaya Vishwakarma wrote: > Hi, > > Is my interceptor configuration correct? Or there is some problem with that ? > Can someone guide me in this > Instead of tail I am using CAT now but same error its giving > > Regards, > Chhaya Vishwakarma > > > > -----Original Message----- > From: Chhaya Vishwakarma [mailto:[email protected]] > Sent: Wednesday, January 22, 2014 10:20 AM > To: [email protected] > Subject: RE: flume agent not starting > > Hi , > > What i should use then for interceptor to work shall I use CAT ?? > > -----Original Message----- > From: Wolfgang Hoschek [mailto:[email protected]] > Sent: Tuesday, January 21, 2014 6:30 PM > To: [email protected] > Subject: Re: flume agent not starting > > 'tail' with exec source spits out one flume event per line into the > interceptor. But readMultiLine expects multiple lines per event, not one line > per event. In other words, by the time the data arrives in the interceptor > it's already too late for readMultiLine to make sense. > > Wolfgang. > > > On Jan 17, 2014, at 2:42 PM, Chhaya Vishwakarma wrote: > >> Hi, >> >> Agent is starting I am using morphline interceptor to parse events >> >> Below is my morphline configuration >> >> morphlines : [ >> { >> id : morphline1 >> >> commands : [ >> { >> readMultiLine { >> regex : "Started session*" >> what : next >> charset : UTF-8 >> } >> } >> ] >> } >> ] >> >> This is my flume configuration file >> >> host1.sources = src1 >> host1.sinks = sink1 >> host1.channels = ch1 >> >> host1.sources.src1.type = exec >> host1.sources.src1.command = tail -F >> /home/user/apache-flume-1.4.0-bin/try.log >> host1.sources.src1.channels = ch1 >> host1.sources.src1.restart = true >> #host1.sources.src1.batchsize=10 >> >> host1.sources.src1.interceptors = morphlineinterceptor >> host1.sources.src1.interceptors.morphlineinterceptor.type = >> org.apache.flume.sink.solr.morphline.MorphlineInterceptor$Builder >> host1.sources.src1.interceptors.morphlineinterceptor.morphlineFile = >> /home/user/apache-flume-1.4.0-bin/morphline.conf >> >> host1.sinks.sink1.type = file_roll >> host1.sinks.sink1.channel = ch1 >> host1.sinks.sink1.sink.directory = /home/user/apache-flume-1.4.0-bin >> host1.sinks.sink1.sink.serializer= TEXT >> host1.sinks.sink1.sink.rollInterval=0 >> >> host1.channels.ch1.capacity=1000 >> host1.channels.ch1.transactionCapacity=100 >> >> host1.channels.ch1.type=memory >> >> input file content >> >> Started session >> a >> b >> c >> Started session >> c >> d >> e >> Started session >> >> But in /home/user/apache-flume-1.4.0-bin I can see a file named >> 1389960784130-1 but it's not showing any content. I can see only new >> line characters in output file >> >> Regards, >> Chhaya Vishwakarma >> >> >> The contents of this e-mail and any attachment(s) may contain confidential >> or privileged information for the intended recipient(s). Unintended >> recipients are prohibited from taking action on the basis of information in >> this e-mail and using or disseminating the information, and must notify the >> sender and delete it from their system. L&T Infotech will not accept >> responsibility or liability for the accuracy or completeness of, or the >> presence of any virus or disabling code in this e-mail" >
