Firstly, to print diagnostic information such as the content of records as they pass through the morphline commands, consider enabling TRACE log level, for example by adding the following line to your log4j.properties file:
log4j.logger.org.kitesdk.morphline=TRACE Secondly, is it expected that the *entire* line matches the regex labelled "word"? If not the grok command will fail, of course. http://kitesdk.org/docs/current/kite-morphlines/morphlinesReferenceGuide.html#grok Wolfgang. On Feb 26, 2014, at 11:41 AM, Sandesh Hurli wrote: > Hi, > > I am trying process the flume event body using the Morphline interceptor, > grok pattern, and write it to a file using file_roll sink (started with a > simple setup for testing purpose, later will be using HDFS sink). > The source and the sink start successfully but the output file remains empty. > Below are the details : > > Source is tailing a sample file containing the word "sample". I append more > words to the file, still dont see any output in the sink destination file. > > flume.conf file: > --------------------------------------------------------------------------- > agent.sources = seqGenSrc > agent.channels = memoryChannel > agent.sinks = loggerSink > > agent.sources.seqGenSrc.type = exec > agent.sources.seqGenSrc.command = tail -F /home/sandesh/sample.txt > agent.sources.seqGenSrc.interceptors = morphlineinterceptor > agent.sources.seqGenSrc.interceptors.morphlineinterceptor.type = > org.apache.flume.sink.solr.morphline.MorphlineInterceptor$Builder > agent.sources.seqGenSrc.interceptors.morphlineinterceptor.morphlineFile = > /usr/local/lib/flume-ng/conf/morphline.conf > agent.sources.seqGenSrc.interceptors.morphlineinterceptor.morphlineId = > morphline1 > agent.sources.seqGenSrc.channels = memoryChannel > > agent.sinks.loggerSink.type = file_roll > agent.sinks.loggerSink.sink.directory = /home/sandesh/ > agent.sinks.loggerSink.channel = memoryChannel > > agent.channels.memoryChannel.type = memory > agent.channels.memoryChannel.capacity = 1000 > > -------------------------------------------------------------------------------- > > morphline.conf file: > -------------------------------------------------------------------------------- > morphlines : [ > { > id : morphline1 > importCommands : ["com.cloudera.**"] > commands : [ > #Reading the line from Flume event: > { > readLine { > charset : UTF-8 > } > } > > { > #Regex - pattern matching and transformation will be taken care > by below config > grok { > dictionaryFiles : > [/home/sandesh/cdk/cdk-morphlines/cdk-morphlines-core/src/test/resources/grok-dictionaries] > expressions : { > message : """%{WORD:word}""" > } > } > } > ] > } > ] > ------------------------------------------------------------------------------ > > Am I missing something obvious? Please help me if yes.. > > Regards, > Sandesh > > > >
