I found the issue. I removed the quotes around the GROK. Keith
From: Keith Wiggins Sent: Wednesday, November 11, 2015 11:02 AM To: [email protected] Subject: Taildir multiLineRegex patch FLUME-2800 help Hi, 1. Hi, 2. 3. I am testing the FLUME-2800 multiLine for taildir and it doesn't seem to be working for me. Does anyone have any experience with this patch? Does the config I am using look ok? 4. agent.sources.CDRS.type = TAILDIR agent.sources.CDRS.channels = memoryChannel agent.sources.CDRS.filegroups = f1 agent.sources.CDRS.filegroups.f1 = /opt/OneEMS_LOGS/G6MM.log agent.sources.CDRS.batchSize = 100 agent.sources.CDRS.multiLineRegex = "^%{TIMESTAMP_ISO8601}" agent.sources.CDRS.maxNumberLines = 4096 log line entry; 2015-08-05 13:46:21,030 INFO [main] [com.centeredlogic.ecx.tomcat.ServerLifecycleListener] - Got lifecycle event: before_init It seems to detect that I am asking for the ISO8601 but it doesn't seem to read the file as nothing is coming out to my sink. If I remove the multiLine it does write to my sink (file_roll). I am using the GROK expression as that is what I am using in Logstash multiline which works with the file I am tailing. 2015-11-11 10:50:57,645 (conf-file-poller-0) [DEBUG - org.apache.flume.source.taildir.grok.dictionary.GrokDictionary.digestExpression(GrokDictionary.java:136)] Digested ["^%{TIMESTAMP_ISO8601}"] into ["^(?>\d\d){1,2}-(?:0?[1-9]|1[0-2])-(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])[T ](?:2[0123]|[01]?[0-9]):?(?:[0-5][0-9])(?::?(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))?(?:Z|[+-](?:2[0123]|[01]?[0-9])(?::?(?:[0-5][0-9])))?"] before compilation 2015-11-11 10:50:57,756 (conf-file-poller-0) [INFO - org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:42)] Creating instance of sink: k1, type: file_roll Keith Wiggins Software Designer 500 Palladium Drive, Suite 2100 Ottawa ON Canada K2V 1C2 www.genband.com<http://www.genband.com/> office:+1.343.883.2656 [email protected]<mailto:[email protected]> [Description: http://www.genband.com/extras/email/email_signature_line_below.jpg]<http://www.genband.com/> Connect with us:[Description: http://www.genband.com/extras/email/facebookBTN.gif]<http://www.facebook.com/genband>[Description: http://www.genband.com/extras/email/twitterBTN.gif]<https://twitter.com/#!/genband>[Description: http://www.genband.com/extras/email/linkedBTN.gif]<http://www.linkedin.com/company/genband>[Description: http://www.genband.com/extras/email/youtubeBTN.gif]<http://www.youtube.com/GenbandClips>[Description: http://www.genband.com/extras/email/blogBTN.gif]<http://www.genband.com/driving-innovation>
