Thanks Ahmed. At the moment, I’m only concerned with RFC 5424 formatted syslog <https://tools.ietf.org/html/rfc5424>, especially the structured data ( the data in the []).
Such as: <14>1 2014–06–20T09:14:07+00:00 loggregator d0602076-b14a–4c55–852a–981e7afeed38 DEA MSG–01 [exampleSDID@32473 iut=“3” eventSource=“Application” eventID=“1011”][exampleSDID@32480 iut=4 eventSource=Other Application eventID=2022] Removing instance On May 20, 2018 at 19:03:29, Ahmed Shah (ahmeds...@cmail.carleton.ca) wrote: Hello, If needed this is what our syslog config files look like and our GROK statement (used with Metron 0.4.2) Server side syslog config files (messages sent to syslog are passed on to Kafka): https://github.com/LTW-GCR-CSOC/csoc-installation-scripts/blob/master/SampleLogFiles/configForServer-Encypted/rsyslog.conf https://github.com/LTW-GCR-CSOC/csoc-installation-scripts/blob/master/SampleLogFiles/configForServer-Encypted/00-GCRserverReciDionaea.conf Client/honeypot side config file: https://github.com/LTW-GCR-CSOC/csoc-installation-scripts/blob/master/SampleLogFiles/configForHP-Encrypted/00-GCRdionaeaHP.conf GROK Statement: https://github.com/LTW-GCR-CSOC/csoc-installation-scripts/blob/master/SampleLogFiles/README.md -Ahmed _______________________________________________________________ Ahmed Shah (PMP, M. Eng.) Cybersecurity Analyst & Developer GCR - Cybersecurity Operations Center Carleton University - cugcr.com<https://cugcr.com/tiki/lce/index.php> ________________________________ From: Casey Stella <ceste...@gmail.com> Sent: May 18, 2018 10:59 AM To: dev@metron.apache.org Subject: Re: Request for Comment on new Syslog 5424 Parsing library Cool! I'd welcome a syslog parser! On Fri, May 18, 2018 at 10:02 AM Otto Fowler <ottobackwa...@gmail.com> wrote: > There have been some issues and talk about they way we parse syslog, and > the deficiencies of our grok and regex based approaches, mainly not > supporting structured data as I recall. > I played around with it some and decided to try to write an Antlr grammar > based on the RFC 5424 spec BNF to parse valid syslogs. > > I have chosen to create this in my own github org, and will be distributing > through bintray/mvn central down the line. I *may* end up doing PR’s to > Metron and Nifi around this but that is not definite. > > If anyone is interested, I would really appreciate any review or feedback. > Also, if anyone has any ‘clean’ 5424 logs that they can safely contribute > to expand my test set, that would be much appreciated. > > https://github.com/palindromicity/simple-syslog-5424 > > > thanks > ottO >