On Sun, 4 Nov 2012, John P. Rouillard wrote: > On Sat, 20 Oct 2012, Risto Vaarandi wrote: >> >> hi David, >> I have completed some work on the alpha version of the next release, > > Does this alpha release include any support for incremental parsing? > > Reference: > > http://www.mail-archive.com/[email protected]/msg01120.html > > This would speed up parsing as I cpould write short rules that match > key elements (host, service/daemon, pid...) and extracts that > data. Then it would jump to a ruleset that parses the body of the > message for additional fields. Currently I have to parse the same data > over and over again and can't optimize the regexp by removing the > initial fields from the regexp that I know are already there. E.G. Assume: > > 2012-1103T23:15:10 example.com kernel: ACCEPT 10.201.20.32 from > 158.121.100.3 port 30 > > I would like to have one command to parse out the: date/time, host, > "kernel" and "ACCEPT" fields, then a set of rules/regexps that start > with the word "from" (which can be quickly scanned for in the string) > and a set of rule for the "from" ip address. This means I parse the > host field only once. Not once for every "from" rule that I have. > > However I need the date/time, host and other data available in places > where $N variables are now. > > There is another event correlation engine that does this sort of > incrmental parsing. It allows you to parse the event and pull data > from it before sending it to another ruleset to do finer grained > parsing and data extraction. Wish I could remeber what the engine was > called.
My understanding is that what this provides is the ability to run an arbitrary perl parser against the line (which can just parse a few things if you want), and then 'fix' the results of that parse operation as variables that other rules can then use for their decisionmaking without needing to parse the line. So for what you are wanting, you could do two level parsing, something like: first parser, extract timestamp, host, programname, first word of message decide that this is an iptables ACCEPT log line and jump to a specific ruleset second parser, extract the data in the line to named (or numbered) variables. additional rules based on these variables with no additional parsing needed. In many ways, this parse-and-save-to-named-variables feature has the potential to completely revolutionize how SEC rules are written (especially for high performance environments) David Lang ------------------------------------------------------------------------------ LogMeIn Central: Instant, anywhere, Remote PC access and management. Stay in control, update software, and manage PCs from one command center Diagnose problems and improve visibility into emerging IT issues Automate, monitor and manage. Do more in less time with Central http://p.sf.net/sfu/logmein12331_d2d _______________________________________________ Simple-evcorr-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
