Thank you very much, Risto. This is exactly what I was looking for. Also thank you David for pointing out liblognorm. I didn't know about it and it's pretty interesting.
Best regards, On Tue, 13 Dec 2011 14:05:28 +0200 Risto Vaarandi <[email protected]> wrote: > On 12/12/2011 04:01 PM, Alberto Cortón wrote: > > Hi, > > > > I would like to know if any of you have used SEC for normalizing log data. > > My first approach to this was to generate normalized events like this: > > > > action = event > > 'TIME=$1:::CODE=$3:::SRC_IP=$4:::SRC_PORT=$5:::DST_IP=$6:::DST_PORT=$7:::IFACE=$8:::PROTOCOL=$9:::ACTION=$10' > > > > and use a single pattern for generating the alerts > > > > pattern = > > TIME=(.+?):::CODE=(.+?):::SRC_IP=(.+?):::SRC_PORT=(.+?):::DST_IP=(.+?):::DST_PORT=(.+?):::IFACE=(.+?):::PROTOCOL=(.+?):::ACTION=(.+?) > > > > But this list could grow up to twenty or more parameters, so you would need > > a VERY large pattern for capturing the normalized events. Moreover, due to > > not all the devices log the same info (sometimes there is no interface > > information, for example) I would need to add several empty fields on > > almost any action. > > > > Any thoughts or suggestions? > > hi Alberto, > > using the 'event' action for generating new events is one opportunity, > but if you are worried that matching normalized events with regular > expressions is expensive, you can employ 'Cached' pattern types for this > purpose. > > The following simple example normalizes ssh and su session start events, > creating a cache entry for both event types from the the first two > rules. The third rule uses the Cached pattern for checking the presence > of the cache entry, and producing a generic output message for su and > ssh session starts: > > type=Single > ptype=RegExp > pattern=su:.*\bsession opened for user (\w+) by (\w+)\(uid=(\d+)\) > varmap=usersessopen; sulogin; from=1; to=2; uid=3 > continue=TakeNext > desc=user $+{from} switched to user $+{to} > action=none > > type=Single > ptype=RegExp > pattern=sshd\[(\d+)\]: Accepted password for (\w+) from ([\d.]+) port > (\d+) ssh2 > varmap=usersessopen; sshlogin; from=3; to=2; pid=1; rport=4 > continue=TakeNext > desc=user $+{to} logged in from $+{from} > action=none > > type=Single > ptype=Cached > pattern=usersessopen > desc=A session for user $+{to} was established from user/host $+{from} > action=write - %s > > Apart from the 'usersessopen' cache entry, specific entries are created > for su and ssh logins ('sulogin' and 'sshlogin'). By using them as > patterns, you can easily match one event type only and use information > that was stored for these event types (in the case of ssh login, process > number and remote port, and in the case of su, the user id). > > Note that the regular expression matching is done only once for each > event type, for recognizing the input event and creating the match > variables (in rules 1 and 2). The matching for 'Cached' pattern is > simply a matter of extracting previously stored match data from the > cache, where it was stored by the 'varmap' statement of rules 1 and 2. > > There is one subtlety you should know -- if you use 'context' and > 'varmap' fields together, 'varmap' will create a cache entry after a > regular expression match regardless of the context expression truth > value. In other words, the presence/absence of the cache entry only > reflects the result of the regular expression match. > > hope this helps, > risto > > > > > Regards, > > > > > > ------------------------------------------------------------------------------ > > Learn Windows Azure Live! Tuesday, Dec 13, 2011 > > Microsoft is holding a special Learn Windows Azure training event for > > developers. It will provide a great way to learn Windows Azure and what it > > provides. You can attend the event by watching it streamed LIVE online. > > Learn more at http://p.sf.net/sfu/ms-windowsazure > > _______________________________________________ > > Simple-evcorr-users mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users > > > > > ------------------------------------------------------------------------------ > Systems Optimization Self Assessment > Improve efficiency and utilization of IT resources. Drive out cost and > improve service delivery. Take 5 minutes to use this Systems Optimization > Self Assessment. http://www.accelacomm.com/jaw/sdnl/114/51450054/ > _______________________________________________ > Simple-evcorr-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users -- Alberto Cortón Dept. Integración de productos propios Tlf: 902 222 521 www.s21sec.com 10 años comprometidos con la seguridad. La información contenida en este mail, así como los archivos adjuntos, es CONFIDENCIAL. Grupo S21sec Gestión, S.A. garantiza la adopción de las medidas necesarias para asegurar el tratamiento confidencial de los datos de carácter personal. En el caso de que el destinatario del correo no sea usted, le rogamos envíe una notificación al remitente y lo destruya de forma inmediata. La lectura y/o manipulación de esta información en la situación señalada anteriormente será considerada ilegal, permitiendo a la empresa remitente realizar acciones legales de diferente envergadura. ------------------------------------------------------------------------------ Systems Optimization Self Assessment Improve efficiency and utilization of IT resources. Drive out cost and improve service delivery. Take 5 minutes to use this Systems Optimization Self Assessment. http://www.accelacomm.com/jaw/sdnl/114/51450054/ _______________________________________________ Simple-evcorr-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
