SEC works just fine reading from named pipe or stdin from a socket. I have
this working with rsyslog with the only problem being that when I want to
change the SEC rules, it involves a restart of rsyslog.
David Lang
On Sat, 13 Aug 2011, Jean Baptiste Favre wrote:
> - second is: in order to increase performances, and since logs are
> collected with rsyslog and stored in a file, is there any solution to
> use a socket or a pipe as entry for SEC ? That could avoid disks I/O
> operations. Disks I/O are well known bottleneck on Amazon, which is were
> I host my server.
>
> Best regards,
> JB
>
>
> On 07/08/2011 23:08, Risto Vaarandi wrote:
>> hi Jean Baptiste,
>>
>> you might try the following rule:
>>
>> type=EventGroup
>> ptype=regexp
>> pattern=" (\d)[0-9]{2} \d+
>> count=lcall %ret $1 -> ( sub { ++$req; if ($_[0] == 2) { ++$req2; } } );
>> desc=Count HTTP request response codes
>> action=none
>> init=lcall %ret -> ( sub { $req = 0; $req2 = 0; } )
>> end=lcall %ret -> ( sub { return "$req requests, $req2 2xx requests"; } ); \
>> write - %ret
>> window=60
>>
>> This rule keeps two Perl variables ($req and $req2) in memory which
>> are accessed with the 'lcall' action. I am proposing 'lcall' instead
>> of 'eval', since the 'eval' action compiles code before each execution
>> and is thus much slower. Since your event rate is 3000 per second,
>> 'lcall' is much better suited for maintaining the counters.
>> The above rule starts an event correlation operation at the first
>> occurrence of the web server event. Unlike the example rule you found
>> from the archives, this rule does not have 'multact' field set to yes,
>> which means that the first event freezes the event correlation window
>> (since 'thresh' field defaults to 1), and the operation terminates
>> after 60 seconds. Before terminating, the 'end' action creates the
>> report string and writes it to standard output. The next web server
>> event will start the operation again, and the $req and $req2 counters
>> will be reset to 0 with the 'init' action. Note that we are not using
>> the 'action' field at all, but tackle the problem by putting auxiliary
>> 'init', 'count' and 'end' actions together. In other words, although
>> the main purpose of EventGroup rule is to detect groups of different
>> events in a time window, it can also employed for handling various
>> counting tasks for events of one type only.
>> Hope this example is helpful.
>> regards,
>> risto
>>
>> 2011/8/5 Jean Baptiste Favre <[email protected]>:
>>> Hello,
>>> I need to count event number and report it every minutes.
>>> I'm looking for a more elegant and powerfull way (I have to analyze 3000
>>> req/s) to do it than what I found already (see bellow).
>>>
>>> I saw http://sourceforge.net/mailarchive/message.php?msg_id=27274908 but
>>> can't figure out if this can help me.
>>>
>>> My current config
>>>
>>> ## INITialize SEC
>>> type=Single
>>> ptype=RegExp
>>> pattern=(SEC_STARTUP|SEC_RESTART)
>>> context=SEC_INTERNAL_EVENT
>>> desc=Init counters with 0
>>> action=assign %REQ_NBR_COUNT 0; \
>>> assign %REQ_2xx_COUNT 0;
>>>
>>> ## Extract HTTP code from request and through event
>>> type=Single
>>> ptype=RegExp
>>> pattern=.*" (\d)[0-9]{2} \d+ .*
>>> desc=Request received
>>> action=event 0 REQ_$1
>>>
>>> ## Counts 2xx HTTP code number
>>> type=Single
>>> ptype=SubStr
>>> pattern=REQ_2
>>> continue=TakeNext
>>> desc=HTTP 2xx code found
>>> action=eval %REQ_2xx_COUNT ( ++$REQ_2xx_COUNT )
>>>
>>> [...]
>>> skipped other counters as they use the same definition as above
>>> [...]
>>>
>>> ## Counts requests number
>>> type=Single
>>> ptype=SubStr
>>> pattern=REQ_
>>> desc=Request counter
>>> action=eval %REQ_NBR_COUNT ( ++$REQ_NBR_COUNT )
>>>
>>> ## Flush counters every minute
>>> type=Calendar
>>> time=* * * * *
>>> desc=monitoring.logs.http
>>> context=!SEC_INTERNAL_EVENT
>>> action=write - - %s.req[tot] %REQ_NBR_COUNT; \
>>> eval %REQ_NBR_COUNT ( $REQ_NBR_COUNT = 0 ); \
>>> write - - %s.req[2xx] %REQ_2xx_COUNT; \
>>> eval %REQ_2xx_COUNT ( $REQ_2xx_COUNT = 0 );
>>>
>>> Any comment welcomed,
>>>
>>> Regards,
>>> JB
>>>
>>> ------------------------------------------------------------------------------
>>> BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
>>> The must-attend event for mobile developers. Connect with experts.
>>> Get tools for creating Super Apps. See the latest technologies.
>>> Sessions, hands-on labs, demos & much more. Register early & save!
>>> http://p.sf.net/sfu/rim-blackberry-1
>>> _______________________________________________
>>> Simple-evcorr-users mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>>>
>>
>> ------------------------------------------------------------------------------
>> BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
>> The must-attend event for mobile developers. Connect with experts.
>> Get tools for creating Super Apps. See the latest technologies.
>> Sessions, hands-on labs, demos & much more. Register early & save!
>> http://p.sf.net/sfu/rim-blackberry-1
>> _______________________________________________
>> Simple-evcorr-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>>
>
>
> ------------------------------------------------------------------------------
> FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
> Subversion made easy with a complete admin console. Easy
> to use, easy to manage, easy to install, easy to extend.
> Get a Free download of the new open ALM Subversion platform now.
> http://p.sf.net/sfu/wandisco-dev2dev
> _______________________________________________
> Simple-evcorr-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>
------------------------------------------------------------------------------
FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
Subversion made easy with a complete admin console. Easy
to use, easy to manage, easy to install, easy to extend.
Get a Free download of the new open ALM Subversion platform now.
http://p.sf.net/sfu/wandisco-dev2dev
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
