Thank you Risto for your quick answer.
We have also seen an increasing memory usage in a generic PERL process that
does not use SEC rules, so it's possible that the problem is located in the
PERL interpreter, as you have pointed. We will do some more research and let
you know our findings, if any.
Best regards,
Natalia
De: Risto Vaarandi [mailto:[email protected]]
Enviado el: lunes, 19 de mayo de 2014 22:39
Para: Natalia Iglesias
CC: [email protected]
Asunto: Re: [Simple-evcorr-users] Memory use of machines executing SEC rules
with Strawberry PERL under Windows 2008 server VMs
...also, in the case you can't generate the SIGUSR1 signal from commandline on
Windows, including the following SEC rule in your configuration might help:
type=single
ptype=substr
pattern=generate_dump
desc=produce the dumpfile
action=lcall %o -> ( sub { kill('USR1', $$); } )
This rule will match a line which contains the string "generate_dump", and if
such line is observed, SEC will send itself the SIGUSR1 signal. In order to
generate the dump file, you have to write the string "generate_dump" to one of
the SEC input files. By default, the dumpfile is written to /tmp/sec.dump.
Since the /tmp directory might not exist on the Windows platform, you can
override the name of the dump file with the --dump command line option.
kind regards,
risto
2014-05-19 18:34 GMT+03:00 Risto Vaarandi <[email protected]>:
hi Natalia,
...
As we need to find an explanation for this memory usage we would like to ask
you the following:
- when is the varmap variable disposed? We have read something about each time
a log is processed, but would like to be sure.
The 'varmap' statements create entries in pattern match cache, and these
entries can later be lookued up with 'Cached' patterns. The pattern match cache
is cleared after *each* incoming line has been processed, so that the
processing of the following line can start with an empty cache. This is
necessary, since otherwise results of the previous matching iteration would
interfere with the current iteration.
- is there any configuration parameter in SEC that we could adjust to control
the memory usage?
The only parameter which controls some aspects of memory usage is --evstoresize
command line option. This option is used for setting the upper limit for
context event stores. Since event stores have been designed for aggregating
events over time, they can occasionally grow too large. Also, this option was
introduced during early versions of SEC when it was not possible to remove
individual events from the store. The 2.7.X versions allow for this, and the
user can write an action list for shifting events out from an event store.
- how is the working memory usage reset?
There is no concept of memory reset, since SEC deletes a data structure
immediately if it is no longer used. After deletion, the actual memory
releasing is done by the Perl garbage collection engine. The problem you are
having could have two possible reasons -- Perl's internal garbage collection is
not working properly, or your rules trigger a large number of event correlation
operations or contexts which stay in memory without being dropped. While the
first issue can not be addresses with SEC, the second one can be investigated
more closely by letting SEC create a dump file with its internal state. On UNIX
platforms, the dump file can be created with the SIGUSR1 signal (I am not sure,
though, if/how Strawberry Perl is emulating this signal). Provided you can
create the dump file, inspecting its content should reveal the number of
objects created by your rules. If you see entries for a very large number of
contexts and/or event correlation operations in the dump file, the rules would
need appropriate modifications (for example, setting reasonable lifetimes for
contexts, or resetting counting operations with 'reset' action).
kind regards,
risto
Also we would like to ask you if there are any issues known to you regarding a
bad performance of SEC under windows. We have read some comments about it, but
have no official confirmation of that being the case.
And if you have any clue as to what is possibly going on we would very much
appreciate your advice.
We have SEC version 2.6.2 and Strawberry PERL 5.14.3.1-64 bits.
Thanks in advance and best regards,
Natalia Iglesias
------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
