Hi all:
When sec creates a dump file, the input sources are reported as:
Input sources:
============================================================
/var/log/messages (status: Open, type: regular file, device/inode:
64774/8339528, received data: 180797 lines, context: _FILE_EVENT_messages)
/var/spool/sec/Control (status: Open, type: pipe, device/inode: 64770/584644,
received data: 0 lines, context: CONTROL)
would it be possible to get a byte ofset at which SEC is reading put
in there as well for "type: regular file".
What I am trying to do is see how far SEC has to process to reach the
end of the file (aka realtime). Ideally I should be able to ls -l the
file and be able to compare that to the reported offset. The
difference between the two is how many bytes sec has to process to
reach realtime.
Counting the number of lines in the file takes too long to get an
accurate reading and can't be compared to the number of lines of
received data unless the file was being processed from the beginning.
Something like:
/var/log/messages (status: Open, type: regular file, device/inode:
64774/8339528, received data: 180797 lines, file position: 321992882 bytes,
context: _FILE_EVENT_messages)
Granted the number of bytes/event isn't constant or even known, but if
I start seeing a 5 KB difference I know I have an issue I need to
address.
I am currently using the following command:
date
tail -n 1 /data/log/system/messages
echo
sudo /etc/init.d/sec_rsyslog_master_mon dump | \
sed -n '1,/Last 100 input/p'| tail -n 10
to get:
* the current date (for comparison aginst the datestamps in the file),
* a tail to get the last logged line (if the date doesn't match
the timestamp on that line then I have a log receipt issue)
* the last few lines seen by sec so I can find the same line as
reported by tail.
not a great tool I have to admit. But it is showing me that I need to
optimize my ruleset somewhere. Which leads me to wonder if some sort
of profiling mode could be added to sec that tells me for every rule:
how many events are compared to the rule (the current metrics only
tell me how many events were matched by the rule including
context processing). With this info I can restructure the order
to put more expensive/inefficient rules later in processing.
how many times an event is processed by a rule. This is sort of the
inverse of the stat above, but is a good metric to use to tune
rules as reducing the number of event -> rule applications
seems to be the best way to reduce processing time.
how long on average (in real and cpu second/centiseconds) it takes
to process an event against a rule (provides an indication of
how efficient the regexp/rule is).
I expect this would toss performance through the floor, but I could
see this as being a useful offline mode to throw thousands of same
events against and use it to tune the ruleset order, regexps etc.
Comments?
--
-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.
------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
