Hi Tim,
You might want to consider that every event adds an entry
into all three hashes, and these are kept in the same process
space as SEC itself. With a large number of events (many thousands
or hundreds of thousands as in a Denial of Service attack), you may
run out of process memory or some other resource limit and SEC
would stop working.
You could achieve the same effect by marking your log file every
n calendar hours, and running a separate process at your designated
report hour to grep for log bunches (n, 2n, 3n, etc.) and process
your histogram.
Best Regards,
Jim B.
________________________________
From: [EMAIL PROTECTED] on behalf of Tim Peiffer
Sent: Mon 4/28/2008 11:21 PM
To: simple-evcorr-users@lists.sourceforge.net
Subject: [Simple-evcorr-users] Counting and profiling events
I am interested in profiling events. I like making histograms of
patterned events as a way of guiding efforts and understand what events
are important in the logs. So how would one use SEC to profile logs to
arrive at top-N sources, and top-N events? Below is what I am thinking.
# report every 24 hours.
type=Calendar
time=2 0 * * *
desc=Report events
action= = {
for ( sort keys %sources ) { \
printf "Source %s Frequency %d arrival rate
%5.2f \n", $_, $sources{$_}, $sources{$_}/86400; \
} \
for ( sort keys %events ) {
printf "Event %s Frequency %d arrival rate
%5.2f\n", $_, $events{$_}, $events{$_}/86400; \
} \
for ( sort keys %eventsources ) {
printf "Event-sources %s Frequency %d %5.2f\n",
$_, $eventsources{$_}, $eventsources{$_}/86400; \
} \
} ; \
= { %sources=(); %events=(); %eventsources=(); }
# Look for Cisco style events
type=single
continue=takeNext
ptype=regexp
pattern=\d+:\d+:\d+.*?(\S+)\s+\d+:.*?(%\S+-\d-\S+): .*
desc=$1:$2
context = ={ $sources{$1}++ ; $events{$2}++; $ $eventsources{"$1:$2"}++ }
action=none
Regards,
Tim Peiffer
Networking and Telecommunications Services
University of Minnesota/NorthernLights GigaPOP
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
Note: The information contained in this message may be privileged and
confidential and protected from disclosure. If the reader of this message is
not the intended recipient, or an employee or agent responsible for delivering
this message to the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication is strictly
prohibited. If you have received this communication in error, please notify us
immediately by replying to the message and deleting it from your computer.
Thank you. ThruPoint, Inc.
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
