-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
John P. Rouillard wrote:
| In message <[EMAIL PROTECTED]>,
| Hugo van der Kooij writes:
|> I have events sets that consists of 4 lines that will be written over
|> the space of several seconds (at this moment about 45 to 60 seconds).
|> They consist of 4 events and I want to define 1 action on the 4th event.
|>
|> For example part of the data:
|>
|> VERIFY 194.151.25.153 1200237982 START 1200237982
|> VERIFY 194.151.25.153 1200237982 CONNECTED 1200237982
|> VERIFY 194.151.25.153 1200237982 SEND 1200238013
|> VERIFY 194.151.25.153 1200237982 RECEIVED 1200238035
|
|> VERIFY 194.151.25.153 1200238201 START 1200238201
|> VERIFY 194.151.25.153 1200238201 CONNECTED 1200238201
|> VERIFY 194.151.25.153 1200238201 SEND 1200238232
|> VERIFY 194.151.25.153 1200238201 RECEIVED 1200238248
|
|> VERIFY 194.151.25.153 1200238501 START 1200238501
|> VERIFY 194.151.25.153 1200238501 CONNECTED 1200238501
|> VERIFY 194.151.25.153 1200238501 SEND 1200238532
|> VERIFY 194.151.25.153 1200238501 RECEIVED 1200238550
|>
|> The 3rd and 5th column are unixtime.
|>
|> To make matters more complicated I might get intersected events from
|> other hosts in here as well. So then the logs will look much more
|> complicated.
|>
|> Each event set is in fact unique and one can think of the IP address
|> (2nd column) and the ID (3rd column) as the keys to find unique sets.
|
| Good info to have we will use it below.
|
|> I want to learn on each RECEIVED event the time difference between the
|> CONNECTED and SEND events and the SEND and RECEIVED events.
|>
|> I also want to know if there was no RECEIVED event after a CONNECTED
|> event within N seconds (N being about 2 to 5 minutes).
|>
|> Some thing like: Host 194.151.25.153 took 32 seconds to receive and 28
|> seconds to process event 1200238501
|
| I will leave producing the exact output you want to the reader, but a
| ruleset the matches your data could look something like:
|
| # rule 1
| type=single
| desc= detect start of sequence for host $1 id $2
| ptype=regexp
| pattern=VERIFY ([0-9.]*) ([0-9]*) START ([0-9]*)
| rem= may need to set a max tine on this context
| action=create context_$1_$2; add context_$1_$2 $0
|
| # rule 2
| type=single
| desc= detect connected part of sequence for host $1 id $2
| ptype=regexp
| pattern=VERIFY ([0-9.]*) ([0-9]*) CONNECTED ([0-9]*)
| rem= match this rule only if a START was seen prior
| rem= otherwise something is wrong and we need to punt.
| context= context_$1_$2
| rem = store connected time in a context for later use
| action=add context_$1_$2 $0; add context_connected_time_$1_$2 $3
|
| # rule 3
| type=single
| cont=takenext
| desc= detect send part of sequence for host $1 id $2
| ptype=regexp
| pattern=VERIFY ([0-9.]*) ([0-9]*) SEND ([0-9]*)
| rem= match this rule only if a START was seen
| context= context_$1_$2
| action=copy context_connected_time_$1_$2 %{connectedtime}; \
| eval %{diff} =($3 - %{connectedtime}; \
| add context_$1_$2 $0; \
| add context_$1_$2 CONNECTED to SEND time %{diff} seconds; \
| add context_connected_time_$1_$2 $3
|
| # rule 4
| type=pairwithwindow
| desc = detect RECEIVED for host $1 id $2 or detect missing received
| context= context_$1_$2
| ptype=regexp
| pattern=VERIFY ([0-9.]*) ([0-9]*) SEND ([0-9]*)
| context= context_$1_$2
| ptype2=regexp
| pattern2=VERIFY ([0-9.]*) ([0-9]*) RECEIVED ([0-9]*)
| rem = report missing received after 3 minutes (180 seconds)
| window = 180
| rem = no RECEIVED event
| action= report context_$1_$2 mail -s "failed to get a RECEIVED entry
for host $1 id $2. Log is"
| rem=received is found $3 is the time extracted using "pattern2",
| rem=%3 is the time extracted from the send line using "pattern"
| action=eval %{diff} =($3 - %3); \
| add context_$1_$2 $0; \
| add context_$1_$2 SEND to RECEIVED time %{diff} seconds; \
| report context_$1_$2 mail -s "Report on host $1 id $2 connection"
|
|
| may work (untested). I usually get the algorthm in the right ballpark,
| but get the exact syntax wrong. It should produce output like:
|
| VERIFY 194.151.25.153 1200238501 START 1200238501
| VERIFY 194.151.25.153 1200238501 CONNECTED 1200238501
| VERIFY 194.151.25.153 1200238501 SEND 1200238532
| CONNECTED to SEND time 31 seconds
| VERIFY 194.151.25.153 1200238501 RECEIVED 1200238550
| SEND to RECEIVED time 18 seconds
It seems I either use another version of sec.pl as only 2 rules compiled
correctly.
sec.pl -conf=/usr/local/etc/sec-smtp.conf
- -input=/var/log/loopback/20080119.log
SEC (Simple Event Correlator) 2.4.1
Reading configuration from /usr/local/etc/sec-smtp.conf
/usr/local/etc/sec-smtp.conf line 31: Invalid keyword cont
Rule in /usr/local/etc/sec-smtp.conf at line 30: Invalid action list '
copy context_connected_time_$1_$2 %{connectedtime}; eval %{diff}
=($3 - %{connectedtime}; add context_$1_$2 $0; add context_$1_$2
CONNECTED to SEND time %{diff} seconds; add
context_connected_time_$1_$2 $3 '
Rule in /usr/local/etc/sec-smtp.conf at line 45: Keyword 'desc2' missing
(needed for the rule type PAIRWITHWINDOW)
Rule in /usr/local/etc/sec-smtp.conf at line 45: Keyword 'action2'
missing (needed for the rule type PAIRWITHWINDOW)
2 rules loaded from /usr/local/etc/sec-smtp.conf
I got the bit where cont= needs to be replaced continue= fixed. And I
did something to get rid of desc2 and action2 lines.
So now it looks like:
sec.pl -conf=/usr/local/etc/sec-smtp.conf
- -input=/var/log/loopback/20080119.log
SEC (Simple Event Correlator) 2.4.1
Reading configuration from /usr/local/etc/sec-smtp.conf
Rule in /usr/local/etc/sec-smtp.conf at line 30: Invalid action list '
copy context_connected_time_$1_$2 %{connectedtime}; eval %{diff}
=($3 - %{connectedtime}; add context_$1_$2 $0; add context_$1_$2
CONNECTED to SEND time %{diff} seconds; add
context_connected_time_$1_$2 $3 '
Rule in /usr/local/etc/sec-smtp.conf at line 45: Variable %{diff} does
not have the form %<letter>[<letter>|<digit>|<underscore>]...
Rule in /usr/local/etc/sec-smtp.conf at line 45: Invalid action list '
eval %{diff} =($3 - %3); add context_$1_$2 $0; add
context_$1_$2 SEND to RECEIVED time %{diff} seconds; report
context_$1_$2 mail -s "Report on host $1 id $2 connection" '
2 rules loaded from /usr/local/etc/sec-smtp.conf
Creating context 'context_194.151.25.153_1200774001'
Adding event 'VERIFY 194.151.25.153 1200774001 START 1200774001' to
context 'context_194.151.25.153_1200774001'
Adding event 'VERIFY 194.151.25.153 1200774001 CONNECTED 1200774001' to
context 'context_194.151.25.153_1200774001'
Adding event '1200774001' to context
'context_connected_time_194.151.25.153_1200774001'
Creating context 'context_194.151.25.153_1200774301'
Adding event 'VERIFY 194.151.25.153 1200774301 START 1200774301' to
context 'context_194.151.25.153_1200774301'
Adding event 'VERIFY 194.151.25.153 1200774301 CONNECTED 1200774301' to
context 'context_194.151.25.153_1200774301'
Adding event '1200774301' to context
'context_connected_time_194.151.25.153_1200774301'
Creating context 'context_194.151.25.153_1200774601'
Adding event 'VERIFY 194.151.25.153 1200774601 START 1200774601' to
context 'context_194.151.25.153_1200774601'
Adding event 'VERIFY 194.151.25.153 1200774601 CONNECTED 1200774601' to
context 'context_194.151.25.153_1200774601'
Adding event '1200774601' to context
'context_connected_time_194.151.25.153_1200774601'
Unfortunatly I am not exactly fluent in SEC. I may work it out in time
if I get a bundle of pages printed stick them under a pillow, sleep over
it, read them a coule of times and play with it for a couple of weeks.
But if someone can easily see what is wrong with them then I sure would
appriciate it.
Hugo.
- --
[EMAIL PROTECTED] http://hugo.vanderkooij.org/
PGP/GPG? Use: http://hugo.vanderkooij.org/0x58F19981.asc
A: Yes.
>Q: Are you sure?
>>A: Because it reverses the logical flow of conversation.
>>>Q: Why is top posting frowned upon?
Bored? Click on http://spamornot.org/ and rate those images.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
iD8DBQFHkl+ABvzDRVjxmYERApakAKCh6qFJ+mryIw1JuFIaoutMW00jYgCgpibi
QLe7H3V6ArnD8rl9P5q6zmw=
=zlxJ
-----END PGP SIGNATURE-----
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
