...to complement my previous mail, I think it is simpler to set up
dedicated perl scripts for preparing input for sec, rather than trying to
incorporate data preprocessing into sec rules.
Here are two perl scripts which follow the example Mark has already
provided. Firstly, generate_timestamp.pl looks like follows (note that it
currently works for timestamps like "2015 Jun 20 12:00:00" and "Jun 21
12:13:41", and for other formats you need to adjust the timestamp parsing
regular expression):
#!/usr/bin/perl -w
#
# generate_timestamp.pl
# Prepend 'seconds since epoch' to each input line
use Time::Local;
%months = ( 'Jan' => 0, 'Feb' => 1, 'Mar' => 2, 'Apr' => 3,
'May' => 4, 'Jun' => 5, 'Jul' => 6, 'Aug' => 7,
'Sep' => 8, 'Oct' => 9, 'Nov' => 10, 'Dec' => 11 );
@time = localtime(time());
$year = $time[5] + 1900;
$month = $time[4];
while (<STDIN>) {
if
(/^(?:(\d{4})\s+)?([A-Z][a-z]{2})\s+(\d{1,2})\s+(\d{2}):(\d{2}):(\d{2})/) {
$y = defined($1)?$1:$year;
$m = exists($months{$2})?$months{$2}:$month;
$time = timelocal($6, $5, $4, $3, $m, $y);
print "$time ", $_;
}
}
And then the replay_events.pl script:
#!/usr/bin/perl -w
#
# replay_events.pl
# Replay sorted events generated by generate_timestamp.pl
select STDOUT;
$| = 1;
while (<STDIN>) {
if ($_ !~ /^(\d+) (.*)/) { next; }
if (!defined($previous)) {
print $2, "\n";
$previous = $1;
next;
}
$d = $1 - $previous;
if ($d < 0) { next; }
sleep($d);
print $2, "\n";
$previous = $1;
}
After having those scripts implemented, replaying past events becomes a
matter of simple UNIX pipeline. For example, the following pipeline joins
/var/log/messages and /var/log/secure into a single event stream and
replays this to sec:
cat /var/log/messages /var/log/secure | ./generate_timestamp.pl | sort | ./
replay_events.pl | /usr/bin/sec --conf=test.conf --input=- --notail
Also, I would suggest to include any database queries into a dedicated
preprocessing script which precedes sec in the UNIX pipeline.
To summarize, I strongly believe that the clear separation of data
preprocessing from sec-based event correlation is the best solution for
you, in order to keep your configuration manageable and efficient.
hope this helps,
risto
2015-07-01 13:15 GMT+03:00 Rajesh M <[email protected]>:
> Hi Risto,
>
> I am implementing a perl script which basically accepts the input file and
> time in sec, after search for particular pattern and take out the time
> stamp and adds input time to that, generates "New Time stamp". Later the
> output is syslog events from input file between Original TS and New TS.
>
> my .conf file:
> ------------------
> type=Single
> ptype=RegExp
> pattern="ALARM RAISE 70307" [Very first rasie event in alarm.log]
> desc=$0
> action=spawn /var/test/my.pl
>
> type=Single
> ptype=RegExp
> pattern=.*logf started
> desc=Matched event
> action=write /home/test.out
>
>
> Basically I am trying to trigger the scrpit based on my 1st rule match and
> wants to fed the script output to SEC for 2nd rule match.
>
> Whenever I was executing this .conf, the script was calling and it
> wouldn't goto 2nd rule and also the script was exectuing how many number of
> times the 1st rule matches.Please suggest what is the correct way of doing
> this operation in SEC.
>
> Thanks & Regards,
> Karthik
>
>
> ------------------------------------------------------------------------------
> Don't Limit Your Business. Reach for the Cloud.
> GigeNET's Cloud Solutions provide you with the tools and support that
> you need to offload your IT needs and focus on growing your business.
> Configured For All Businesses. Start Your Cloud Today.
> https://www.gigenetcloud.com/
> _______________________________________________
> Simple-evcorr-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>
>
------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
