2012/11/29 Boyles, Gary P <[email protected]>:
> Hi,
>
>
>
> I was wondering if anyone is using JSON for events with SEC, and if so… how
> had you implemented it.
>
>
>
> In addition, David Lang wrote this about the SEC alpha version. I was
> wondering if there are any examples around this?
>
> A SEC rule snippet would be appreciated.
>
>
The e-mail David was referring to is quoted below.
Although this rule snippet is quite long, setting up the flatten()
function would not be necessary if incoming JSON log messages are
known to contain no subrecords (i.e., another {...} or [...] inside
original record which might in turn contain subrecords).
Since the message part of the JSON log messages are required to
conform to the format @cee: {...} , the last rule could be rewritten
as:
type=single
ptype=PerlFunc
pattern=sub { \
if ($_[0] !~ /\@cee:\s*(\{.*\})\s*$/) { return 0 }; \
my($ptr) = JSON::decode_json($1); \
my(%hash) = (); flatten($ptr, \%hash, ""); \
return \%hash; }
desc=find hostname and first 2 IP's from json structure
action=write - $+{host!ip!0} $+{host!ip!1} $+{host!dns}
This rule also assumes that all the keywords begin with either a
letter and underscore, and have letters, digits and underscores
following the first character. Also, exclamation mark is put as a
separator between parts of a multipart keyword. This should be in
accordance with the Lumberjack standard.
If illegal characters are used in keywords, resulting match variable
is not recognized by SEC, e.g., if $+{@field} is use in action list,
it is treated as a literary string and will not be substituted.
Unfortunately, some applications are using weird characters for field
names, although according to JSON definition they should never be
there :(
with kind regards,
risto
----------
hi David,
in order to give some idea how JSON can be handled with the current
alpha version, here is a ruleset which parses arbitrary single line
json messages. Fortunately, there is an excellent JSON module in CPAN
which is also a standard part of recent Perl installation. One feature
it lacks, though, is the flattening mechanism, so I've written this
myself. Hopefully some flattening routine will be part of the JSON
module in the future, and then the below ruleset would become really
short. In order to make it look more compact, the flatten() routine
could be put into a separate Perl module file and included with
'require' like JSON modul itself.
type=single
ptype=regexp
pattern=(SEC_STARTUP|SEC_RESTART)
desc=load JSON module and define flattening function
action=eval %o ( require JSON ); eval %o ( \
\
sub flatten { \
my($ref) = $_[0]; \
my($ret) = $_[1]; \
my($prefix) = $_[2]; \
my($key, $i); \
\
if (ref($ref) eq "HASH") { \
\
foreach $key (keys %{$ref}) { \
if (ref($ref->{$key}) eq "") { \
$ret->{$prefix . $key} = $ref->{$key}; \
} elsif (ref($ref->{$key}) eq "HASH") { \
flatten($ref->{$key}, $ret, $prefix . $key . "!"); \
} elsif (ref($ref->{$key}) eq "ARRAY") { \
flatten($ref->{$key}, $ret, $prefix . $key . "!"); \
} else { \
$ret->{$prefix . $key} = ${$ref->{$key}}; \
} \
} \
\
} elsif (ref($ref) eq "ARRAY") { \
\
for ($i = 0; $i < scalar(@{$ref}); ++$i) { \
if (ref($ref->[$i]) eq "") { \
$ret->{$prefix . $i} = $ref->[$i]; \
} elsif (ref($ref->[$i]) eq "HASH") { \
flatten($ref->[$i], $ret, $prefix . $i . "!"); \
} elsif (ref($ref->[$i]) eq "ARRAY") { \
flatten($ref->[$i], $ret, $prefix . $i . "!"); \
} else { \
$ret->{$prefix . $i} = ${$ref->[$i]}; \
} \
} \
\
} \
} \
)
type=single
ptype=PerlFunc
pattern=sub { \
chomp($_[0]); my($ptr) = JSON::decode_json($_[0]); \
my(%hash) = (); flatten($ptr, \%hash, ""); \
return \%hash; }
desc=find hostname and first 2 IP's from json structure
action=write - $+{host!ip!0} $+{host!ip!1} $+{host!dns}
If I feed SEC with the following structure
{"host":{"ip":["127.0.0.1","127.0.0.2"],"dns":"localhost"}}
the following line is echoed back:
127.0.0.1 127.0.0.2 localhost
Of course, since the above parsing is done without any knowledge about
the field names, the user has to know which field names are available.
with kind regards,
risto
------------------------------------------------------------------------------
Keep yourself connected to Go Parallel:
VERIFY Test and improve your parallel project with help from experts
and peers. http://goparallel.sourceforge.net
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
