Hi Risto,
OK, recall that I want to grab an integer from a log line, look it up in a
hash, and write the resulting value somewhere
Or, to provide context, one of my platforms (Isilon NAS cluster), logs messages
identifying the affected member by something Isilon calls an ‘Array ID’. I
don’t care about ‘Array ID’; rather, I care about “Logical Node Number” (LNN).
Fortunately, I can convert Array ID into LNN through the use of a look-up
table: I can use a Perl hash to perform this look-up.
I can print this mapping using a command-line tool run on an Isilon node.
isilon-cluster-15# isi_nodes "%{name}: LNN %{lnn}, Array ID %{id}"
isilon-cluster-1: LNN 1, Array ID 21
isilon-cluster-2: LNN 2, Array ID 24
isilon-cluster-3: LNN 3, Array ID 3
isilon-cluster-4: LNN 4, Array ID 4
isilon-cluster-5: LNN 5, Array ID 5
isilon-cluster-6: LNN 6, Array ID 6
isilon-cluster-7: LNN 7, Array ID 23
isilon-cluster-8: LNN 8, Array ID 8
isilon-cluster-9: LNN 9, Array ID 9
isilon-cluster-10: LNN 10, Array ID 10
isilon-cluster-11: LNN 11, Array ID 11
isilon-cluster-12: LNN 12, Array ID 12
isilon-cluster-13: LNN 13, Array ID 13
isilon-cluster-14: LNN 14, Array ID 18
isilon-cluster-15: LNN 15, Array ID 22
isilon-cluster-1#
I took the output from above to define %arrayid_to_lnn as follows:
# Global variables
type=Single
ptype=SubStr
pattern=SEC_STARTUP
context=SEC_INTERNAL_EVENT
desc=initialize array-id to node mapping hash
action=lcall %o-> (sub {\
%arrayid_to_lnn = (21 => 1,\
24 => 2,\
3 => 3,\
4 => 4,\
5 => 5,\
6 => 6,\
23 => 7,\
8 => 8,\
9 => 9,\
10 => 10,\
11 => 11,\
12 => 12,\
13 => 13,\
18 => 14,\
22 => 15,\
);\
);\
return 1;\
}\
)\
I am explicitly returning ‘1’ to signal that I am creating a variable and do
not care about the return value of the variable assignment.
I create a rule which uses %arrayid_to_lnn to translate Array ID into LNN
# Handle Isilon node down messages
type=SingleWithSuppress
ptype=regexp
pattern=T(\d\d:\d\d:\d\d)\-\d\d:\d\d (.*?) .*gmp.info.c.* group change:.* (node
\d+ drive \d+ changed to up)
desc=Drive Recover: $3
action=lcall %node $3 -> ( sub { $arrayid_to_lnn{$_[0]} } );\
if %node (write /home/tocops/.tocpipe ops $1 $2 Drive Recover: %node) else (
write /home/tocops/.tocpipe ops $1 $2 Drive Recover: $3)
window=5
Recall that the syslog line of interest might look as follows:
2017-08-03T06:07:31-07:00 isilon-cluster-10 /boot/kernel.amd64/kernel:
[gmp_info.c:1863](pid 38910="kt: gmp-config")(tid=103609) new group: <3,2302>:
{ 3-6:0-34, 8:0
-20,22-31,33-34,36-37, 9:1-21,24, 10-11:0-21, 12-13:0-34, 18:0-21, 21:0-34,
22:0-21, 24:0-34, down: 23, smb: 3-6,8-13,18,21-22,24, nfs:
3-6,8-13,18,21-22,24, all_enabled_protocols: 3-6,8-13,18,21-22,24 }
The 3rd match in the pattern grabs the Array ID, i.e. ‘23’ in the log line
above.
However, the action=lcall statement does not work the way I want it to … the
application reading the other end of .tocpipe doesn’t receive anything.
Furthermore, I see the following in syslog:
2017-08-04T04:20:58.861352-07:00 guru sec[23647]: Calling code
'CODE(0x264db30)' and setting variable '%node'
2017-08-04T04:20:58.861543-07:00 guru sec[23647]: Variable '%node' set to ''
<aside>Helpful informational messages – thank you for adding these</aside>
<aside>A management application is reading ‘.tocpipe’ and propagating what it
sees to a Web page … what I want to see is something like this
06:07:31 isilon-cluster-10 Down Nodes: 23
Which means something like ‘At 06:07:31, Node Isilon-Cluster-10 reported that
Array ID 23 went down”. Except that I don’t want to see Array ID ‘23’ there, I
want to see LNN ‘7’ instead, i.e. I want:
06:07:31 isilon-cluster-10 Down Nodes: 7
Translating ‘Array ID’ into ‘LNN’ is what I’m focused on with the effort I
describe in this posting</aside>
Anyway, back to this action / lcall syntax.
Clearly, I’m fumbling something about the lcall syntax
So, as I understand it, the “action=lcall…” line does something like this:
* Call the anonymous function “sub { $array_to_node{$_[0} }”, sending it
the argument $3 (in this example, $3 contains the string ‘23’)
* This anonymous function returns the string ‘7’
* Which gets assigned to the sec variable ‘%node’
But … as the syslog lines from sec indicate … instead of being assigned ‘7’,
%node gets assigned the empty string ‘’
QUESTIONS
* Would you have a suggestion on strategies for debugging this?
* Is there a way for me to insert ‘print’ statements into sec rule files
and then watch the output – in the example above, I want to print the value of
$arr
--sk
From: Risto Vaarandi [mailto:risto.vaara...@gmail.com]
Sent: Tuesday, July 25, 2017 2:47 PM
To: Stuart Kendrick <stua...@alleninstitute.org>
Cc: simple-evcorr-users@lists.sourceforge.net
Subject: Re: [Simple-evcorr-users] look-up a string in a hash, then write hash
value
hi Stuart,
you are on the right track and the PerlFunc pattern in your rule properly maps
the integer into a string. As explained in the documentation section of
different pattern types (see http://simple-evcorr.github.io/man.html#lbAG),
return values from the PerlFunc pattern function initialize match variables $1,
$2, etc. It is similar how RegExp pattern sets match variables, but this time
the variables and their values are not defined by regular expression capture
groups, but rather by values that you return from the function.
Since your rule returns only one value, it sets the $1 variable ($0 is set to
the entire matching line automatically by SEC). For example, the following rule
will write the string "down event with color <colortext>" to standard output
for each matching event:
type=Single
ptype=PerlFunc
pattern = sub { \
my %hash = ( 1 => "red", 2 => "green", 3 => "blue" ); \
my $line = $_[0]; \
my ($node) = ($line) =~ /down: (\d+)/; \
my $color = $hash{$node}; \
return $color; }
desc=extract color
action=write - down event with color $1
One side note -- if you are going to use %hash with integer-color mappings in
more than one pattern, I would recommend to make it a global hash that is set
once when SEC starts (this also requires --intevents command line option). That
would avoid unnecessary creation of the same hash in each rule:
type=Single
ptype=SubStr
pattern=SEC_STARTUP
context=SEC_INTERNAL_EVENT
desc=initialize integer-color mapping table
action=lcall %o -> ( sub { %hash = ( 1 => "red", 2 => "green", 3 => "blue" ) } )
type=Single
ptype=PerlFunc
pattern = sub { \
my $line = $_[0]; \
my ($node) = ($line) =~ /down: (\d+)/; \
my $color = $hash{$node}; \
return $color; }
desc=extract color
action=write - down event with color $1
Using a global hash would also allow to convert the integer into color not just
in PerlFunc patterns, but in any part of the rule where Perl code can be
executed. For example, here is another version of the above rule which uses a
RegExp pattern for matching the event, and employs the 'lcall' action for
obtaining the color in the action list. The 'if' action is then used to output
a string only if the integer has a corresponding color:
type=Single
ptype=RegExp
pattern=down: (\d+)
desc=extract color
action=lcall %color $1 -> ( sub { $hash{$_[0]} } ); \
if %color ( write - down event with color %color )
I hope these examples are helpful.
kind regards,
risto
2017-07-25 15:53 GMT+03:00 Stuart Kendrick
<stua...@alleninstitute.org<mailto:stua...@alleninstitute.org>>:
I want to grab a string from a log line, return a hash value, and write that
hash value
e.g. I grab an integer from a log line and want to write a color
LOG LINE
2017-07-25T04:31:10-07:00 server-foo … down: 3 ….
PSEUDO-SEC CONFIG
%hash = ( 1 => red, 2 => green, 3 => blue );
type=Single
ptype=RegExp
pattern=down: (\d+)
{somehow translate ‘3’ into ‘blue’}
action = write - “Blue is down”
MAN PAGE
I’m reading https://simple-evcorr.github.io/man.html , focused on the Perl
Integration section.
type=Jump
ptype=PerlFunc
pattern = sub {
my %hash = ( 1 => red, 2 => green, 3 => blue );
my $line = $_[0];
my ($node) = ($line) =~ /down: (\d+)/;
my $color = $hash{$node};
return $color;
}
QUESTION
* Am I headed in the right direction?
* Where would you point me next, in terms of reading about how to propagate
“$color” into an ‘action’ line?
--sk
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net<mailto:Simple-evcorr-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
