I think a full scripting language is a bad idea, there are just too many
assumptions about variable scope, lifetime, thread safety, flow control (loops),
etc
David Lang
On Fri, 22 May 2015, Rainer Gerhards wrote:
I tried to stay away from making the config script language a full
programming language. If we now think it needs to be, I won't object but
we should be sure there is good reason to do so and not just feature creep.
..
Sent from phone, thus brief.
Am 22.05.2015 21:48 schrieb "singh.janmejay" <[email protected]>:
Cool, will add doc soon.
I see what you are saying about functions, but keywords make the
language's core harder to grasp, which affects language's learning
curve. Functions are very different, because every-time a user
encounters one, they know its just another procedure/function which
will do something or return something. To be able to use keywords
effectively, one needs to know very well where they fit in the
language's constructs. For functions, they already know that.
But I do see the point you are making about arg order.
How about this: In addition to current function-calling-style, we
support python or R like calling style too, so people can optionally
mention argument names?
Another option is to use object-oriented style:
$.array.includes("foo") etc. We can make this work similar to how
python or c++ do, first argument is assumed to variable(in our case
expression) the function is dispatched upon. It'll not solve all
problems, but it'll do a brilliant job of disambiguating 2-arg
functions, while keeping language core lean.
On Sat, May 23, 2015 at 1:06 AM, David Lang <[email protected]> wrote:
On Sat, 23 May 2015, singh.janmejay wrote:
Oh, just recalled you mentioned documentation of array-subscripting.
Im unsure where to put it.
I am not sure if I documented it at all at the time of implementing
it, because I just didn't know where to put it.
Suggestions?
I would put it in two places.
1. in the section that talks about variables
2. as an example in the section that talks about tests
David Lang
On Sat, May 23, 2015 at 12:13 AM, singh.janmejay
<[email protected]> wrote:
Actually for contains, I was thinking of function rather than keyword.
I was proposing we leave the already supported keyword alone, but
don't add anymore, and develop more things in the form of functions as
opposed to keywords, so that it doesn't complicate the language too
much (Im thinking from the pov of a new person trying to learn this,
we can make stdlib very large, no one really worries about it, but if
we have too many keywords, it's going to scare people away). I know
stdlib is a little bit of a stretch, but closest analogy that comes to
my mind right now.
So I meant 'contains' the function, not the keyword.
Or may be we can use has(array, elem) or include(array, elem) or
hasItem(array, elem) etc to avoid confusing people by having similarly
named keyword and function.
Thoughts?
On Sat, May 23, 2015 at 12:04 AM, singh.janmejay
<[email protected]> wrote:
Ok, we'll have to get into a little bit of code here, I know no better
way to explain why it doesn't work for you, my apologies.
This is the form that the array that you built is loaded to:
https://github.com/rsyslog/rsyslog/blob/master/grammar/grammar.y#L215
We actually convert cnfarray to first element of that array for
expr-evaluation:
https://github.com/rsyslog/rsyslog/blob/master/grammar/rainerscript.c#L2314
Beyond this, all code assumes no cnfarray:
https://github.com/rsyslog/rsyslog/blob/master/runtime/msg.c#L4558
So cnfarray was actually meant for a different purpose, I haven't yet
dug into what that purpose is (@Rainer/David help?).
But its definitely not equal to json-array, which is why your loop
doesn't work.
So in order to loop, you need a json array, which generally comes from
mmjsonparse or mmnormalize generated object-graph.
Take a look at the test in the PR I linked-to in this thread. That
actually loops over an array and decides if it should stop processing
a message depending on presence of a certain value in an array. That
should serve as a working example of how to use foreach, except you'll
probably have something useful in place of stop.
On Fri, May 22, 2015 at 11:12 PM, David Lang <[email protected]> wrote:
On Fri, 22 May 2015, David Boles (dboles) wrote:
Yes, it is a string. Taking your clarification that foreach operates
on
arrays, I've tried the config below. When I push a log message from
an
application I observe:
- The action with the msg1 template occurs
- The action with the msg2 template does not occur
- The action with the msg3 template occurs
What should be changed below to cause the loop to be traversed?
Thanks,
David Boles
--------
module(load="imuxsock") # local system logging support
module(load="imkmsg") # structured kernel logging support
module(load="mmjsonparse")
template(name="msg1" type="list") { constant(value="pre-foreach
record\n")
}
template(name="msg2" type="list") { constant(value=" foreach
record\n")
}
template(name="msg3" type="list") { constant(value="post-foreach
record\n") }
if ($fromhost-ip == '127.0.0.1' and $syslogfacility-text != 'kern')
then {
action(type="mmjsonparse")
if $parsesuccess == "OK" then {
action(type="omfile" template="msg1"
file="/var/log/db_local_user_structured.log")
set $.myarray = [ "0", "1", "2", "3", "4", "5" ];
set $.index = 0;
there is no need for $.index = 0; it's not a counter, it just gets
assigned
to each of the contents of myarray
foreach ($.index in $.myarray) do {
action(type="omfile" template="msg2"
file="/var/log/db_local_user_structured.log")
}
this action should happen 6 times as I understand it.
action(type="omfile" template="msg3"
file="/var/log/db_local_user_structured.log")
}
}
--------
having multiple action writing to the same file can result in some
interesting issues (at least conceptually, I'll let Rainer speak up
if
they
have been solved in practice)
what I would do is
$template custom,"%$.custom%"
ruleset(name="db_local_user_structured"){
action(type="omfile" template="custom"
file="/var/log/db_local_user_structured.log")
}
and then replace each of the actions in your example with
set $.custom = "message";
call custom
David Lang
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of
singh.janmejay
Sent: Thursday, May 21, 2015 10:43 AM
To: rsyslog-users
Subject: Re: [rsyslog] Unable to use foreach
I think $!mse.element_indices is a string, right?
If its a string, you'll need to parse it to make it an array before
you
can loop on it. Foreach works only with arrays. It can be array or
anything
(string, object, numbers whatever), but it has to be an array.
You can use tokenized field-type to parse it and since you are
already
using mmjsonparse it shouldn't be a problem.
On Thu, May 21, 2015 at 8:47 PM, David Boles (dboles)
<[email protected]>
wrote:
Hi,
I am using rsyslog (v8.9) to process structured log data from
umberlog
and Linux's printk_emit. In the log message is a field
"$!mse.element_indices" that can have values such as "0", "0 1",
"0 1
2",
and so on. I would like to iterate over the delimited elements of
that value
and had supposed that foreach would do something like that.
With the config below I comment/uncomment the foreach loop. When
the
foreach loop is commented out rsyslog creates entries in both
mongodb
and
the file. When the foreach loop is uncommented, rsyslog produces
nothing in
either destination.
Why does this use of foreach fail? What should I be doing to
iterate?
Thanks,
David Boles
----------------------------------------------------------------------
--------
module(load="imuxsock")
module(load="imkmsg")
module(load="imtcp")
input(type="imtcp" port="10514")
module(load="mmjsonparse")
module(load="ommongodb")
kern.* /var/log/db_kernel.log
*.* /var/log/db_full.log
template(name="mse-structured-info" type="subtree" subtree="$!")
template(name="mse-all-info" type="list") {
property(name="jsonmesg" outname="msg") }
if ($fromhost-ip == '127.0.0.1' and $syslogfacility-text != 'kern')
then
{
action(type="mmjsonparse")
if $parsesuccess == "OK" then {
set $!foo = $!mse.element_indices;
set $!amph = "toad";
# foreach ($.index in $!mse.element_indices) do {
# set $!amph = "turtle";
# }
action(type="ommongodb" server="somemachine.somewhere.org"
db="logs" collection="syslog"
template="mse-structured-info")
action(type="omfile" template="mse-all-info"
file="/var/log/db_local_user_structured.log")
}
}
$WorkDirectory /var/spool/rsyslog
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T
LIKE THAT.
--
Regards,
Janmejay
http://codehunk.wordpress.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL:
This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites
beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
you
DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T
LIKE THAT.
--
Regards,
Janmejay
http://codehunk.wordpress.com
--
Regards,
Janmejay
http://codehunk.wordpress.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
--
Regards,
Janmejay
http://codehunk.wordpress.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
