I tried to stay away from making the config script language a full programming language. If we now think it needs to be, I won't object but we should be sure there is good reason to do so and not just feature creep. ..
Sent from phone, thus brief. Am 22.05.2015 21:48 schrieb "singh.janmejay" <[email protected]>: > Cool, will add doc soon. > > I see what you are saying about functions, but keywords make the > language's core harder to grasp, which affects language's learning > curve. Functions are very different, because every-time a user > encounters one, they know its just another procedure/function which > will do something or return something. To be able to use keywords > effectively, one needs to know very well where they fit in the > language's constructs. For functions, they already know that. > > But I do see the point you are making about arg order. > > How about this: In addition to current function-calling-style, we > support python or R like calling style too, so people can optionally > mention argument names? > > Another option is to use object-oriented style: > $.array.includes("foo") etc. We can make this work similar to how > python or c++ do, first argument is assumed to variable(in our case > expression) the function is dispatched upon. It'll not solve all > problems, but it'll do a brilliant job of disambiguating 2-arg > functions, while keeping language core lean. > > > On Sat, May 23, 2015 at 1:06 AM, David Lang <[email protected]> wrote: > > On Sat, 23 May 2015, singh.janmejay wrote: > > > >> Oh, just recalled you mentioned documentation of array-subscripting. > >> Im unsure where to put it. > >> > >> I am not sure if I documented it at all at the time of implementing > >> it, because I just didn't know where to put it. > >> > >> Suggestions? > > > > > > I would put it in two places. > > > > 1. in the section that talks about variables > > > > 2. as an example in the section that talks about tests > > > > David Lang > > > > > >> On Sat, May 23, 2015 at 12:13 AM, singh.janmejay > >> <[email protected]> wrote: > >>> > >>> Actually for contains, I was thinking of function rather than keyword. > >>> > >>> I was proposing we leave the already supported keyword alone, but > >>> don't add anymore, and develop more things in the form of functions as > >>> opposed to keywords, so that it doesn't complicate the language too > >>> much (Im thinking from the pov of a new person trying to learn this, > >>> we can make stdlib very large, no one really worries about it, but if > >>> we have too many keywords, it's going to scare people away). I know > >>> stdlib is a little bit of a stretch, but closest analogy that comes to > >>> my mind right now. > >>> > >>> So I meant 'contains' the function, not the keyword. > >>> > >>> Or may be we can use has(array, elem) or include(array, elem) or > >>> hasItem(array, elem) etc to avoid confusing people by having similarly > >>> named keyword and function. > >>> > >>> Thoughts? > >>> > >>> On Sat, May 23, 2015 at 12:04 AM, singh.janmejay > >>> <[email protected]> wrote: > >>>> > >>>> Ok, we'll have to get into a little bit of code here, I know no better > >>>> way to explain why it doesn't work for you, my apologies. > >>>> > >>>> This is the form that the array that you built is loaded to: > >>>> https://github.com/rsyslog/rsyslog/blob/master/grammar/grammar.y#L215 > >>>> > >>>> We actually convert cnfarray to first element of that array for > >>>> expr-evaluation: > >>>> > >>>> > https://github.com/rsyslog/rsyslog/blob/master/grammar/rainerscript.c#L2314 > >>>> > >>>> Beyond this, all code assumes no cnfarray: > >>>> https://github.com/rsyslog/rsyslog/blob/master/runtime/msg.c#L4558 > >>>> > >>>> So cnfarray was actually meant for a different purpose, I haven't yet > >>>> dug into what that purpose is (@Rainer/David help?). > >>>> > >>>> But its definitely not equal to json-array, which is why your loop > >>>> doesn't work. > >>>> > >>>> So in order to loop, you need a json array, which generally comes from > >>>> mmjsonparse or mmnormalize generated object-graph. > >>>> > >>>> Take a look at the test in the PR I linked-to in this thread. That > >>>> actually loops over an array and decides if it should stop processing > >>>> a message depending on presence of a certain value in an array. That > >>>> should serve as a working example of how to use foreach, except you'll > >>>> probably have something useful in place of stop. > >>>> > >>>> > >>>> On Fri, May 22, 2015 at 11:12 PM, David Lang <[email protected]> wrote: > >>>>> > >>>>> On Fri, 22 May 2015, David Boles (dboles) wrote: > >>>>> > >>>>>> Yes, it is a string. Taking your clarification that foreach operates > >>>>>> on > >>>>>> arrays, I've tried the config below. When I push a log message from > an > >>>>>> application I observe: > >>>>>> > >>>>>> - The action with the msg1 template occurs > >>>>>> - The action with the msg2 template does not occur > >>>>>> - The action with the msg3 template occurs > >>>>>> > >>>>>> What should be changed below to cause the loop to be traversed? > >>>>>> > >>>>>> Thanks, > >>>>>> > >>>>>> David Boles > >>>>>> > >>>>>> -------- > >>>>>> module(load="imuxsock") # local system logging support > >>>>>> module(load="imkmsg") # structured kernel logging support > >>>>>> > >>>>>> module(load="mmjsonparse") > >>>>>> > >>>>>> template(name="msg1" type="list") { constant(value="pre-foreach > >>>>>> record\n") > >>>>>> } > >>>>>> template(name="msg2" type="list") { constant(value=" foreach > >>>>>> record\n") > >>>>>> } > >>>>>> template(name="msg3" type="list") { constant(value="post-foreach > >>>>>> record\n") } > >>>>>> > >>>>>> if ($fromhost-ip == '127.0.0.1' and $syslogfacility-text != 'kern') > >>>>>> then { > >>>>>> action(type="mmjsonparse") > >>>>>> if $parsesuccess == "OK" then { > >>>>>> > >>>>>> action(type="omfile" template="msg1" > >>>>>> file="/var/log/db_local_user_structured.log") > >>>>>> > >>>>>> set $.myarray = [ "0", "1", "2", "3", "4", "5" ]; > >>>>>> set $.index = 0; > >>>>> > >>>>> > >>>>> > >>>>> there is no need for $.index = 0; it's not a counter, it just gets > >>>>> assigned > >>>>> to each of the contents of myarray > >>>>> > >>>>>> foreach ($.index in $.myarray) do { > >>>>>> action(type="omfile" template="msg2" > >>>>>> file="/var/log/db_local_user_structured.log") > >>>>>> } > >>>>> > >>>>> > >>>>> > >>>>> this action should happen 6 times as I understand it. > >>>>> > >>>>>> action(type="omfile" template="msg3" > >>>>>> file="/var/log/db_local_user_structured.log") > >>>>>> } > >>>>>> } > >>>>>> -------- > >>>>> > >>>>> > >>>>> > >>>>> having multiple action writing to the same file can result in some > >>>>> interesting issues (at least conceptually, I'll let Rainer speak up > if > >>>>> they > >>>>> have been solved in practice) > >>>>> > >>>>> what I would do is > >>>>> > >>>>> $template custom,"%$.custom%" > >>>>> > >>>>> ruleset(name="db_local_user_structured"){ > >>>>> action(type="omfile" template="custom" > >>>>> file="/var/log/db_local_user_structured.log") > >>>>> } > >>>>> > >>>>> and then replace each of the actions in your example with > >>>>> > >>>>> set $.custom = "message"; > >>>>> call custom > >>>>> > >>>>> David Lang > >>>>> > >>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -----Original Message----- > >>>>>> From: [email protected] > >>>>>> [mailto:[email protected]] On Behalf Of > singh.janmejay > >>>>>> Sent: Thursday, May 21, 2015 10:43 AM > >>>>>> To: rsyslog-users > >>>>>> Subject: Re: [rsyslog] Unable to use foreach > >>>>>> > >>>>>> I think $!mse.element_indices is a string, right? > >>>>>> > >>>>>> If its a string, you'll need to parse it to make it an array before > >>>>>> you > >>>>>> can loop on it. Foreach works only with arrays. It can be array or > >>>>>> anything > >>>>>> (string, object, numbers whatever), but it has to be an array. > >>>>>> > >>>>>> You can use tokenized field-type to parse it and since you are > already > >>>>>> using mmjsonparse it shouldn't be a problem. > >>>>>> > >>>>>> On Thu, May 21, 2015 at 8:47 PM, David Boles (dboles) > >>>>>> <[email protected]> > >>>>>> wrote: > >>>>>>> > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> I am using rsyslog (v8.9) to process structured log data from > >>>>>>> umberlog > >>>>>>> and Linux's printk_emit. In the log message is a field > >>>>>>> "$!mse.element_indices" that can have values such as "0", "0 1", > "0 1 > >>>>>>> 2", > >>>>>>> and so on. I would like to iterate over the delimited elements of > >>>>>>> that value > >>>>>>> and had supposed that foreach would do something like that. > >>>>>>> > >>>>>>> With the config below I comment/uncomment the foreach loop. When > the > >>>>>>> foreach loop is commented out rsyslog creates entries in both > mongodb > >>>>>>> and > >>>>>>> the file. When the foreach loop is uncommented, rsyslog produces > >>>>>>> nothing in > >>>>>>> either destination. > >>>>>>> > >>>>>>> Why does this use of foreach fail? What should I be doing to > iterate? > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>> David Boles > >>>>>>> > >>>>>>> > >>>>>>> > ---------------------------------------------------------------------- > >>>>>>> -------- > >>>>>>> > >>>>>>> module(load="imuxsock") > >>>>>>> module(load="imkmsg") > >>>>>>> module(load="imtcp") > >>>>>>> input(type="imtcp" port="10514") > >>>>>>> module(load="mmjsonparse") > >>>>>>> module(load="ommongodb") > >>>>>>> > >>>>>>> kern.* /var/log/db_kernel.log > >>>>>>> *.* /var/log/db_full.log > >>>>>>> > >>>>>>> template(name="mse-structured-info" type="subtree" subtree="$!") > >>>>>>> > >>>>>>> template(name="mse-all-info" type="list") { > >>>>>>> property(name="jsonmesg" outname="msg") } > >>>>>>> > >>>>>>> if ($fromhost-ip == '127.0.0.1' and $syslogfacility-text != 'kern') > >>>>>>> then > >>>>>>> { > >>>>>>> action(type="mmjsonparse") > >>>>>>> if $parsesuccess == "OK" then { > >>>>>>> set $!foo = $!mse.element_indices; > >>>>>>> > >>>>>>> set $!amph = "toad"; > >>>>>>> # foreach ($.index in $!mse.element_indices) do { > >>>>>>> # set $!amph = "turtle"; > >>>>>>> # } > >>>>>>> action(type="ommongodb" server="somemachine.somewhere.org" > >>>>>>> db="logs" collection="syslog" > >>>>>>> template="mse-structured-info") > >>>>>>> > >>>>>>> action(type="omfile" template="mse-all-info" > >>>>>>> file="/var/log/db_local_user_structured.log") > >>>>>>> } > >>>>>>> } > >>>>>>> > >>>>>>> $WorkDirectory /var/spool/rsyslog > >>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> rsyslog mailing list > >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>>>> http://www.rsyslog.com/professional-services/ > >>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > >>>>>>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > >>>>>>> of > >>>>>>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > >>>>>>> DON'T > >>>>>>> LIKE THAT. > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Regards, > >>>>>> Janmejay > >>>>>> http://codehunk.wordpress.com > >>>>>> _______________________________________________ > >>>>>> rsyslog mailing list > >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>>> http://www.rsyslog.com/professional-services/ > >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > >>>>>> WELL: > >>>>>> This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites > >>>>>> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > >>>>>> LIKE > >>>>>> THAT. > >>>>>> _______________________________________________ > >>>>>> rsyslog mailing list > >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>>> http://www.rsyslog.com/professional-services/ > >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > >>>>>> myriad > >>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > >>>>>> DON'T > >>>>>> LIKE THAT. > >>>>>> > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com/professional-services/ > >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > >>>>> myriad of > >>>>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > >>>>> DON'T > >>>>> LIKE THAT. > >>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Regards, > >>>> Janmejay > >>>> http://codehunk.wordpress.com > >>> > >>> > >>> > >>> > >>> -- > >>> Regards, > >>> Janmejay > >>> http://codehunk.wordpress.com > >> > >> > >> > >> > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > > LIKE THAT. > > > > -- > Regards, > Janmejay > http://codehunk.wordpress.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
