Thx for the patches. Just FYI: I am deep inside complex work, will look at merging later the week, if things go really bad early next week.
Rainer 2014-10-21 9:32 GMT+02:00 singh.janmejay <[email protected]>: > On Tue, Oct 14, 2014 at 10:06 AM, singh.janmejay <[email protected] > > > wrote: > > > > > > > On Tue, Oct 14, 2014 at 9:23 AM, David Lang <[email protected]> wrote: > > > >> On Tue, 14 Oct 2014, singh.janmejay wrote: > >> > >> Agree. > >>> > >>> I was not thinking of it like that. Will make the necessary changes. > >>> > >>> Another question, do you want me to keep the template support and add > >>> variable support? > >>> > >>> or should I replace it completely with variable support(im thinking > >>> replacing makes more sense)? > >>> > >> > >> I think replacing makes more sense. Since we have the ability to set a > >> variable equal to the output of a template, not having a template in the > >> action doesn't prevent you from doing anything, you just have to do it > in > >> two steps instead of one. > >> > >> > > Sounds good, will rework it in the next patch. > > > > > >> > >> David Lang > >> > >> > >> > >>> > >>> > >>> On Tue, Oct 14, 2014 at 8:53 AM, David Lang <[email protected]> wrote: > >>> > >>> On Tue, 14 Oct 2014, singh.janmejay wrote: > >>>> > >>>> Yep, sounds good. > >>>> > >>>>> > >>>>> The only concern I can think of: its different from how other actions > >>>>> work. > >>>>> > >>>>> But that shouldn't be a problem if we make other actions accept > values > >>>>> too(and possibly deprecate usage of template in favour of variable + > >>>>> exec_template). Something to think about, I guess. > >>>>> > >>>>> I'll make the change and send it back for review. > >>>>> > >>>>> > >>>> it's the difference between input and output. > >>>> > >>>> For other actions, the template is formatting the output of the > module, > >>>> and you almost never are going to be sending just a single variable as > >>>> the > >>>> output (I can't think of any case, other than possibly rawmsg) > >>>> > >>>> In this case we are defining the input to the message modification > >>>> module, > >>>> msg, rawmsg, or an already defined variable. > >>>> > >>>> viewing it this way, I think it makes perfect sense that it's not a > >>>> template, but let's see what others think. > >>>> > >>>> David Lang > >>>> > >>>> > >>>> > >>>> On Tue, Oct 14, 2014 at 7:59 AM, David Lang <[email protected]> wrote: > >>>>> > >>>>> As an efficiency thing, I would suggest having it get a variable > >>>>> instead > >>>>> > >>>>>> of a templage. You can assign a template to a variable if you need > to, > >>>>>> but > >>>>>> if you don't (because a prior step already parsed out the data to a > >>>>>> variable) you skip the time needed to go through the template > engine. > >>>>>> > >>>>>> I expect that most of the time you are going to have the part you > >>>>>> want to > >>>>>> normalize as the contents of a variable being passed in via JSON or > a > >>>>>> structured data field, not as something that you would need the full > >>>>>> power > >>>>>> and complexity of the template string generator to get at. > >>>>>> > >>>>>> The template string handler is slower than you would think. When we > >>>>>> added > >>>>>> the ability to have string generation modules in C instead of using > >>>>>> the > >>>>>> template engine, it made a noticable difference in the total > >>>>>> throughput > >>>>>> of > >>>>>> rsyslog, and if we are just going to be starting from a string to > >>>>>> begin > >>>>>> with, running it through the template engine is pure overhead. > >>>>>> > >>>>>> Thanks for doing this sort of thing. I was wanting to get > mmnormalize > >>>>>> and > >>>>>> mmjsonparse to work on arbitrary strings instead of just $msg, but > >>>>>> hadn't > >>>>>> had time to deal with it. > >>>>>> > >>>>>> David Lang > >>>>>> > >>>>>> On Tue, 14 Oct 2014, singh.janmejay wrote: > >>>>>> > >>>>>> Date: Tue, 14 Oct 2014 01:03:44 +0530 > >>>>>> > >>>>>> From: singh.janmejay <[email protected]> > >>>>>>> Reply-To: rsyslog-users <[email protected]> > >>>>>>> To: [email protected] > >>>>>>> Subject: [rsyslog] Patch to allow choice of templates in > mmnormalize > >>>>>>> > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> As of now mmnormalize supports only raw-msg or msg via the boolean > >>>>>>> flag > >>>>>>> 'useRawMsg'. This patch allows using template="template_name" with > >>>>>>> mmnormalize just like a few other actions(such as omfwd, > >>>>>>> omelasticsearch > >>>>>>> etc) do. > >>>>>>> > >>>>>>> It expects a param called 'template' following the convention in > >>>>>>> other > >>>>>>> action-types. And when both useRawMsg and template params are set, > it > >>>>>>> prints a warning that template will be ignored and uses > raw-message > >>>>>>> instead. > >>>>>>> > >>>>>>> The patch is called 0003... because its the 3rd patch in my > >>>>>>> local-repo. > >>>>>>> But > >>>>>>> since it doesn't have any files common with previous patches, it > >>>>>>> should > >>>>>>> apply just fine on any recent commit. > >>>>>>> > >>>>>>> Please let me know if the patch looks good for merging, I'll update > >>>>>>> the > >>>>>>> documentation. > >>>>>>> > >>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> > >>>>>>> rsyslog mailing list > >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>>> http://www.rsyslog.com/professional-services/ > >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > >>>>>> myriad > >>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > >>>>>> DON'T LIKE THAT. > >>>>>> > >>>>>> > >>>>>> > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com/professional-services/ > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > >>>> DON'T LIKE THAT. > >>>> > >>>> > >>> > >>> > >>> > >>> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com/professional-services/ > >> What's up with rsyslog? Follow https://twitter.com/rgerhards > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > >> DON'T LIKE THAT. > >> > > > > > > > > -- > > Regards, > > Janmejay > > http://codehunk.wordpress.com > > > > > Hi, > > Sorry for the loss the continuity. > > Here are the patches for mmnormalize variable support. > > I have left patch 0001-... as is (it is the same old patch which enables > template support) > > The patch 0002-... follows that and replaces support for templates with > variable. > > My mmnormalize invocation now looks like this: > > action(type="mmnormalize" ruleBase="/tmp/foo.rulebase" variable="$.foo") > > Variable can be any property or a local/global variable etc. It uses > MsgGetProp, so it should support everything. > > -- > Regards, > Janmejay > http://codehunk.wordpress.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
