2016-03-22 20:34 GMT+01:00 singh.janmejay <[email protected]>: > What about wrapDynamicObjects="on|off"? That is required regardless, > right? (if we want to preserve backward compatibility). Unless we > choose to change it anyway (which im fine with). > > @Rainer: what do you think about foreach handling object?
so far I could only manage to have a glimpse at the conversation. I admit that I am really concerned about all the extra complexity we introduce. And do so for what I think is a border-use case at best. There is a lot of work that should really be done in regard to variables and performance and I am unsure if this is the right time to do large extensions... I was happy with the dynstats as it looked like a very contained solution that did not affect much else. But now it looks we need to touch a big deal of code ... code that's not really ready for that... Maybe I get a different view when I find time to read all this more in-depth.... Rainer > > On Wed, Mar 23, 2016 at 1:01 AM, David Lang <[email protected]> wrote: >> On Wed, 23 Mar 2016, singh.janmejay wrote: >> >>> Foreach can only work with arrays as of now. It can't work with >>> objects (key-value pairs). So [{name: ... value: ...}, {..},...] is >>> the only format that will work as of now. >>> >>> We can enhance foreach to work with objects. >>> >>> I can make a flag available at dyn-stats bucket level, which can >>> control serialization format, but that would really be a hack. >>> >>> From single-responsibility pattern pov, impstats should be the only >>> component that decides how to layout data for user to see. >>> >>> How about this: >>> >>> impstats(format="json" wrapDynamicObjects="on"...)? >>> >>> It defaults to off, which keeps backward compatibility. >>> >>> So what do you guys think about: >>> - wrapDynamicObjects="on|off" >>> - generating [{name: a, value: 10}, {...} ...] vs. {a: 10, ...} >>> (foreach will handle the former out of the box, but later is concise, >>> readable and light-weight in addition to being more json-y. >>> - enhancing foreach to work with {a: 10, b: 20...} >> >> >> If we can enhance foreach to work with the concise format, I would rather >> wait for it instead of introducing the wrapping version. >> >> I'm thinking that foreach walks through arrays, rather than mixing concepts, >> a foreachobject that gives us a name and contents for a {} list of objects >> may be the right thing to do? >> >> foreach just returns a single object while foreachobject needs to return the >> object and name. >> >> although, if we ever get the ability to address arrays directly, being able >> to look at the array position would be the equivalent of the name. >> >> David Lang >> >> >> >>> On Wed, Mar 23, 2016 at 12:26 AM, David Lang <[email protected]> wrote: >>>> >>>> On Tue, 22 Mar 2016, singh.janmejay wrote: >>>> >>>>> How about this: we support a new flag in impstats which allows >>>>> json-formatted stats-line to optionally use >>>>> encapsulated/wrapped-layout? >>>>> >>>>> impstats(format="json" ...) generates >>>>> >>>>> >>>>> {"name":"msg_per_host","origin":"dynstats.bucket","z-scribe1r-b":80,"SWEB10":67} >>>>> >>>>> however, >>>>> >>>>> impstats(format="json/w" ...) generates {"header": >>>>> {"name":"msg_per_host","origin":"dynstats.bucket"}, "counters" : >>>>> {"z-scribe1r-b":80,"SWEB10":67}} >>>>> >>>>> This is relevant, the serilization format we use right now mixes >>>>> pre-defined keys with counter-names and it can affect regular static >>>>> counters too. >>>> >>>> >>>> >>>> with the existing pstats output, there is no ability for user-defined >>>> data >>>> to become a tag name, so there is no potential for ambiguity. but with >>>> dynastats, this is not a possibility, and the format we use should >>>> prevent >>>> problems. >>>> >>>> Also, just from a conceptual point of view, why should the bucket >>>> contents >>>> be at the same level as the bucket name? >>>> >>>> other than backwards compatibility, what advantage is there of the >>>> current >>>> version in JSON? the documentation uses the plain text equivalant, which >>>> is >>>> perfectly legitimate because there is an order to the line, and after you >>>> get past the name and origin, everything else on the line is name-value >>>> pairs of counters, again, no ambiguity. >>>> >>>> But with JSON, I don't believe that you can depend on tools maintaining >>>> (or >>>> even identifying) the order of the elements, and if you have multiple >>>> elements with the same name, it's implementation dependent as to which >>>> one >>>> will be seen. >>>> >>>> So purely from a correctness and defensive programming point of view, I >>>> think the current JSON serialization should be changed, with the old >>>> format >>>> no longer being an option. >>>> >>>> >>>> >>>> >>>> As to the details of the new format >>>> >>>> What I'm wanting to do with the counters is something like >>>> >>>> if $!origin == "dynstats.bucket" then { >>>> foreach $.tag $!counters { >>>> /var/log/stats;format >>>> } >>>> } >>>> >>>> to output one line per counter. >>>> >>>> I'm very flexible in how to do this, but I would much rather be able to >>>> do >>>> this inside rsyslog than have to serialize things to an external script, >>>> have it parse the json and process it. >>>> >>>> my initial thinking was just do >>>> >>>> counters: [ "z-scribe1r-b":80,"SWEB10":67 ] >>>> >>>> but as I'm typing this, I realize that doesn't work as I don't have a way >>>> to >>>> break $.tag down to reference the name and the value. >>>> >>>> I'd hate to have to do something like >>>> >>>> counters: [{"name":"z-scribe1r-b","value":80 >>>> },{"name":"SWEB10","value":67}] >>>> >>>> this mirrors the misuse of XML that gives it such a horrible reputation. >>>> But >>>> unless we introduce some new function to rsyslog to break things down, I >>>> don't see a better way. If we do need to do something like this, I sure >>>> would not want to make it the default JSON, which would result in two >>>> different formats. I hate the idea of starting to have different formats >>>> because of subtypes of data (what is someone wants the cee version of >>>> this >>>> for example, you start to have orthoginal format decisions) >>>> >>>> >>>> David Lang >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of >>>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >>>> LIKE THAT. >>> >>> >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. > > > > -- > Regards, > Janmejay > http://codehunk.wordpress.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
