Another note - we should update Travis to include compiling docs. I did this in my Travis-test-docs branch. It does --enable-docs --enable-test bench --enable-debug --disable-valgrind so that we can tell if it errors out on the docs compile (which is how I caught the underline error). I can ff my master to this if you would like. On Tue, Jan 27, 2015 at 10:17 AM Chris Schafer <chrisp.scha...@gmail.com> wrote:
> I don't like 'or' because that is exactly what multiple rules with same > prefix do. > > It's a little more verbose, but much more readable too. > > > What if we could tag the same tag on two things, and if the first fails > the second works? > > %tag1:quoted-string%%tag1:word% > > On Tue Jan 27 2015 at 12:44:37 AM Rainer Gerhards < > rgerha...@hq.adiscon.com> wrote: > >> First of all, it's nice to see Chris patch and work. Much appreciated. I >> am >> still a bit on my CI-induced backlog, but hope to be able to finish that >> either today or tomorrow. Than at latest I can have a deeper look. Just a >> quick comment for now: >> >> 2015-01-27 7:56 GMT+01:00 singh.janmejay <singh.janme...@gmail.com>: >> >> > Sorry, I missed the documentation patch somehow. Thanks for correcting. >> > >> > I don't like 'or' because that is exactly what multiple rules with same >> > prefix do. >> > >> > >> I agree, I don't like the idea as well. >> >> >> > It's a little more verbose, but much more readable too. >> > >> > I don't think introducing multiple ways of doing the same thing is a >> good >> > idea. >> > >> > >> >> Without the ability to go into more specifics at the moment, it very much >> looks like I will be able to concentrate much more on liblognorm for quite >> a bit in the coming monthm, as part of a research project. If so, I will >> probably also re-think the rule language and there may be smarter ways to >> handle it than we currently do. At least for now, I think we should not do >> the "or" part. >> >> Rainer >> >> -- >> > Regards, >> > Janmejay >> > >> > PS: Please blame the typos in this mail on my phone's uncivilized soft >> > keyboard sporting it's not-so-smart-assist technology. >> > >> > On Jan 27, 2015 11:43 AM, "Chris Schafer" <chrisp.scha...@gmail.com> >> > wrote: >> > >> > > @Janmejay: >> > > I'll be honest - I <strike>don't</strike> didn't know if it'll handle >> > > escape sequences. I didn't test it earlier, just tested it now. >> Totally >> > > worked, woo! >> > > I did put in documentation - you can check the file. Actually >> included a >> > > little bugfix on existing documentation that kept it from compiling as >> > > well. >> > > I'm not against putting tests in at all, though I didn't because I >> didn't >> > > see any tests for the non-special functions, only regex and >> > tokenization. I >> > > can throw them in. What I did do is test this against a couple >> thousand >> > log >> > > lines that I actually needed to parse, just to make sure it worked. >> > > >> > > On Mon Jan 26 2015 at 10:01:21 PM Kendall Green < >> kendallar...@gmail.com> >> > > wrote: >> > > >> > > > I like the 'or' option, precisely for doing type check condition >> when a >> > > > whole lot of fields exists in records. This is currently cumbersome >> and >> > > > quickly becomes a daunting mess of a Cartesian Product set rule base >> > for >> > > > all the combination of fields that could have single values >> unquoted, >> > or >> > > > possibly quoted. Not to mention how this use case caries over to >> other >> > > > scenarios where an or operation would be invaluable to type casting. >> > > > >> > > > %<tag>:<type>:or:<type>% >> > > > could be very useful, not just to solve the issue of which behavior >> > > should >> > > > be default, as it would be set by the syntax. >> > > > >> > > > For example, if type quoted-string is set first, then should check >> > > without >> > > > quotes up to space. >> > > > Wouldn't the default be for what the type is, so with quoted-string, >> > then >> > > > it's quoted, unless an 'or' condition exists for an alternate >> expected >> > > data >> > > > type. >> > > > >> > > > With so very many fields in verbose messages, it is great to have a >> > > single >> > > > rule which would otherwise be an exponentially lengthy ruleset to >> > > > accommodate all the possible known type setting combinations. >> > > > %Description:quoted-string:or:word% >> > > > >> > > > An ''type:or:type" option could also be useful in other cases where >> > > > unpopulated fields exists with a default type value which doesn't >> match >> > > the >> > > > field when populated with specific typed value. >> > > > >> > > > %IP Address:ipv4:or:word% >> > > > The IP Address is provided, or a hyphen exists in the field when >> > > > unpopulated. In this scenario more specific literal matching would >> also >> > > be >> > > > nice option, which please correct me if literals already exists >> beyond >> > > > annotations. Having a char type match as char-sep somewhat >> resembles, >> > > where >> > > > field extraction only when the literal matches. The difference being >> > that >> > > > the literal would be matched for field value not just up to that >> > > position. >> > > > To give a more strict rule: >> > > > %IP Address:ipv4:or:char:\x2d% >> > > > >> > > > Similarly, it would be good to have string type, like described for >> the >> > > > purposed char type above, but for capturing the string literal >> instead >> > of >> > > > only the literal char. Rulebase could use string parse >> > > > enhancement with capture of literal string at specific field start >> > > > position within rulebase, since existing features could likely be >> used >> > > like >> > > > annotation fields. Additionally, please inform of any contributions >> for >> > > > the discussion regarding data type of fields to match string as a >> > > > string-to, as char-to / char-sep feature of char >> > > > separator on string, like the function, field($!path, >> string-or-char). >> > > So >> > > > please also elaborate on what has already been done for rulebase >> > matching >> > > > string literals. Thanks! >> > > > >> > > > -Kendall >> > > > >> > > > >> > > > >> > > > On Mon, Jan 26, 2015 at 5:49 PM, David Lang <da...@lang.hm> wrote: >> > > > >> > > > > I don't like the "or" option as I think it makes the rules harder >> to >> > > > read. >> > > > > unless you are doing this on a lot of fields in a line, just make >> a >> > new >> > > > > line with the different type. >> > > > > >> > > > > We need feedback from others, but at the very least I think making >> > this >> > > > an >> > > > > option to the standard quoted-string type would be better than a >> new >> > > type >> > > > > (the question is if this should be enabled by default or disabled >> by >> > > > > default) >> > > > > >> > > > > >> > > > > David Lang >> > > > > >> > > > > On Tue, 27 Jan 2015, Chris Schafer wrote: >> > > > > >> > > > > It comes back as a full fail. I thought about modifying that, >> but I >> > > > didn't >> > > > >> want to wreck anything currently in place. >> > > > >> A coworker of mine had a great idea for an "or" ability, going >> > > > >> %tag:or:quoted-string:word% where i attempts the first, and if >> that >> > > > fails, >> > > > >> goes to the second. However, that's not going to be easy, and I >> > wanted >> > > > to >> > > > >> push this change before you guys got too many commits ahead. >> > > > >> >> > > > >> On Mon Jan 26 2015 at 4:43:02 PM David Lang <da...@lang.hm> >> wrote: >> > > > >> >> > > > >> hmm, I'm wondering if we should do this for the normal quoted >> type? >> > > If >> > > > >>> you >> > > > >>> say >> > > > >>> quoted string and there isn't a quote does it just not match? >> > > > >>> >> > > > >>> David Lang >> > > > >>> >> > > > >>> On Tue, 27 Jan 2015, Chris Schafer wrote: >> > > > >>> >> > > > >>> This only handles " because that's what the current quoted >> string >> > > > does. >> > > > >>>> If it doesn't start with ", it implements the "word" >> functionality >> > > > >>>> >> > > > >>> (which I >> > > > >>> >> > > > >>>> shamelessly copied). The idea is to capture inputs where the >> > source >> > > > >>>> >> > > > >>> system >> > > > >>> >> > > > >>>> only quotes it if it contains a space, but leaves it unquoted >> > > > otherwise. >> > > > >>>> Example: >> > > > >>>> No data = - >> > > > >>>> One Word = word >> > > > >>>> Two words+ = "Two Words" >> > > > >>>> >> > > > >>>> The function should handle all three. >> > > > >>>> Chris >> > > > >>>> >> > > > >>>> On Mon Jan 26 2015 at 4:36:25 PM David Lang <da...@lang.hm> >> > wrote: >> > > > >>>> >> > > > >>>> does this handle embedded quotes in the string? and do you >> handle >> > > > >>>>> >> > > > >>>> strings >> > > > >>> >> > > > >>>> starting with ' and " or just one of them? >> > > > >>>>> >> > > > >>>>> David Lang >> > > > >>>>> >> > > > >>>>> On Tue, 27 Jan 2015, Chris Schafer wrote: >> > > > >>>>> >> > > > >>>>> Date: Tue, 27 Jan 2015 00:30:54 +0000 >> > > > >>>>>> From: Chris Schafer <chrisp.scha...@gmail.com> >> > > > >>>>>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com> >> > > > >>>>>> To: rsyslog@lists.adiscon.com >> > > > >>>>>> Subject: [rsyslog] New Pull request for liblognorm - >> additional >> > > > >>>>>> >> > > > >>>>> mmnormalize >> > > > >>>>> >> > > > >>>>>> functionality >> > > > >>>>>> >> > > > >>>>>> Just submitted the following pull request: >> > > > >>>>>> https://github.com/rsyslog/liblognorm/pull/20 >> > > > >>>>>> And I believe it could solve a lot of issues (at least, it >> > solves >> > > a >> > > > >>>>>> lot >> > > > >>>>>> >> > > > >>>>> of >> > > > >>>>> >> > > > >>>>>> mine) surrounding mmnormalize parsing in rsyslog. I'm looking >> > for >> > > > >>>>>> comments/issues/holy-crap-you-can't-code-what-are-you-doing, >> if >> > > you >> > > > >>>>>> >> > > > >>>>> guys >> > > > >>> >> > > > >>>> have any. This is my first time submitting a patch to a large >> > > project >> > > > >>>>>> >> > > > >>>>> (or >> > > > >>> >> > > > >>>> at least one where I didn't know the maintainer personally), >> so be >> > > > >>>>>> >> > > > >>>>> gentle >> > > > >>> >> > > > >>>> please :) >> > > > >>>>>> >> > > > >>>>>> Chris >> > > > >>>>>> _______________________________________________ >> > > > >>>>>> rsyslog mailing list >> > > > >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > >>>>>> http://www.rsyslog.com/professional-services/ >> > > > >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > > >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >> by >> > a >> > > > >>>>>> >> > > > >>>>> myriad >> > > > >>> >> > > > >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >> POST if >> > > you >> > > > >>>>> DON'T LIKE THAT. >> > > > >>>>> >> > > > >>>>>> >> > > > >>>>>> _______________________________________________ >> > > > >>>>> rsyslog mailing list >> > > > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > >>>>> http://www.rsyslog.com/professional-services/ >> > > > >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > > >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >> by a >> > > > >>>>> myriad >> > > > >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >> POST >> > if >> > > > you >> > > > >>>>> DON'T LIKE THAT. >> > > > >>>>> >> > > > >>>>> _______________________________________________ >> > > > >>>> rsyslog mailing list >> > > > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > >>>> http://www.rsyslog.com/professional-services/ >> > > > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > > >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >> by a >> > > > myriad >> > > > >>>> >> > > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >> if >> > > you >> > > > >>> DON'T LIKE THAT. >> > > > >>> >> > > > >>>> >> > > > >>>> _______________________________________________ >> > > > >>> rsyslog mailing list >> > > > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > >>> http://www.rsyslog.com/professional-services/ >> > > > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by >> a >> > > > myriad >> > > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >> if >> > > you >> > > > >>> DON'T LIKE THAT. >> > > > >>> >> > > > >>> _______________________________________________ >> > > > >> rsyslog mailing list >> > > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > >> http://www.rsyslog.com/professional-services/ >> > > > >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> > > myriad >> > > > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >> if >> > you >> > > > >> DON'T LIKE THAT. >> > > > >> >> > > > >> _______________________________________________ >> > > > > rsyslog mailing list >> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > > http://www.rsyslog.com/professional-services/ >> > > > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> > > myriad >> > > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> > you >> > > > > DON'T LIKE THAT. >> > > > > >> > > > _______________________________________________ >> > > > rsyslog mailing list >> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > http://www.rsyslog.com/professional-services/ >> > > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> > myriad >> > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you >> > > > DON'T LIKE THAT. >> > > > >> > > _______________________________________________ >> > > rsyslog mailing list >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > http://www.rsyslog.com/professional-services/ >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > > DON'T LIKE THAT. >> > > >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > DON'T LIKE THAT. >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
