Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread Rainer Gerhards
Folks, please bear with me. Right now, I can't comment in a way that makes sense, as I need to check with some third parties. Once I have done that, you'll understand. Please bear a day or maybe some few with me. Rainer 2015-01-28 8:26 GMT+01:00 Kendall Green : > >>Thoughts? > > Thanks for the e

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread Kendall Green
>>Thoughts? Thanks for the examples, as I understand what you mean about missing fields. I just want to clarify, for what I've described, when a field is not populated, the label still exists, so it's the same sample, which takes on a different shape, as pattern changes depending on the field valu

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread Chris Schafer
I like the nullmarker idea a lot, since that's one of the most common issue. Also, it solves it pretty efficiently. I think it needs to be in the rulebase, or liblognorm is tied to being only a part of rsyslog. Chris On Tue Jan 27 2015 at 10:27:42 PM singh.janmejay wrote: > I see what you are th

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread singh.janmejay
I see what you are thinking of, but somethings that may be worth thinking about before we decide: - Does it make sense for users to pack unrelated samples in the same rulebase? There are 3 problems with this: * The tree will become large, and back-tracking several unrelated branches will b

Re: [rsyslog] rsyslog normalization

2015-01-27 Thread Kendall Green
Thank you everyone for your thoughtful responses. After contemplating normalization and recent contributions that will be great in helping with much needed to-string rule base type, it looks like these problems will soon be resolved. The new feature provides functionality to mmnormalize rulebase,

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread Kendall Green
Thank you, David, has done well describing the challenges with having so many fields, and that a prefix would only provide an or condition up to the first discrepancy. If there are a LOT of fields, and most can have a different type default value. Or more specifically, if there was another type for

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread David Lang
I'm thinking that it needs to only apply to part of a ruleset. I can't see why you would use the same rulebase with different values overall, but I can easily see a rulebase that covers more than one type of logs needing different values for the different types of logs. remember that liblognor

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread singh.janmejay
I think action parameter is the most flexible place to have it at. Because same rulebase can be used with different values. Either module or rulebase level param will be less flexible compared to this. -- Regards, Janmejay PS: Please blame the typos in this mail on my phone's uncivilized soft ke

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread David Lang
On Wed, 28 Jan 2015, singh.janmejay wrote: Ok, one way I can think of doing it: expose a parameter at action/module level which turns on defaulting and picks a default string. Eg. action(type="mmnormalize " nullMarker="-") Where nullMarker is a string (not a char). Whenever a "-" is encount

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread singh.janmejay
Ok, one way I can think of doing it: expose a parameter at action/module level which turns on defaulting and picks a default string. Eg. action(type="mmnormalize " nullMarker="-") Where nullMarker is a string (not a char). Whenever a "-" is encountered and a field is expected, it should skip t

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread David Lang
On Wed, 28 Jan 2015, singh.janmejay wrote: May be it'll be useful to discuss what you want to achieve with such representations of sample. I mean if possible, take a few samples from your existing rulebase which you think highlight the problem(s) you are facing. I think the example is the Apac

[rsyslog] re_extract does not support all regex

2015-01-27 Thread Boylan, James
list). 1050.149212163:imptcp.c : Parser 'rsyslog.rfc5424' returned -2160 1050.149219838:imptcp.c : Message will now be parsed by the legacy syslog parser (one size fits all... ;)). 1050.149228720:currAppLog.main_Q:Reg/w0: rainerscript: var 1: ' appname|20.409|0

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread singh.janmejay
-- Regards, Janmejay PS: Please blame the typos in this mail on my phone's uncivilized soft keyboard sporting it's not-so-smart-assist technology. On Jan 27, 2015 11:47 PM, "Chris Schafer" wrote: > > I don't like 'or' because that is exactly what multiple rules with same > prefix do. > > It's a

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread Chris Schafer
Another note - we should update Travis to include compiling docs. I did this in my Travis-test-docs branch. It does --enable-docs --enable-test bench --enable-debug --disable-valgrind so that we can tell if it errors out on the docs compile (which is how I caught the underline error). I can ff my m

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread Chris Schafer
I don't like 'or' because that is exactly what multiple rules with same prefix do. It's a little more verbose, but much more readable too. What if we could tag the same tag on two things, and if the first fails the second works? %tag1:quoted-string%%tag1:word% On Tue Jan 27 2015 at 12:44:37 AM

Re: [rsyslog] experiencing memory leak in multiple rsyslog versions

2015-01-27 Thread Koral Ilgun
Hi Rainer, no need for apology, as I have also been away, and only now am able to write a follow-up on this. I will try the latest rsyslog (8.7.0) as you suggested, and will also try to get more info out of valgrind if I can. Thanks and regards, Koral -

Re: [rsyslog] Configuration errors in tests not detected?

2015-01-27 Thread Thomas D.
Hello, Rainer wrote: > HOWEVER, it is correct that the testbench does not throw an error, > because the config is to NOT abort on errors. So in that regard it > works like it should. ...wait, to be clear: 1) It is one thing to run *all* the tests and don't break on the first error. That would be

[rsyslog] TLS handshake issuw

2015-01-27 Thread Muhammad Asif
Hi! I have created certificates for server and clients multiple time but the following error is not easing my like. Jan 27 21:35:06 demo rsyslogd-2353: imrelp[20514]: error 'TLS handshake failed [gnutls error -15: An unexpected TLS packet was received.]', object 'lstn 20514: conn to clt IP/syste

Re: [rsyslog] Configuration errors in tests not detected?

2015-01-27 Thread Rainer Gerhards
Yup, these are errors and I have asked Tim (whom I am currently mentoring towards rsyslog development in the long term) to fix them. HOWEVER, it is correct that the testbench does not throw an error, because the config is to NOT abort on errors. So in that regard it works like it should. Tim will

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

2015-01-27 Thread Rainer Gerhards
First of all, it's nice to see Chris patch and work. Much appreciated. I am still a bit on my CI-induced backlog, but hope to be able to finish that either today or tomorrow. Than at latest I can have a deeper look. Just a quick comment for now: 2015-01-27 7:56 GMT+01:00 singh.janmejay : > Sorry,