[rsyslog] mmnormalize thoughts

2015-02-03 Thread David Lang
as I'm spending a bunch of time making templates from cisco logs, a few thoughts on mmnormalize 1. It should probably set parsesuccess like mmjsonparse does 2. it would be useful to have something like char-to that accepted multiple characters as the termination pattern. thanks to the addition

Re: [rsyslog] mmnormalize thoughts

2015-02-04 Thread singh.janmejay
On Wed, Feb 4, 2015 at 7:17 AM, David Lang wrote: > as I'm spending a bunch of time making templates from cisco logs, a few > thoughts on mmnormalize > > 1. It should probably set parsesuccess like mmjsonparse does > This will be very useful. > > 2. it would be useful to have something like ch

Re: [rsyslog] mmnormalize thoughts

2015-02-04 Thread David Lang
On Wed, 4 Feb 2015, singh.janmejay wrote: On Wed, Feb 4, 2015 at 7:17 AM, David Lang wrote: as I'm spending a bunch of time making templates from cisco logs, a few thoughts on mmnormalize 1. It should probably set parsesuccess like mmjsonparse does This will be very useful. 2. it woul

Re: [rsyslog] mmnormalize thoughts

2015-02-04 Thread singh.janmejay
On Wed, Feb 4, 2015 at 6:22 PM, David Lang wrote: > On Wed, 4 Feb 2015, singh.janmejay wrote: > > On Wed, Feb 4, 2015 at 7:17 AM, David Lang wrote: >> >> as I'm spending a bunch of time making templates from cisco logs, a few >>> thoughts on mmnormalize >>> >>> 1. It should probably set parses

Re: [rsyslog] mmnormalize thoughts

2015-02-05 Thread David Lang
On Wed, 4 Feb 2015, singh.janmejay wrote: On Wed, Feb 4, 2015 at 6:22 PM, David Lang wrote: On Wed, 4 Feb 2015, singh.janmejay wrote: On Wed, Feb 4, 2015 at 7:17 AM, David Lang wrote: Field type 'descent' does this, but not exactly in the same way. does it? I understood it to just b

Re: [rsyslog] mmnormalize thoughts

2015-02-05 Thread singh.janmejay
It's going to be in the coming release, just master build for now. -- Regards, Janmejay PS: Please blame the typos in this mail on my phone's uncivilized soft keyboard sporting it's not-so-smart-assist technology. On Feb 6, 2015 6:37 AM, "David Lang" wrote: > On Wed, 4 Feb 2015, singh.janmejay

Re: [rsyslog] mmnormalize thoughts

2015-02-06 Thread David Lang
While I'm working to build packages of this to test with, what happens if you descend into a ruleset like the following rule=:%ip:ipv4%%last:rest% rule=:%ip:ipv4%/%port:number%%last:rest% will it work to find the match that has the least left in last? David Lang On Fri, 6 Feb 2015, singh.jan

Re: [rsyslog] mmnormalize thoughts

2015-03-11 Thread singh.janmejay
Tried re-ordering it? Put the one with /port first? Yes, rest must get atleast one char to succeed. I'll create some new tests without rest-capture (and see what fails). On Thu, Mar 12, 2015 at 1:09 AM, David Lang wrote: > I just upgraded to liblognorm 1.1.1 (unfortunantly I didn't get a chance

Re: [rsyslog] mmnormalize thoughts

2015-03-11 Thread David Lang
On Thu, 12 Mar 2015, singh.janmejay wrote: Tried re-ordering it? Put the one with /port first? no, lognorm rules are not supposed to be order dependent, so I didn't try that (especially after finding things failing to parse with rsyslog that worked manually) Yes, rest must get atleast one

Re: [rsyslog] mmnormalize thoughts

2015-03-11 Thread singh.janmejay
On Thu, Mar 12, 2015 at 9:19 AM, David Lang wrote: > On Thu, 12 Mar 2015, singh.janmejay wrote: > >> Tried re-ordering it? Put the one with /port first? > > > no, lognorm rules are not supposed to be order dependent, so I didn't try > that (especially after finding things failing to parse with rsy

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread David Lang
On Thu, 12 Mar 2015, singh.janmejay wrote: On Thu, Mar 12, 2015 at 9:19 AM, David Lang wrote: On Thu, 12 Mar 2015, singh.janmejay wrote: Tried re-ordering it? Put the one with /port first? no, lognorm rules are not supposed to be order dependent, so I didn't try that (especially after fin

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread David Lang
I just upgraded to liblognorm 1.1.1 (unfortunantly I didn't get a chance to compile it myself and test it earlier) I ran into two problems first, %last:rest% does not match if there is nothing left on the line i.e. a line that ends with an IP address will not match rule=:%ip:ipv4%%last:rest%

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread singh.janmejay
It never goes back up because if any other rule was going to match the current line, it would be a subtree of the current node (this is an invariant). It does try all sub-trees from any node before giving up. It first tries all field-nodes, then appropriate literal-node. In this case anything at

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread Chris Schafer
David, As far as docs go, when i went into documentation for liblognorm.com, i found http://www.liblognorm.com/files/manual/index.html Which includes string-to. That said, I know it's there because I put the function in, and if you have a suggestion as to better document the functions, that could

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread Rainer Gerhards
2015-02-04 2:47 GMT+01:00 David Lang : > as I'm spending a bunch of time making templates from cisco logs, a few > thoughts on mmnormalize > > 1. It should probably set parsesuccess like mmjsonparse does > > 2. it would be useful to have something like char-to that accepted > multiple characters a

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread Rainer Gerhards
2015-02-04 13:52 GMT+01:00 David Lang : > On Wed, 4 Feb 2015, singh.janmejay wrote: > > On Wed, Feb 4, 2015 at 7:17 AM, David Lang wrote: >> >> as I'm spending a bunch of time making templates from cisco logs, a few >>> thoughts on mmnormalize >>> >>> 1. It should probably set parsesuccess like

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread Rainer Gerhards
2015-03-12 12:50 GMT+01:00 Rainer Gerhards : > 2015-02-04 13:52 GMT+01:00 David Lang : > >> On Wed, 4 Feb 2015, singh.janmejay wrote: >> >> On Wed, Feb 4, 2015 at 7:17 AM, David Lang wrote: >>> >>> as I'm spending a bunch of time making templates from cisco logs, a few thoughts on mmnormal

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread Rainer Gerhards
2015-03-12 5:55 GMT+01:00 singh.janmejay : > On Thu, Mar 12, 2015 at 9:19 AM, David Lang wrote: > > On Thu, 12 Mar 2015, singh.janmejay wrote: > > > >> Tried re-ordering it? Put the one with /port first? > > > > > > no, lognorm rules are not supposed to be order dependent, so I didn't try > > tha

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread David Lang
On Thu, 12 Mar 2015, Rainer Gerhards wrote: 2015-03-12 5:55 GMT+01:00 singh.janmejay : On Thu, Mar 12, 2015 at 9:19 AM, David Lang wrote: On Thu, 12 Mar 2015, singh.janmejay wrote: Tried re-ordering it? Put the one with /port first? no, lognorm rules are not supposed to be order depende

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread Rainer Gerhards
2015-03-12 16:41 GMT+01:00 David Lang : > On Thu, 12 Mar 2015, Rainer Gerhards wrote: > > 2015-03-12 5:55 GMT+01:00 singh.janmejay : >> >> On Thu, Mar 12, 2015 at 9:19 AM, David Lang wrote: >>> On Thu, 12 Mar 2015, singh.janmejay wrote: Tried re-ordering it? Put the one with /po

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread singh.janmejay
On Thu, Mar 12, 2015 at 9:29 PM, Rainer Gerhards wrote: > 2015-03-12 16:41 GMT+01:00 David Lang : > >> On Thu, 12 Mar 2015, Rainer Gerhards wrote: >> >> 2015-03-12 5:55 GMT+01:00 singh.janmejay : >>> >>> On Thu, Mar 12, 2015 at 9:19 AM, David Lang wrote: > On Thu, 12 Mar 2015, singh.ja

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread Rainer Gerhards
2015-03-12 18:16 GMT+01:00 singh.janmejay : > On Thu, Mar 12, 2015 at 9:29 PM, Rainer Gerhards > wrote: > > 2015-03-12 16:41 GMT+01:00 David Lang : > > > >> On Thu, 12 Mar 2015, Rainer Gerhards wrote: > >> > >> 2015-03-12 5:55 GMT+01:00 singh.janmejay : > >>> > >>> On Thu, Mar 12, 2015 at 9:19

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread singh.janmejay
I haven't seen the reordering code yet, but the loading does preserve order. It still is deterministic, just that the criteria is rule-order (and it being applicable only for field-subtrees makes it slightly odd). On Thu, Mar 12, 2015 at 10:55 PM, Rainer Gerhards wrote: > 2015-03-12 18:16 GMT+01

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread David Lang
On Thu, 12 Mar 2015, David Lang wrote: On Thu, 12 Mar 2015, Rainer Gerhards wrote: 2015-03-12 5:55 GMT+01:00 singh.janmejay : On Thu, Mar 12, 2015 at 9:19 AM, David Lang wrote: On Thu, 12 Mar 2015, singh.janmejay wrote: Tried re-ordering it? Put the one with /port first? no, lognorm r

Re: [rsyslog] mmnormalize thoughts

2015-03-12 Thread David Lang
On Thu, 12 Mar 2015, singh.janmejay wrote: I haven't seen the reordering code yet, but the loading does preserve order. It still is deterministic, just that the criteria is rule-order (and it being applicable only for field-subtrees makes it slightly odd). this is definantly an issue looking

Re: [rsyslog] mmnormalize thoughts

2015-03-13 Thread Rainer Gerhards
2015-03-13 1:26 GMT+01:00 David Lang : > On Thu, 12 Mar 2015, singh.janmejay wrote: > > I haven't seen the reordering code yet, but the loading does preserve >> order. >> >> It still is deterministic, just that the criteria is rule-order (and >> it being applicable only for field-subtrees makes i