Hello,
It's been a couple years, so have been reading back through the mailing
list on normalization threads, searching for updates for how to deal with
'-' hyphen place holders for nulls when normalizing logs with large number
of positional elements/values...
Any option to set a nullMarker="-",
On Wed, 28 Jan 2015, Kendall Green wrote:
David, thank you for clarifying this. This is all making more sense now
along with your responses on the other message threads explaining other
differences between rulebase and parser. Also, I must comment, that your
pdf, 'log filtering with rsyslog', ha
David, thank you for clarifying this. This is all making more sense now
along with your responses on the other message threads explaining other
differences between rulebase and parser. Also, I must comment, that your
pdf, 'log filtering with rsyslog', has been very helpful read,
http://www.sclug.or
Action level param doesn't necessarily mean it can be used only via
rsyslog. Im thinking of it similar to allowRegex param, which can be used
with liblognorm command-line or anyone else that uses it as a library.
It'll have to be something which is passed at the time of normalizing each
event thoug
On Wed, 28 Jan 2015, Kendall Green wrote:
But I understand the number of combinations / per rule in a rulebase, would
affect performance.
This is actually not the case (at least unless you use regex types)
This is the power of liblognorm and why it isn't just a 'typical' regex engine
liblogn
On Wed, 28 Jan 2015, singh.janmejay wrote:
I see what you are thinking of, but somethings that may be worth thinking
about before we decide:
- Does it make sense for users to pack unrelated samples in the same
rulebase?
There are 3 problems with this:
* The tree will become large, and bac
Folks, please bear with me. Right now, I can't comment in a way that makes
sense, as I need to check with some third parties. Once I have done that,
you'll understand. Please bear a day or maybe some few with me.
Rainer
2015-01-28 8:26 GMT+01:00 Kendall Green :
> >>Thoughts?
>
> Thanks for the e
>>Thoughts?
Thanks for the examples, as I understand what you mean about missing
fields. I just want to clarify, for what I've described, when a field is
not populated, the label still exists, so it's the same sample, which takes
on a different shape, as pattern changes depending on the field valu
I like the nullmarker idea a lot, since that's one of the most common
issue. Also, it solves it pretty efficiently. I think it needs to be in the
rulebase, or liblognorm is tied to being only a part of rsyslog.
Chris
On Tue Jan 27 2015 at 10:27:42 PM singh.janmejay
wrote:
> I see what you are th
I see what you are thinking of, but somethings that may be worth thinking
about before we decide:
- Does it make sense for users to pack unrelated samples in the same
rulebase?
There are 3 problems with this:
* The tree will become large, and back-tracking several unrelated
branches will b
Thank you, David, has done well describing the challenges with having so
many fields, and that a prefix would only provide an or condition up to the
first discrepancy. If there are a LOT of fields, and most can have a
different type default value. Or more specifically, if there was another
type for
I'm thinking that it needs to only apply to part of a ruleset. I can't see why
you would use the same rulebase with different values overall, but I can easily
see a rulebase that covers more than one type of logs needing different values
for the different types of logs.
remember that liblognor
I think action parameter is the most flexible place to have it at. Because
same rulebase can be used with different values.
Either module or rulebase level param will be less flexible compared to
this.
--
Regards,
Janmejay
PS: Please blame the typos in this mail on my phone's uncivilized soft
ke
On Wed, 28 Jan 2015, singh.janmejay wrote:
Ok, one way I can think of doing it: expose a parameter at action/module
level which turns on defaulting and picks a default string.
Eg.
action(type="mmnormalize " nullMarker="-")
Where nullMarker is a string (not a char).
Whenever a "-" is encount
Ok, one way I can think of doing it: expose a parameter at action/module
level which turns on defaulting and picks a default string.
Eg.
action(type="mmnormalize " nullMarker="-")
Where nullMarker is a string (not a char).
Whenever a "-" is encountered and a field is expected, it should skip t
On Wed, 28 Jan 2015, singh.janmejay wrote:
May be it'll be useful to discuss what you want to achieve with such
representations of sample. I mean if possible, take a few samples from your
existing rulebase which you think highlight the problem(s) you are facing.
I think the example is the Apac
; > > >
> > > > > >
> > > > > > David Lang
> > > > > >
> > > > > > On Tue, 27 Jan 2015, Chris Schafer wrote:
> > > > > >
> > > > > > It comes back as a full fail. I thought about modifying that,
but
> > I
fault or disabled
>> by
>> > > > > default)
>> > > > >
>> > > > >
>> > > > > David Lang
>> > > > >
>> > > > > On Tue, 27 Jan 2015, Chris Schafer wrote:
>> > > > >
>> > > > &g
:quoted-string:word% where i attempts the first, and if
> that
> > > > fails,
> > > > >> goes to the second. However, that's not going to be easy, and I
> > wanted
> > > > to
> > > > >> push this c
; > >> On Mon Jan 26 2015 at 4:43:02 PM David Lang wrote:
> > > >>
> > > >> hmm, I'm wondering if we should do this for the normal quoted type?
> > If
> > > >>> you
> > > >>> say
> > > >>> quoted string
" because that's what the current quoted string
> > does.
> > >>>> If it doesn't start with ", it implements the "word" functionality
> > >>>>
> > >>> (which I
> > >>>
> > >>>> shame
gt; (which I
> >>>
> >>>> shamelessly copied). The idea is to capture inputs where the source
> >>>>
> >>> system
> >>>
> >>>> only quotes it if it contains a space, but leaves it unquoted
> otherwise.
> >>>> Example:
&g
>>>> No data = -
>>>> One Word = word
>>>> Two words+ = "Two Words"
>>>>
>>>> The function should handle all three.
>>>> Chris
>>>>
>>>> On Mon Jan 26 2015 at 4:36:25 PM David Lang wrote:
>>&
it unquoted
>>>>> otherwise.
>>>>> Example:
>>>>> No data = -
>>>>> One Word = word
>>>>> Two words+ = "Two Words"
>>>>>
>>>>> The function should handle all three.
>>>>> Chris
>>>>
>>>> On Mon Jan 26 2015 at 4:36:25 PM David Lang wrote:
>>>>
>>>> does this handle embedded quotes in the string? and do you handle
>>>>>
>>>> strings
>>>
>>>> starting with ' and " or
id Lang
On Tue, 27 Jan 2015, Chris Schafer wrote:
Date: Tue, 27 Jan 2015 00:30:54 +0000
From: Chris Schafer
Reply-To: rsyslog-users
To: rsyslog@lists.adiscon.com
Subject: [rsyslog] New Pull request for liblognorm - additional
mmnormalize
functionality
Just submitted the following pull request:
just one of them?
> >>
> >> David Lang
> >>
> >> On Tue, 27 Jan 2015, Chris Schafer wrote:
> >>
> >>> Date: Tue, 27 Jan 2015 00:30:54 +
> >>> From: Chris Schafer
> >>> Reply-To: rsyslog-users
> >>> To:
: Tue, 27 Jan 2015 00:30:54 +
From: Chris Schafer
Reply-To: rsyslog-users
To: rsyslog@lists.adiscon.com
Subject: [rsyslog] New Pull request for liblognorm - additional
mmnormalize
functionality
Just submitted the following pull request:
https://github.com/rsyslog/liblognorm/pull/20
And I
th ' and " or just one of them?
>
> David Lang
>
> On Tue, 27 Jan 2015, Chris Schafer wrote:
>
> > Date: Tue, 27 Jan 2015 00:30:54 +
> > From: Chris Schafer
> > Reply-To: rsyslog-users
> > To: rsyslog@lists.adiscon.com
> > Subject: [rsyslog] Ne
om
Subject: [rsyslog] New Pull request for liblognorm - additional mmnormalize
functionality
Just submitted the following pull request:
https://github.com/rsyslog/liblognorm/pull/20
And I believe it could solve a lot of issues (at least, it solves a lot of
mine) surrounding mmnormalize parsing
Just submitted the following pull request:
https://github.com/rsyslog/liblognorm/pull/20
And I believe it could solve a lot of issues (at least, it solves a lot of
mine) surrounding mmnormalize parsing in rsyslog. I'm looking for
comments/issues/holy-crap-you-can't-code-what-are-you-doing, if you g
31 matches
Mail list logo
