Re: [rsyslog] saving liblognorm literals
El 21/12/16 a las 17:46, Rainer Gerhards escribió:
see https://github.com/rsyslog/liblognorm/pull/238
wow! that was fast!
2016-12-21 16:54 GMT+01:00 David Lang :
Can you explain your ruleset where you need to store literal as a
value in the json?
I think the original thinking was that since this is a fixed value,
storing it as a variable doesn't help.
slapd [1] messages were processed in grok with the following expression:
MYEXPR ^(.*?)conn=(?\d+) (fd=(?\d+)|op=(?\d+))
((?ACCEPT) from IP=%{IPORHOST:ip}|(%{WORD} )?RESULT
(?.*)|%{WORD} (?attr=.*)|%{WORD:cmd}( (?.*))?)
The relevant part of that is that messages can have 5 formats:
ACCEPT from IP=...
FOO RESULT... # FOO can be different words
RESULT...
BAR attr=... # BAR can be different words
FOOBAR anything # FOOBAR can be different words
# anything is the rest (not one of the above)
And with that grok we were indexing them "properly"...
[1] http://www.openldap.org/software/man.cgi?query=slapd
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
Re: [rsyslog] saving liblognorm literals
Storing a literal value would be dependent on the desired output (which would also then be dependent on what the output is being used by). As an example i have lots of rules that represent different types of action events from Cisco. Just looking at the built and teardown sessions shows a wide range of variances in meaning and format, often complicated by optional segments. So when i write the rule i used the literal Build or Teardown to distinguish the differences; and while i could use prefixes to do this, its gets harder to do as prefix only a single layer and I am already using them to denote the difference in overarching types of sources/logs. In my output i want my users to be able to search using a field called action and that action is denoted by "Built" or "Teardown". With out the ability to capture literals the only way to accommodate is to use annotations. Since annotations require tags things can quickly becomes convoluted and messy the larger they become; more so when the literal might be longer, possibly presenting conflicts and undesired results. Repetition is also annoying, why do something twice? Side note: Prefix's should support the tags function and apply that tag to any rule that applies to said prefix. At the moment i have to manually added product/model/version tags to every applicable rule. Thanks ~Regards - ~Regards Matthew Gaetano -- View this message in context: http://rsyslog-users.1305293.n2.nabble.com/saving-liblognorm-literals-tp7591994p7592011.html Sent from the rsyslog-users mailing list archive at Nabble.com. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] saving liblognorm literals
see https://github.com/rsyslog/liblognorm/pull/238 Rainer 2016-12-21 17:22 GMT+01:00 Rainer Gerhards : > There is something fishy. I did a quick test, and it looks like the > optimizer combines literal parser where it should not do... > > Rainer > > 2016-12-21 17:05 GMT+01:00 Rainer Gerhards : > >> IMHO storing a literal should work, as long as a name is assigned to the >> literal parser. >> >> Rainer >> >> 2016-12-21 16:54 GMT+01:00 David Lang : >> >>> On Wed, 21 Dec 2016, mostolog--- via rsyslog wrote: >>> >>> Does this means we shouldn't store the literal as variable? What If we need to? >>> >>> Can you explain your ruleset where you need to store literal as a value >>> in the json? >>> >>> I think the original thinking was that since this is a fixed value, >>> storing it as a variable doesn't help. >>> >>> David Lang >>> >>> ___ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >> >> > ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] saving liblognorm literals
There is something fishy. I did a quick test, and it looks like the optimizer combines literal parser where it should not do... Rainer 2016-12-21 17:05 GMT+01:00 Rainer Gerhards : > IMHO storing a literal should work, as long as a name is assigned to the > literal parser. > > Rainer > > 2016-12-21 16:54 GMT+01:00 David Lang : > >> On Wed, 21 Dec 2016, mostolog--- via rsyslog wrote: >> >> Does this means we shouldn't store the literal as variable? What If we >>> need to? >>> >> >> Can you explain your ruleset where you need to store literal as a value >> in the json? >> >> I think the original thinking was that since this is a fixed value, >> storing it as a variable doesn't help. >> >> David Lang >> >> ___ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] saving liblognorm literals
IMHO storing a literal should work, as long as a name is assigned to the literal parser. Rainer 2016-12-21 16:54 GMT+01:00 David Lang : > On Wed, 21 Dec 2016, mostolog--- via rsyslog wrote: > > Does this means we shouldn't store the literal as variable? What If we >> need to? >> > > Can you explain your ruleset where you need to store literal as a value in > the json? > > I think the original thinking was that since this is a fixed value, > storing it as a variable doesn't help. > > David Lang > > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] saving liblognorm literals
On Wed, 21 Dec 2016, mostolog--- via rsyslog wrote:
Hi
(Moving from grok to liblognorm) We are trying to store a literal as a
variable, as we were doing using grok. eg: (?ACCEPT)
We aren't sure if this is correct:
rule=:%{"type":"literal", "text":"a", "name":"var"}%
As stated in
http://www.liblognorm.com/files/manual/configuration.html#full-json-format
/*the literal text shall not be stored inside an output variable*;
for this reason no name attribute is given (we could also have used
"name":"-" which achives the same effect but is more verbose)./
Does this means we shouldn't store the literal as variable? What If we
need to?
you can set a variable to a fixed value as an ammend, or try to use word instead
of literal for the item (if this doesn't make the match ambiguous)
or, depending on what you are needing the literal for, you may be able to set it
as a tag.
David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
Re: [rsyslog] saving liblognorm literals
On Wed, 21 Dec 2016, mostolog--- via rsyslog wrote: Does this means we shouldn't store the literal as variable? What If we need to? Can you explain your ruleset where you need to store literal as a value in the json? I think the original thinking was that since this is a fixed value, storing it as a variable doesn't help. David Lang ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
[rsyslog] saving liblognorm literals
Hi
(Moving from grok to liblognorm) We are trying to store a literal as a
variable, as we were doing using grok. eg: (?ACCEPT)
We aren't sure if this is correct:
rule=:%{"type":"literal", "text":"a", "name":"var"}%
As stated in
http://www.liblognorm.com/files/manual/configuration.html#full-json-format
/*the literal text shall not be stored inside an output variable*;
for this reason no name attribute is given (we could also have used
"name":"-" which achives the same effect but is more verbose)./
Does this means we shouldn't store the literal as variable? What If we
need to?
/*using the “literal” parser in JSON should be avoided currently*;
the experimental version does have some rough edges where conflicts
in literal processing will not be properly handled. This should not
be an issue in “closed environments”, like “repeat”, where no such
conflict can occur./
Is that still recommended? How could I "parse and store" a literal using
condensed format?
Regards
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
