Rainer: is liblognorm doc also at github using rst format? On Wednesday, December 21, 2016, matthew.gaetano <matthew.gaet...@gmx.ca> wrote:
> Alright! Almost out the door for the holiday's and swore i would come back > to > this before I left in the event anyone stumbled across it. Note that this > is > intended to primary provide information on using the current iptables data > type in version 2 of liblognorm. > > For reference tests were conducted on CentOS Linux Release 7.2.1511 with > liblognorm5-2.0.2-1 and liblognorm5-utils-2.0.2-1 installed from the > Addison > repository. > > > > The documentation for liblognorm is significantly lacking for its current > version and the iptables section demonstrates that severely. In version 2.x > the data type "iptables" no longer exists, instead it has been replaced by > its predecessor "v2-iptables". As per the documentation "v2-iptables" works > exactly as stated: > > "Name=value pairs, separated by spaces, as in Netfilter log messages. Name > of the selector is not used; names from the line are used instead. This > selector always matches everything till end of the line. Cannot match zero > characters." > > However there are some additional restrictions that are not mentioned. > These > can be seen when checking out the test files under > "https://github.com/rsyslog/liblognorm/blob/master/tests"; and they are as > follows: > > 1. Key names can only be uppercase > 2. Key name can only contain alphabetic characters (no special characters > or > numbers) > 3. All Key-value pairs must be separated by a single space > 4. A minimum of two pairs are required; a single key-value pair will not > work > 5. value will contain all characters until the next space, this includes > any > notation like brackets that are not necessarily part of said value. > > > A quick working test: > test.rb: > version=2 > rule=:%field:v2-iptables% > > lognormalizer: > echo 'KA=test1 KB="test2"' | lognormalizer -e json -r test.rb > > > > > It appears that iptables was built for a specific use case, aka Netfilter, > and then never adapted to be more applicable to key-value's. Other data > types exist like "cee" and "cef" who more accurately act as expected, > though > when i started i simple assumed that they in themselves made use of the > iptables data type. > > The problem now is that other formats that utilize key-value pairs cant be > parsed, especially as most of the time they do not come in any particular > order. One could possible use the "alternative" and "repeat" types to solve > the problem but this A) seems incredibly inefficient and B) is limited to > know key names. The more arbitrary the key-value output the harder it > becomes. CEF is very good example of this, though, lucky we have a parser > for it. The problem is when we start to look similar formats like MEF, > LEEF, > or Fortinet logs. > > CEF and CEE data types are locked down to a very specfic header that can > not > be altered. In terms of CEF this also problematic because the RFC3164 > parser > will often extract the "CEF:" part of the message as the syslog tag thus > making it difficult (not impossible) to use the "CEF" data type. > > *Note: a new CEF version is expected sometime next year > > > IMO it seems that the CEE and CEF specific data types shouldn't even exist, > or if they do, then only as an abstraction; something like: > > type=@CEF:CEF:0|%vendor:chart-to:|%|%product:chart-to:|%|% > version:chart-to:|%|%signature:chart-to:|%|%name: > chart-to:|%|%severity:chart-to:|%|%extensions:iptables% > > It would also be advantages to assume that the key-value pair might not > always be separated by a single space, or have values encased in standard > brackets (ie. ", '). Having advanced options to alter those would increase > its flexibility. > > I also do not necessarily agree with the the whole "until end of line" > portion. I see that as something that should only ever apply when using the > data type "rest" as it is feasible that a portion contain key-values (or > other type) but does is proceed by something else. > > At present I do not have any examples but could probably dig up some vendor > sources/docs that might. I have seen in the past custom client applications > do awkward things as well as seeing key-value pair submitted encased in > other formats; tho i cant really supply those as examples :( > > > Thanks > > ~Regards > > > > > > ----- > ~Regards > > Matthew Gaetano > -- > View this message in context: http://rsyslog-users.1305293. > n2.nabble.com/Liblognorm-Usage-of-field-type-iptables- > tp7591441p7592021.html > Sent from the rsyslog-users mailing list archive at Nabble.com. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
