Alright! Almost out the door for the holiday's and swore i would come back to this before I left in the event anyone stumbled across it. Note that this is intended to primary provide information on using the current iptables data type in version 2 of liblognorm.
For reference tests were conducted on CentOS Linux Release 7.2.1511 with liblognorm5-2.0.2-1 and liblognorm5-utils-2.0.2-1 installed from the Addison repository. The documentation for liblognorm is significantly lacking for its current version and the iptables section demonstrates that severely. In version 2.x the data type "iptables" no longer exists, instead it has been replaced by its predecessor "v2-iptables". As per the documentation "v2-iptables" works exactly as stated: "Name=value pairs, separated by spaces, as in Netfilter log messages. Name of the selector is not used; names from the line are used instead. This selector always matches everything till end of the line. Cannot match zero characters." However there are some additional restrictions that are not mentioned. These can be seen when checking out the test files under "https://github.com/rsyslog/liblognorm/blob/master/tests"; and they are as follows: 1. Key names can only be uppercase 2. Key name can only contain alphabetic characters (no special characters or numbers) 3. All Key-value pairs must be separated by a single space 4. A minimum of two pairs are required; a single key-value pair will not work 5. value will contain all characters until the next space, this includes any notation like brackets that are not necessarily part of said value. A quick working test: test.rb: version=2 rule=:%field:v2-iptables% lognormalizer: echo 'KA=test1 KB="test2"' | lognormalizer -e json -r test.rb It appears that iptables was built for a specific use case, aka Netfilter, and then never adapted to be more applicable to key-value's. Other data types exist like "cee" and "cef" who more accurately act as expected, though when i started i simple assumed that they in themselves made use of the iptables data type. The problem now is that other formats that utilize key-value pairs cant be parsed, especially as most of the time they do not come in any particular order. One could possible use the "alternative" and "repeat" types to solve the problem but this A) seems incredibly inefficient and B) is limited to know key names. The more arbitrary the key-value output the harder it becomes. CEF is very good example of this, though, lucky we have a parser for it. The problem is when we start to look similar formats like MEF, LEEF, or Fortinet logs. CEF and CEE data types are locked down to a very specfic header that can not be altered. In terms of CEF this also problematic because the RFC3164 parser will often extract the "CEF:" part of the message as the syslog tag thus making it difficult (not impossible) to use the "CEF" data type. *Note: a new CEF version is expected sometime next year IMO it seems that the CEE and CEF specific data types shouldn't even exist, or if they do, then only as an abstraction; something like: type=@CEF:CEF:0|%vendor:chart-to:|%|%product:chart-to:|%|%version:chart-to:|%|%signature:chart-to:|%|%name:chart-to:|%|%severity:chart-to:|%|%extensions:iptables% It would also be advantages to assume that the key-value pair might not always be separated by a single space, or have values encased in standard brackets (ie. ", '). Having advanced options to alter those would increase its flexibility. I also do not necessarily agree with the the whole "until end of line" portion. I see that as something that should only ever apply when using the data type "rest" as it is feasible that a portion contain key-values (or other type) but does is proceed by something else. At present I do not have any examples but could probably dig up some vendor sources/docs that might. I have seen in the past custom client applications do awkward things as well as seeing key-value pair submitted encased in other formats; tho i cant really supply those as examples :( Thanks ~Regards ----- ~Regards Matthew Gaetano -- View this message in context: http://rsyslog-users.1305293.n2.nabble.com/Liblognorm-Usage-of-field-type-iptables-tp7591441p7592021.html Sent from the rsyslog-users mailing list archive at Nabble.com. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
