I'm still trying to reproduce/understand what is happening and building
a test case for the github issue if needed.
Consider the following HTTP access lines:
127.0.0.1 - - [17/Mar/2016:18:15:06 +0100] "GET /redacted HTTP/1.1"
200 59506
127.0.0.1 - - [17/Mar/2016:18:15:24 +0100] "OPTIONS / HTTP/1.1" 403
205 "-" "-"
And the following rule:
# This is just access_log. Perhaps literal is more efficient than
whitespace?
type=@apache_common:%[
{"type":"ipv4", "name":"ip"},
{"type":"whitespace"},
{"type":"word", "name":"ident"},
{"type":"whitespace"},
{"type":"word", "name":"user"},
{"type":"literal", "text":" ["},
{"type":"char-to", "name":"date", "extradata":"]"},
{"type":"literal", "text":"] \""},
{"type":"word", "name":"method"},
{"type":"whitespace"},
{"type":"char-to", "name":"request", "extradata":" "},
{"type":"literal", "text":" HTTP/"},
{"type":"float", "name":"httpversion"},
{"type":"literal", "text":"\""},
{"type":"whitespace"},
{"type":"number", "name":"response"},
{"type":"whitespace"},
{"type":"word", "name":"bytes"}
]%
#AFAIK this should accept null or apache combined log fields
type=@apache_combined:-
type=@apache_combined:%[
{"type":"whitespace"},
{"type":"quoted-string","name":"referrer"},
{"type":"whitespace"},
{"type":"quoted-string","name":"useragent"}
]%
rule=access_combined:%[
{"type":"@apache_common", "name":"."}
]%
*As expected*, the first line matches and the second doesn't:
{ "bytes": "59506", "response": "200", "httpversion": "1.1",
"request": "\/redacted", "method": "GET", "date":
"17\/Mar\/2016:18:15:06 +0100", "user": "-", "ident": "-", "ip":
"127.0.0.1" }
{ "originalmsg": "127.0.0.1 - - [17\/Mar\/2016:18:15:24 +0100]
\"OPTIONS \/ HTTP\/1.1\" 403 205 \"-\" \"-", "unparsed-data": "
\"-\" \"-" }
But if we try:
rule=access_combined:%[
{"type":"@apache_common", "name":"."},
{"type":"@apache_combined","name":"."}
]%
Doesnt parse any of them properly, and we're getting:
{ "originalmsg": "127.0.0.1 - - [17\/Mar\/2016:18:15:06 +0100] \"GET
\/redacted HTTP\/1.1\" 200 59506", "unparsed-data": "" }
{ "originalmsg": "127.0.0.1 - - [17\/Mar\/2016:18:15:24 +0100]
\"OPTIONS \/ HTTP\/1.1\" 403 205 \"-\" \"-", "unparsed-data": "
\"-\" \"-" }
Here's trace:
liblognorm: loading rulebase file '/test/apps/10-apache.rb'
liblognorm: rulebase version is 2
liblognorm: read rulebase line[~25]: 'type=@apache_common:%[
{"type":"ipv4", "name":"ip"}, {"type":"whitespace"},
{"type":"word", "name":"ident"}, {"type":"whitespace"},
{"type":"word", "name":"user"}, {"type":"literal", "text":"
["}, {"type":"char-to", "name":"date", "extradata":"]"},
{"type":"literal", "text":"] \""}, {"type":"word",
"name":"method"}, {"type":"whitespace"}, {"type":"char-to",
"name":"request", "extradata":" "}, {"type":"literal", "text":"
HTTP/"}, {"type":"float", "name":"httpversion"},
{"type":"literal", "text":"\""}, {"type":"whitespace"},
{"type":"number", "name":"response"}, {"type":"whitespace"},
{"type":"word", "name":"bytes"}]%'
liblognorm: type line to add: '@apache_common:%[ {"type":"ipv4",
"name":"ip"}, {"type":"whitespace"}, {"type":"word",
"name":"ident"}, {"type":"whitespace"}, {"type":"word",
"name":"user"}, {"type":"literal", "text":" ["},
{"type":"char-to", "name":"date", "extradata":"]"},
{"type":"literal", "text":"] \""}, {"type":"word",
"name":"method"}, {"type":"whitespace"}, {"type":"char-to",
"name":"request", "extradata":" "}, {"type":"literal", "text":"
HTTP/"}, {"type":"float", "name":"httpversion"},
{"type":"literal", "text":"\""}, {"type":"whitespace"},
{"type":"number", "name":"response"}, {"type":"whitespace"},
{"type":"word", "name":"bytes"}]%'
liblognorm: type name is '@apache_common'
liblognorm: type line to add: '%[ {"type":"ipv4", "name":"ip"},
{"type":"whitespace"}, {"type":"word", "name":"ident"},
{"type":"whitespace"}, {"type":"word", "name":"user"},
{"type":"literal", "text":" ["}, {"type":"char-to",
"name":"date", "extradata":"]"}, {"type":"literal", "text":"]
\""}, {"type":"word", "name":"method"}, {"type":"whitespace"},
{"type":"char-to", "name":"request", "extradata":" "},
{"type":"literal", "text":" HTTP/"}, {"type":"float",
"name":"httpversion"}, {"type":"literal", "text":"\""},
{"type":"whitespace"}, {"type":"number", "name":"response"},
{"type":"whitespace"}, {"type":"word", "name":"bytes"}]%'
liblognorm: ln_pdagFindType, name '@apache_common', bAdd: 1, nTypes 0
liblognorm: custom type '@apache_common' does not yet exist, adding...
liblognorm: addSampToTree 0 of 603
liblognorm: parsed literal: ''
liblognorm: ln_pdagAddParserInternal: [ { "type": "ipv4", "name":
"ip" }, { "type": "whitespace" }, { "type": "word", "name": "ident"
}, { "type": "whitespace" }, { "type": "word", "name": "user" }, {
"type": "literal", "text": " [" }, { "type": "char-to", "name":
"date", "extradata": "]" }, { "type": "literal", "text": "] \"" }, {
"type": "word", "name": "method" }, { "type": "whitespace" }, {
"type": "char-to", "name": "request", "extradata": " " }, { "type":
"literal", "text": " HTTP\/" }, { "type": "float", "name":
"httpversion" }, { "type": "literal", "text": "\"" }, { "type":
"whitespace" }, { "type": "number", "name": "response" }, { "type":
"whitespace" }, { "type": "word", "name": "bytes" } ]
liblognorm: parser 0: { "type": "ipv4", "name": "ip" }
liblognorm: ln_pdagAddParserInstance: { "type": "ipv4", "name": "ip"
}, nextnode (nil)
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x7f97c606e650, parser 0x7f97c606f010
liblognorm: parser 1: { "type": "whitespace" }
liblognorm: ln_pdagAddParserInstance: { "type": "whitespace" },
nextnode (nil)
liblognorm: assigned priority is 30000
...
liblognorm: pdag: 0x7f97c6071350, parser 0x7f97c606fa90
liblognorm: parser 17: { "type": "word", "name": "bytes" }
liblognorm: ln_pdagAddParserInstance: { "type": "word", "name":
"bytes" }, nextnode (nil)
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x7f97c60716e0, parser 0x7f97c606fa90
liblognorm: parsed literal: ''
liblognorm: end addSampToTree 603 of 603
liblognorm: read rulebase line[~26]: 'type=@apache_combined:-'
liblognorm: type line to add: '@apache_combined:-'
liblognorm: type name is '@apache_combined'
liblognorm: type line to add: '-'
liblognorm: ln_pdagFindType, name '@apache_combined', bAdd: 1, nTypes 1
liblognorm: custom type '@apache_combined' does not yet exist, adding...
liblognorm: addSampToTree 0 of 1
liblognorm: parsed literal: '-'
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "-" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text":
"-" }, nextnode (nil)
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x7f97c6070de0, parser 0x7f97c606fa90
liblognorm: end addSampToTree 1 of 1
liblognorm: read rulebase line[~32]: 'type=@apache_combined:%[
{"type":"whitespace"}, {"type":"quoted-string","name":"referrer"},
{"type":"whitespace"}, {"type":"quoted-string","name":"useragent"}]%'
liblognorm: type line to add: '@apache_combined:%[
{"type":"whitespace"}, {"type":"quoted-string","name":"referrer"},
{"type":"whitespace"}, {"type":"quoted-string","name":"useragent"}]%'
liblognorm: type name is '@apache_combined'
liblognorm: type line to add: '%[ {"type":"whitespace"},
{"type":"quoted-string","name":"referrer"}, {"type":"whitespace"},
{"type":"quoted-string","name":"useragent"}]%'
liblognorm: ln_pdagFindType, name '@apache_combined', bAdd: 1, nTypes 2
liblognorm: addSampToTree 0 of 138
liblognorm: parsed literal: ''
liblognorm: ln_pdagAddParserInternal: [ { "type": "whitespace" }, {
"type": "quoted-string", "name": "referrer" }, { "type":
"whitespace" }, { "type": "quoted-string", "name": "useragent" } ]
liblognorm: parser 0: { "type": "whitespace" }
liblognorm: ln_pdagAddParserInstance: { "type": "whitespace" },
nextnode (nil)
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x7f97c6070de0, parser 0x7f97c60713e0
liblognorm: parser comparison:
{ "type": "literal", "text": "-" }
{ "type": "whitespace" }
liblognorm: parser 1: { "type": "quoted-string", "name": "referrer" }
liblognorm: ln_pdagAddParserInstance: { "type": "quoted-string",
"name": "referrer" }, nextnode (nil)
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x7f97c6070c80, parser 0x7f97c6071b10
liblognorm: parser 2: { "type": "whitespace" }
liblognorm: ln_pdagAddParserInstance: { "type": "whitespace" },
nextnode (nil)
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x7f97c60713e0, parser 0x7f97c6071b10
liblognorm: parser 3: { "type": "quoted-string", "name": "useragent" }
liblognorm: ln_pdagAddParserInstance: { "type": "quoted-string",
"name": "useragent" }, nextnode (nil)
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x7f97c6071770, parser 0x7f97c6071b10
liblognorm: parsed literal: ''
liblognorm: end addSampToTree 138 of 138
liblognorm: read rulebase line[~36]: 'rule=access_combined:%[
{"type":"@apache_common", "name":"."},
{"type":"@apache_combined","name":"."}]%'
liblognorm: rule line to add: 'access_combined:%[
{"type":"@apache_common", "name":"."},
{"type":"@apache_combined","name":"."}]%'
liblognorm: tag found: 'access_combined'
liblognorm: addSampToTree 0 of 82
liblognorm: parsed literal: ''
liblognorm: ln_pdagAddParserInternal: [ { "type": "@apache_common",
"name": "." }, { "type": "@apache_combined", "name": "." } ]
liblognorm: parser 0: { "type": "@apache_common", "name": "." }
liblognorm: ln_pdagAddParserInstance: { "type": "@apache_common",
"name": "." }, nextnode (nil)
liblognorm: ln_pdagFindType, name '@apache_common', bAdd: 0, nTypes 2
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x7f97c606d0a0, parser 0x7f97c6070490
liblognorm: parser 1: { "type": "@apache_combined", "name": "." }
liblognorm: ln_pdagAddParserInstance: { "type": "@apache_combined",
"name": "." }, nextnode (nil)
liblognorm: ln_pdagFindType, name '@apache_combined', bAdd: 0, nTypes 2
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x7f97c6071df0, parser 0x7f97c6071e80
liblognorm: parsed literal: ''
liblognorm: end addSampToTree 82 of 82
liblognorm: optimizing component @apache_common
liblognorm: pre sort, parser 0:ip[7680004]
liblognorm: post sort, parser 0:ip[7680004]
liblognorm: optimizing 0x7f97c606f050: field 0 type 'ipv4', name
'ip': 'UNKNOWN':
...
liblognorm: pre sort, parser 0:bytes[7680032]
liblognorm: post sort, parser 0:bytes[7680032]
liblognorm: optimizing 0x7f97c6071a80: field 0 type 'word', name
'bytes': 'UNKNOWN':
liblognorm: optimizing component @apache_combined
liblognorm: pre sort, parser 0:(null)[7680004]
liblognorm: pre sort, parser 1:(null)[7680004]
liblognorm: post sort, parser 0:(null)[7680004]
liblognorm: post sort, parser 1:(null)[7680004]
liblognorm: optimizing 0x7f97c6070870: field 0 type 'literal', name
'(null)': '-':
liblognorm: optimizing 0x7f97c6070c80: field 1 type 'whitespace',
name '(null)': 'UNKNOWN':
liblognorm: pre sort, parser 0:referrer[7680064]
liblognorm: post sort, parser 0:referrer[7680064]
liblognorm: optimizing 0x7f97c60713e0: field 0 type 'quoted-string',
name 'referrer': 'UNKNOWN':
liblognorm: pre sort, parser 0:(null)[7680004]
liblognorm: post sort, parser 0:(null)[7680004]
liblognorm: optimizing 0x7f97c6071770: field 0 type 'whitespace',
name '(null)': 'UNKNOWN':
liblognorm: pre sort, parser 0:useragent[7680064]
liblognorm: post sort, parser 0:useragent[7680064]
liblognorm: optimizing 0x7f97c606fb80: field 0 type 'quoted-string',
name 'useragent': 'UNKNOWN':
liblognorm: optimizing main pdag component
liblognorm: pre sort, parser 0:.[7680016]
liblognorm: post sort, parser 0:.[7680016]
liblognorm: optimizing 0x7f97c6071df0: field 0 type 'USER-DEFINED',
name '.': 'UNKNOWN':
liblognorm: pre sort, parser 0:.[7680016]
liblognorm: post sort, parser 0:.[7680016]
liblognorm: optimizing 0x7f97c6071b90: field 0 type 'USER-DEFINED',
name '.': 'UNKNOWN':
liblognorm: finished optimizing main pdag component
liblognorm: ---AFTER OPTIMIZATION------------------
liblognorm: COMPONENT: @apache_common
liblognorm: subDAG 0x7f97c606e650 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'ipv4', name 'ip': 'UNKNOWN': called 0
liblognorm: field type 'ipv4', name 'ip': 'UNKNOWN':
liblognorm: subDAG 0x7f97c606f050 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
...
liblognorm: field type 'whitespace', name '(null)': 'UNKNOWN':
liblognorm: subDAG 0x7f97c60716e0 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'word', name 'bytes': 'UNKNOWN': called 0
liblognorm: field type 'word', name 'bytes': 'UNKNOWN':
liblognorm: subDAG [TERM] 0x7f97c6071a80 (children: 0 parsers, ref
1) [called 0, backtracked 0]
liblognorm: COMPONENT: @apache_combined
liblognorm: subDAG 0x7f97c6070de0 (children: 2 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'literal', name '(null)': '-': called 0
liblognorm: field type 'whitespace', name '(null)': 'UNKNOWN': called 0
liblognorm: field type 'literal', name '(null)': '-':
liblognorm: subDAG [TERM] 0x7f97c6070870 (children: 0 parsers, ref
1) [called 0, backtracked 0]
liblognorm: field type 'whitespace', name '(null)': 'UNKNOWN':
liblognorm: subDAG 0x7f97c6070c80 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'quoted-string', name 'referrer': 'UNKNOWN':
called 0
liblognorm: field type 'quoted-string', name 'referrer': 'UNKNOWN':
liblognorm: subDAG 0x7f97c60713e0 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'whitespace', name '(null)': 'UNKNOWN': called 0
liblognorm: field type 'whitespace', name '(null)': 'UNKNOWN':
liblognorm: subDAG 0x7f97c6071770 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'quoted-string', name 'useragent': 'UNKNOWN':
called 0
liblognorm: field type 'quoted-string', name 'useragent': 'UNKNOWN':
liblognorm: subDAG [TERM] 0x7f97c606fb80 (children: 0 parsers, ref
1) [called 0, backtracked 0]
liblognorm: MAIN COMPONENT:
liblognorm: subDAG 0x7f97c606d0a0 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'USER-DEFINED', name '.': 'UNKNOWN': called 0
liblognorm: field type 'USER-DEFINED', name '.': 'UNKNOWN':
liblognorm: subDAG 0x7f97c6071df0 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'USER-DEFINED', name '.': 'UNKNOWN': called 0
liblognorm: field type 'USER-DEFINED', name '.': 'UNKNOWN':
liblognorm: subDAG [TERM] 0x7f97c6071b90 (children: 0 parsers, ref
1) [called 0, backtracked 0]
liblognorm: MAIN COMPONENT (alternative):
liblognorm: 0x7f97c606d0a0[ref 1]:
liblognorm: 0x7f97c6071df0[ref 1]: %.:USER-DEFINED%
liblognorm: 0x7f97c6071b90[ref 1]: %.:USER-DEFINED%%.:USER-DEFINED%
liblognorm: =======================================
number of tree nodes: 28
liblognorm: COMPONENT: @apache_common
liblognorm: subDAG 0x7f97c606e650 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'ipv4', name 'ip': 'UNKNOWN': called 0
liblognorm: field type 'ipv4', name 'ip': 'UNKNOWN':
...
liblognorm: subDAG 0x7f97c60716e0 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'word', name 'bytes': 'UNKNOWN': called 0
liblognorm: field type 'word', name 'bytes': 'UNKNOWN':
liblognorm: subDAG [TERM] 0x7f97c6071a80 (children: 0 parsers, ref
1) [called 0, backtracked 0]
liblognorm: COMPONENT: @apache_combined
liblognorm: subDAG 0x7f97c6070de0 (children: 2 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'literal', name '(null)': '-': called 0
liblognorm: field type 'whitespace', name '(null)': 'UNKNOWN': called 0
liblognorm: field type 'literal', name '(null)': '-':
liblognorm: subDAG [TERM] 0x7f97c6070870 (children: 0 parsers, ref
1) [called 0, backtracked 0]
liblognorm: field type 'whitespace', name '(null)': 'UNKNOWN':
liblognorm: subDAG 0x7f97c6070c80 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'quoted-string', name 'referrer': 'UNKNOWN':
called 0
liblognorm: field type 'quoted-string', name 'referrer': 'UNKNOWN':
liblognorm: subDAG 0x7f97c60713e0 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'whitespace', name '(null)': 'UNKNOWN': called 0
liblognorm: field type 'whitespace', name '(null)': 'UNKNOWN':
liblognorm: subDAG 0x7f97c6071770 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'quoted-string', name 'useragent': 'UNKNOWN':
called 0
liblognorm: field type 'quoted-string', name 'useragent': 'UNKNOWN':
liblognorm: subDAG [TERM] 0x7f97c606fb80 (children: 0 parsers, ref
1) [called 0, backtracked 0]
liblognorm: MAIN COMPONENT:
liblognorm: subDAG 0x7f97c606d0a0 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'USER-DEFINED', name '.': 'UNKNOWN': called 0
liblognorm: field type 'USER-DEFINED', name '.': 'UNKNOWN':
liblognorm: subDAG 0x7f97c6071df0 (children: 1 parsers, ref 1)
[called 0, backtracked 0]
liblognorm: field type 'USER-DEFINED', name '.': 'UNKNOWN': called 0
liblognorm: field type 'USER-DEFINED', name '.': 'UNKNOWN':
liblognorm: subDAG [TERM] 0x7f97c6071b90 (children: 0 parsers, ref
1) [called 0, backtracked 0]
liblognorm: MAIN COMPONENT (alternative):
liblognorm: 0x7f97c606d0a0[ref 1]:
liblognorm: 0x7f97c6071df0[ref 1]: %.:USER-DEFINED%
liblognorm: 0x7f97c6071b90[ref 1]: %.:USER-DEFINED%%.:USER-DEFINED%
To normalize: '127.0.0.1 - - [17/Mar/2016:18:15:06 +0100] "GET
/redacted HTTP/1.1" 200 59506'
liblognorm: 0: enter parser, dag node 0x7f97c606d0a0, json
0x7f97c6071590
liblognorm: 0/0:trying 'USER-DEFINED' parser for field '.', data
'UNKNOWN'
liblognorm: calling custom parser '@apache_common'
liblognorm: 0: enter parser, dag node 0x7f97c606e650, json
0x7f97c60700f0
liblognorm: 0/1:trying 'ipv4' parser for field 'ip', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 9
liblognorm: 0: potential hit, trying subtree 0x7f97c606f050
liblognorm: 9: enter parser, dag node 0x7f97c606f050, json
0x7f97c60700f0
liblognorm: 9/1:trying 'whitespace' parser for field '(null)', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 9: potential hit, trying subtree 0x7f97c606f180
liblognorm: 10: enter parser, dag node 0x7f97c606f180, json
0x7f97c60700f0
liblognorm: 10/1:trying 'word' parser for field 'ident', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 10: potential hit, trying subtree 0x7f97c606f3e0
liblognorm: 11: enter parser, dag node 0x7f97c606f3e0, json
0x7f97c60700f0
liblognorm: 11/1:trying 'whitespace' parser for field '(null)', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 11: potential hit, trying subtree 0x7f97c606f610
liblognorm: 12: enter parser, dag node 0x7f97c606f610, json
0x7f97c60700f0
liblognorm: 12/1:trying 'word' parser for field 'user', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 12: potential hit, trying subtree 0x7f97c606f780
liblognorm: 13: enter parser, dag node 0x7f97c606f780, json
0x7f97c60700f0
liblognorm: 13/1:trying 'literal' parser for field '(null)', data ' ['
liblognorm: parser lookup returns 0, pParsed 2
liblognorm: 13: potential hit, trying subtree 0x7f97c606f820
liblognorm: 15: enter parser, dag node 0x7f97c606f820, json
0x7f97c60700f0
liblognorm: 15/1:trying 'char-to' parser for field 'date', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 26
liblognorm: 15: potential hit, trying subtree 0x7f97c606fc30
liblognorm: 41: enter parser, dag node 0x7f97c606fc30, json
0x7f97c60700f0
liblognorm: 41/1:trying 'literal' parser for field '(null)', data '] "'
liblognorm: parser lookup returns 0, pParsed 3
liblognorm: 41: potential hit, trying subtree 0x7f97c606fdc0
liblognorm: 44: enter parser, dag node 0x7f97c606fdc0, json
0x7f97c60700f0
liblognorm: 44/1:trying 'word' parser for field 'method', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 3
liblognorm: 44: potential hit, trying subtree 0x7f97c6070050
liblognorm: 47: enter parser, dag node 0x7f97c6070050, json
0x7f97c60700f0
liblognorm: 47/1:trying 'whitespace' parser for field '(null)', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 47: potential hit, trying subtree 0x7f97c60703c0
liblognorm: 48: enter parser, dag node 0x7f97c60703c0, json
0x7f97c60700f0
liblognorm: 48/1:trying 'char-to' parser for field 'request', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 9
liblognorm: 48: potential hit, trying subtree 0x7f97c6070530
liblognorm: 57: enter parser, dag node 0x7f97c6070530, json
0x7f97c60700f0
liblognorm: 57/1:trying 'literal' parser for field '(null)', data '
HTTP/'
liblognorm: parser lookup returns 0, pParsed 6
liblognorm: 57: potential hit, trying subtree 0x7f97c6070bd0
liblognorm: 63: enter parser, dag node 0x7f97c6070bd0, json
0x7f97c60700f0
liblognorm: 63/1:trying 'float' parser for field 'httpversion', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 3
liblognorm: 63: potential hit, trying subtree 0x7f97c6070d50
liblognorm: 66: enter parser, dag node 0x7f97c6070d50, json
0x7f97c60700f0
liblognorm: 66/1:trying 'literal' parser for field '(null)', data '"'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 66: potential hit, trying subtree 0x7f97c6070f90
liblognorm: 67: enter parser, dag node 0x7f97c6070f90, json
0x7f97c60700f0
liblognorm: 67/1:trying 'whitespace' parser for field '(null)', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 67: potential hit, trying subtree 0x7f97c6071200
liblognorm: 68: enter parser, dag node 0x7f97c6071200, json
0x7f97c60700f0
liblognorm: 68/1:trying 'number' parser for field 'response', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 3
liblognorm: 68: potential hit, trying subtree 0x7f97c6071350
liblognorm: 71: enter parser, dag node 0x7f97c6071350, json
0x7f97c60700f0
liblognorm: 71/1:trying 'whitespace' parser for field '(null)', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 71: potential hit, trying subtree 0x7f97c60716e0
liblognorm: 72: enter parser, dag node 0x7f97c60716e0, json
0x7f97c60700f0
liblognorm: 72/1:trying 'word' parser for field 'bytes', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 5
liblognorm: 72: potential hit, trying subtree 0x7f97c6071a80
liblognorm: 77: enter parser, dag node 0x7f97c6071a80, json
0x7f97c60700f0
liblognorm: offs 77, strLen 77, isTerm 1
liblognorm: 77 returns 0, pParsedTo 0, parsedTo 0
liblognorm: 72: subtree returns 0, parsedTo 77
liblognorm: 72: parser matches at 72
liblognorm: parsedTo 77, *pParsedTo 77
liblognorm: offs 72, strLen 77, isTerm 0
liblognorm: 72 returns 0, pParsedTo 77, parsedTo 77
liblognorm: 71: subtree returns 0, parsedTo 72
liblognorm: 71: parser matches at 71
liblognorm: parsedTo 72, *pParsedTo 77
liblognorm: offs 71, strLen 77, isTerm 0
liblognorm: 71 returns 0, pParsedTo 77, parsedTo 72
liblognorm: 68: subtree returns 0, parsedTo 71
liblognorm: 68: parser matches at 68
liblognorm: parsedTo 71, *pParsedTo 77
liblognorm: offs 68, strLen 77, isTerm 0
liblognorm: 68 returns 0, pParsedTo 77, parsedTo 71
liblognorm: 67: subtree returns 0, parsedTo 68
liblognorm: 67: parser matches at 67
liblognorm: parsedTo 68, *pParsedTo 77
liblognorm: offs 67, strLen 77, isTerm 0
liblognorm: 67 returns 0, pParsedTo 77, parsedTo 68
liblognorm: 66: subtree returns 0, parsedTo 67
liblognorm: 66: parser matches at 66
liblognorm: parsedTo 67, *pParsedTo 77
liblognorm: offs 66, strLen 77, isTerm 0
liblognorm: 66 returns 0, pParsedTo 77, parsedTo 67
liblognorm: 63: subtree returns 0, parsedTo 66
liblognorm: 63: parser matches at 63
liblognorm: parsedTo 66, *pParsedTo 77
liblognorm: offs 63, strLen 77, isTerm 0
liblognorm: 63 returns 0, pParsedTo 77, parsedTo 66
liblognorm: 57: subtree returns 0, parsedTo 63
liblognorm: 57: parser matches at 57
liblognorm: parsedTo 63, *pParsedTo 77
liblognorm: offs 57, strLen 77, isTerm 0
liblognorm: 57 returns 0, pParsedTo 77, parsedTo 63
liblognorm: 48: subtree returns 0, parsedTo 57
liblognorm: 48: parser matches at 48
liblognorm: parsedTo 57, *pParsedTo 77
liblognorm: offs 48, strLen 77, isTerm 0
liblognorm: 48 returns 0, pParsedTo 77, parsedTo 57
liblognorm: 47: subtree returns 0, parsedTo 48
liblognorm: 47: parser matches at 47
liblognorm: parsedTo 48, *pParsedTo 77
liblognorm: offs 47, strLen 77, isTerm 0
liblognorm: 47 returns 0, pParsedTo 77, parsedTo 48
liblognorm: 44: subtree returns 0, parsedTo 47
liblognorm: 44: parser matches at 44
liblognorm: parsedTo 47, *pParsedTo 77
liblognorm: offs 44, strLen 77, isTerm 0
liblognorm: 44 returns 0, pParsedTo 77, parsedTo 47
liblognorm: 41: subtree returns 0, parsedTo 44
liblognorm: 41: parser matches at 41
liblognorm: parsedTo 44, *pParsedTo 77
liblognorm: offs 41, strLen 77, isTerm 0
liblognorm: 41 returns 0, pParsedTo 77, parsedTo 44
liblognorm: 15: subtree returns 0, parsedTo 41
liblognorm: 15: parser matches at 15
liblognorm: parsedTo 41, *pParsedTo 77
liblognorm: offs 15, strLen 77, isTerm 0
liblognorm: 15 returns 0, pParsedTo 77, parsedTo 41
liblognorm: 13: subtree returns 0, parsedTo 15
liblognorm: 13: parser matches at 13
liblognorm: parsedTo 15, *pParsedTo 77
liblognorm: offs 13, strLen 77, isTerm 0
liblognorm: 13 returns 0, pParsedTo 77, parsedTo 15
liblognorm: 12: subtree returns 0, parsedTo 13
liblognorm: 12: parser matches at 12
liblognorm: parsedTo 13, *pParsedTo 77
liblognorm: offs 12, strLen 77, isTerm 0
liblognorm: 12 returns 0, pParsedTo 77, parsedTo 13
liblognorm: 11: subtree returns 0, parsedTo 12
liblognorm: 11: parser matches at 11
liblognorm: parsedTo 12, *pParsedTo 77
liblognorm: offs 11, strLen 77, isTerm 0
liblognorm: 11 returns 0, pParsedTo 77, parsedTo 12
liblognorm: 10: subtree returns 0, parsedTo 11
liblognorm: 10: parser matches at 10
liblognorm: parsedTo 11, *pParsedTo 77
liblognorm: offs 10, strLen 77, isTerm 0
liblognorm: 10 returns 0, pParsedTo 77, parsedTo 11
liblognorm: 9: subtree returns 0, parsedTo 10
liblognorm: 9: parser matches at 9
liblognorm: parsedTo 10, *pParsedTo 77
liblognorm: offs 9, strLen 77, isTerm 0
liblognorm: 9 returns 0, pParsedTo 77, parsedTo 10
liblognorm: 0: subtree returns 0, parsedTo 9
liblognorm: 0: parser matches at 0
liblognorm: parsedTo 9, *pParsedTo 77
liblognorm: offs 0, strLen 77, isTerm 0
liblognorm: 0 returns 0, pParsedTo 77, parsedTo 9
liblognorm: called CUSTOM PARSER '@apache_common', result 0, offs 0,
*pParsed 0
liblognorm: parser lookup returns 0, pParsed 77
liblognorm: 0: potential hit, trying subtree 0x7f97c6071df0
liblognorm: 77: enter parser, dag node 0x7f97c6071df0, json
0x7f97c6071590
liblognorm: 77/0:trying 'USER-DEFINED' parser for field '.', data
'UNKNOWN'
liblognorm: calling custom parser '@apache_combined'
liblognorm: 77: enter parser, dag node 0x7f97c6070de0, json
0x7f97c606e9c0
liblognorm: 77/1:trying 'literal' parser for field '(null)', data '-'
liblognorm: parser lookup returns -1000, pParsed 0
liblognorm: parsedTo 0, *pParsedTo 0
liblognorm: 77/1:trying 'whitespace' parser for field '(null)', data
'UNKNOWN'
liblognorm: parser lookup returns -1000, pParsed 0
liblognorm: parsedTo 0, *pParsedTo 0
liblognorm: offs 77, strLen 77, isTerm 0
liblognorm: 77 returns -1000, pParsedTo 0, parsedTo 0
liblognorm: called CUSTOM PARSER '@apache_combined', result -1000,
offs 77, *pParsed 0
liblognorm: parser lookup returns -1000, pParsed 18446744073709551539
liblognorm: parsedTo 0, *pParsedTo 0
liblognorm: offs 77, strLen 77, isTerm 0
liblognorm: 77 returns -1000, pParsedTo 0, parsedTo 0
liblognorm: 0: subtree returns -1000, parsedTo 77
liblognorm: 0 nonmatch, backtracking required, parsed to=77
liblognorm: parsedTo 77, *pParsedTo 77
liblognorm: offs 0, strLen 77, isTerm 0
liblognorm: 0 returns -1000, pParsedTo 77, parsedTo 77
liblognorm: final result for normalizer: parsedTo 77, endNode (nil)
liblognorm: DONE, final return is -1000
normalized: '{ "originalmsg": "127.0.0.1 - - [17\/Mar\/2016:18:15:06
+0100] \"GET \/redacted HTTP\/1.1\" 200 59506", "unparsed-data": "" }'
{ "originalmsg": "127.0.0.1 - - [17\/Mar\/2016:18:15:06 +0100] \"GET
\/redacted HTTP\/1.1\" 200 59506", "unparsed-data": "" }
To normalize: '127.0.0.1 - - [17/Mar/2016:18:15:24 +0100] "OPTIONS /
HTTP/1.1" 403 205 "-" "-'
liblognorm: 0: enter parser, dag node 0x7f97c606d0a0, json
0x7f97c6071590
liblognorm: 0/0:trying 'USER-DEFINED' parser for field '.', data
'UNKNOWN'
liblognorm: calling custom parser '@apache_common'
liblognorm: 0: enter parser, dag node 0x7f97c606e650, json
0x7f97c60700f0
liblognorm: 0/1:trying 'ipv4' parser for field 'ip', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 9
liblognorm: 0: potential hit, trying subtree 0x7f97c606f050
liblognorm: 9: enter parser, dag node 0x7f97c606f050, json
0x7f97c60700f0
liblognorm: 9/1:trying 'whitespace' parser for field '(null)', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 9: potential hit, trying subtree 0x7f97c606f180
liblognorm: 10: enter parser, dag node 0x7f97c606f180, json
0x7f97c60700f0
liblognorm: 10/1:trying 'word' parser for field 'ident', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 10: potential hit, trying subtree 0x7f97c606f3e0
liblognorm: 11: enter parser, dag node 0x7f97c606f3e0, json
0x7f97c60700f0
liblognorm: 11/1:trying 'whitespace' parser for field '(null)', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 11: potential hit, trying subtree 0x7f97c606f610
liblognorm: 12: enter parser, dag node 0x7f97c606f610, json
0x7f97c60700f0
liblognorm: 12/1:trying 'word' parser for field 'user', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 12: potential hit, trying subtree 0x7f97c606f780
liblognorm: 13: enter parser, dag node 0x7f97c606f780, json
0x7f97c60700f0
liblognorm: 13/1:trying 'literal' parser for field '(null)', data ' ['
liblognorm: parser lookup returns 0, pParsed 2
liblognorm: 13: potential hit, trying subtree 0x7f97c606f820
liblognorm: 15: enter parser, dag node 0x7f97c606f820, json
0x7f97c60700f0
liblognorm: 15/1:trying 'char-to' parser for field 'date', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 26
liblognorm: 15: potential hit, trying subtree 0x7f97c606fc30
liblognorm: 41: enter parser, dag node 0x7f97c606fc30, json
0x7f97c60700f0
liblognorm: 41/1:trying 'literal' parser for field '(null)', data '] "'
liblognorm: parser lookup returns 0, pParsed 3
liblognorm: 41: potential hit, trying subtree 0x7f97c606fdc0
liblognorm: 44: enter parser, dag node 0x7f97c606fdc0, json
0x7f97c60700f0
liblognorm: 44/1:trying 'word' parser for field 'method', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 7
liblognorm: 44: potential hit, trying subtree 0x7f97c6070050
liblognorm: 51: enter parser, dag node 0x7f97c6070050, json
0x7f97c60700f0
liblognorm: 51/1:trying 'whitespace' parser for field '(null)', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 51: potential hit, trying subtree 0x7f97c60703c0
liblognorm: 52: enter parser, dag node 0x7f97c60703c0, json
0x7f97c60700f0
liblognorm: 52/1:trying 'char-to' parser for field 'request', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 52: potential hit, trying subtree 0x7f97c6070530
liblognorm: 53: enter parser, dag node 0x7f97c6070530, json
0x7f97c60700f0
liblognorm: 53/1:trying 'literal' parser for field '(null)', data '
HTTP/'
liblognorm: parser lookup returns 0, pParsed 6
liblognorm: 53: potential hit, trying subtree 0x7f97c6070bd0
liblognorm: 59: enter parser, dag node 0x7f97c6070bd0, json
0x7f97c60700f0
liblognorm: 59/1:trying 'float' parser for field 'httpversion', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 3
liblognorm: 59: potential hit, trying subtree 0x7f97c6070d50
liblognorm: 62: enter parser, dag node 0x7f97c6070d50, json
0x7f97c60700f0
liblognorm: 62/1:trying 'literal' parser for field '(null)', data '"'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 62: potential hit, trying subtree 0x7f97c6070f90
liblognorm: 63: enter parser, dag node 0x7f97c6070f90, json
0x7f97c60700f0
liblognorm: 63/1:trying 'whitespace' parser for field '(null)', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 63: potential hit, trying subtree 0x7f97c6071200
liblognorm: 64: enter parser, dag node 0x7f97c6071200, json
0x7f97c60700f0
liblognorm: 64/1:trying 'number' parser for field 'response', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 3
liblognorm: 64: potential hit, trying subtree 0x7f97c6071350
liblognorm: 67: enter parser, dag node 0x7f97c6071350, json
0x7f97c60700f0
liblognorm: 67/1:trying 'whitespace' parser for field '(null)', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 67: potential hit, trying subtree 0x7f97c60716e0
liblognorm: 68: enter parser, dag node 0x7f97c60716e0, json
0x7f97c60700f0
liblognorm: 68/1:trying 'word' parser for field 'bytes', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 3
liblognorm: 68: potential hit, trying subtree 0x7f97c6071a80
liblognorm: 71: enter parser, dag node 0x7f97c6071a80, json
0x7f97c60700f0
liblognorm: offs 71, strLen 78, isTerm 1
liblognorm: 71 returns 0, pParsedTo 0, parsedTo 0
liblognorm: 68: subtree returns 0, parsedTo 71
liblognorm: 68: parser matches at 68
liblognorm: parsedTo 71, *pParsedTo 71
liblognorm: offs 68, strLen 78, isTerm 0
liblognorm: 68 returns 0, pParsedTo 71, parsedTo 71
liblognorm: 67: subtree returns 0, parsedTo 68
liblognorm: 67: parser matches at 67
liblognorm: parsedTo 68, *pParsedTo 71
liblognorm: offs 67, strLen 78, isTerm 0
liblognorm: 67 returns 0, pParsedTo 71, parsedTo 68
liblognorm: 64: subtree returns 0, parsedTo 67
liblognorm: 64: parser matches at 64
liblognorm: parsedTo 67, *pParsedTo 71
liblognorm: offs 64, strLen 78, isTerm 0
liblognorm: 64 returns 0, pParsedTo 71, parsedTo 67
liblognorm: 63: subtree returns 0, parsedTo 64
liblognorm: 63: parser matches at 63
liblognorm: parsedTo 64, *pParsedTo 71
liblognorm: offs 63, strLen 78, isTerm 0
liblognorm: 63 returns 0, pParsedTo 71, parsedTo 64
liblognorm: 62: subtree returns 0, parsedTo 63
liblognorm: 62: parser matches at 62
liblognorm: parsedTo 63, *pParsedTo 71
liblognorm: offs 62, strLen 78, isTerm 0
liblognorm: 62 returns 0, pParsedTo 71, parsedTo 63
liblognorm: 59: subtree returns 0, parsedTo 62
liblognorm: 59: parser matches at 59
liblognorm: parsedTo 62, *pParsedTo 71
liblognorm: offs 59, strLen 78, isTerm 0
liblognorm: 59 returns 0, pParsedTo 71, parsedTo 62
liblognorm: 53: subtree returns 0, parsedTo 59
liblognorm: 53: parser matches at 53
liblognorm: parsedTo 59, *pParsedTo 71
liblognorm: offs 53, strLen 78, isTerm 0
liblognorm: 53 returns 0, pParsedTo 71, parsedTo 59
liblognorm: 52: subtree returns 0, parsedTo 53
liblognorm: 52: parser matches at 52
liblognorm: parsedTo 53, *pParsedTo 71
liblognorm: offs 52, strLen 78, isTerm 0
liblognorm: 52 returns 0, pParsedTo 71, parsedTo 53
liblognorm: 51: subtree returns 0, parsedTo 52
liblognorm: 51: parser matches at 51
liblognorm: parsedTo 52, *pParsedTo 71
liblognorm: offs 51, strLen 78, isTerm 0
liblognorm: 51 returns 0, pParsedTo 71, parsedTo 52
liblognorm: 44: subtree returns 0, parsedTo 51
liblognorm: 44: parser matches at 44
liblognorm: parsedTo 51, *pParsedTo 71
liblognorm: offs 44, strLen 78, isTerm 0
liblognorm: 44 returns 0, pParsedTo 71, parsedTo 51
liblognorm: 41: subtree returns 0, parsedTo 44
liblognorm: 41: parser matches at 41
liblognorm: parsedTo 44, *pParsedTo 71
liblognorm: offs 41, strLen 78, isTerm 0
liblognorm: 41 returns 0, pParsedTo 71, parsedTo 44
liblognorm: 15: subtree returns 0, parsedTo 41
liblognorm: 15: parser matches at 15
liblognorm: parsedTo 41, *pParsedTo 71
liblognorm: offs 15, strLen 78, isTerm 0
liblognorm: 15 returns 0, pParsedTo 71, parsedTo 41
liblognorm: 13: subtree returns 0, parsedTo 15
liblognorm: 13: parser matches at 13
liblognorm: parsedTo 15, *pParsedTo 71
liblognorm: offs 13, strLen 78, isTerm 0
liblognorm: 13 returns 0, pParsedTo 71, parsedTo 15
liblognorm: 12: subtree returns 0, parsedTo 13
liblognorm: 12: parser matches at 12
liblognorm: parsedTo 13, *pParsedTo 71
liblognorm: offs 12, strLen 78, isTerm 0
liblognorm: 12 returns 0, pParsedTo 71, parsedTo 13
liblognorm: 11: subtree returns 0, parsedTo 12
liblognorm: 11: parser matches at 11
liblognorm: parsedTo 12, *pParsedTo 71
liblognorm: offs 11, strLen 78, isTerm 0
liblognorm: 11 returns 0, pParsedTo 71, parsedTo 12
liblognorm: 10: subtree returns 0, parsedTo 11
liblognorm: 10: parser matches at 10
liblognorm: parsedTo 11, *pParsedTo 71
liblognorm: offs 10, strLen 78, isTerm 0
liblognorm: 10 returns 0, pParsedTo 71, parsedTo 11
liblognorm: 9: subtree returns 0, parsedTo 10
liblognorm: 9: parser matches at 9
liblognorm: parsedTo 10, *pParsedTo 71
liblognorm: offs 9, strLen 78, isTerm 0
liblognorm: 9 returns 0, pParsedTo 71, parsedTo 10
liblognorm: 0: subtree returns 0, parsedTo 9
liblognorm: 0: parser matches at 0
liblognorm: parsedTo 9, *pParsedTo 71
liblognorm: offs 0, strLen 78, isTerm 0
liblognorm: 0 returns 0, pParsedTo 71, parsedTo 9
liblognorm: called CUSTOM PARSER '@apache_common', result 0, offs 0,
*pParsed 0
liblognorm: parser lookup returns 0, pParsed 71
liblognorm: 0: potential hit, trying subtree 0x7f97c6071df0
liblognorm: 71: enter parser, dag node 0x7f97c6071df0, json
0x7f97c6071590
liblognorm: 71/0:trying 'USER-DEFINED' parser for field '.', data
'UNKNOWN'
liblognorm: calling custom parser '@apache_combined'
liblognorm: 71: enter parser, dag node 0x7f97c6070de0, json
0x7f97c6072650
liblognorm: 71/1:trying 'literal' parser for field '(null)', data '-'
liblognorm: parser lookup returns -1000, pParsed 0
liblognorm: parsedTo 0, *pParsedTo 0
liblognorm: 71/1:trying 'whitespace' parser for field '(null)', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 71: potential hit, trying subtree 0x7f97c6070c80
liblognorm: 72: enter parser, dag node 0x7f97c6070c80, json
0x7f97c6072650
liblognorm: 72/1:trying 'quoted-string' parser for field 'referrer',
data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 3
liblognorm: 72: potential hit, trying subtree 0x7f97c60713e0
liblognorm: 75: enter parser, dag node 0x7f97c60713e0, json
0x7f97c6072650
liblognorm: 75/1:trying 'whitespace' parser for field '(null)', data
'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 1
liblognorm: 75: potential hit, trying subtree 0x7f97c6071770
liblognorm: 76: enter parser, dag node 0x7f97c6071770, json
0x7f97c6072650
liblognorm: 76/1:trying 'quoted-string' parser for field
'useragent', data 'UNKNOWN'
liblognorm: parser lookup returns -1000, pParsed 0
liblognorm: parsedTo 0, *pParsedTo 0
liblognorm: offs 76, strLen 78, isTerm 0
liblognorm: 76 returns -1000, pParsedTo 0, parsedTo 0
liblognorm: 75: subtree returns -1000, parsedTo 76
liblognorm: 75 nonmatch, backtracking required, parsed to=76
liblognorm: parsedTo 76, *pParsedTo 76
liblognorm: offs 75, strLen 78, isTerm 0
liblognorm: 75 returns -1000, pParsedTo 76, parsedTo 76
liblognorm: 72: subtree returns -1000, parsedTo 75
liblognorm: 72 nonmatch, backtracking required, parsed to=75
liblognorm: parsedTo 75, *pParsedTo 76
liblognorm: offs 72, strLen 78, isTerm 0
liblognorm: 72 returns -1000, pParsedTo 76, parsedTo 75
liblognorm: 71: subtree returns -1000, parsedTo 72
liblognorm: 71 nonmatch, backtracking required, parsed to=72
liblognorm: parsedTo 72, *pParsedTo 76
liblognorm: offs 71, strLen 78, isTerm 0
liblognorm: 71 returns -1000, pParsedTo 76, parsedTo 72
liblognorm: called CUSTOM PARSER '@apache_combined', result -1000,
offs 71, *pParsed 0
liblognorm: parser lookup returns -1000, pParsed 5
liblognorm: parsedTo 0, *pParsedTo 0
liblognorm: offs 71, strLen 78, isTerm 0
liblognorm: 71 returns -1000, pParsedTo 0, parsedTo 0
liblognorm: 0: subtree returns -1000, parsedTo 71
liblognorm: 0 nonmatch, backtracking required, parsed to=71
liblognorm: parsedTo 71, *pParsedTo 71
liblognorm: offs 0, strLen 78, isTerm 0
liblognorm: 0 returns -1000, pParsedTo 71, parsedTo 71
liblognorm: final result for normalizer: parsedTo 71, endNode (nil)
liblognorm: DONE, final return is -1000
normalized: '{ "originalmsg": "127.0.0.1 - - [17\/Mar\/2016:18:15:24
+0100] \"OPTIONS \/ HTTP\/1.1\" 403 205 \"-\" \"-", "unparsed-data":
" \"-\" \"-" }'
{ "originalmsg": "127.0.0.1 - - [17\/Mar\/2016:18:15:24 +0100]
\"OPTIONS \/ HTTP\/1.1\" 403 205 \"-\" \"-", "unparsed-data": "
\"-\" \"-" }
liblognorm: exitCtx 0x7f97c606d010
liblognorm: delete 0x7f97c606d0a0[1]:
liblognorm: delete 0x7f97c6071df0[1]: %.:USER-DEFINED%
liblognorm: delete 0x7f97c6071b90[1]: %.:USER-DEFINED%%.:USER-DEFINED%
liblognorm: delete 0x7f97c606e650[1]:
liblognorm: delete 0x7f97c606f050[1]: %ip:ipv4%
liblognorm: delete 0x7f97c606f180[1]: %ip:ipv4%%-:whitespace%
liblognorm: delete 0x7f97c606f3e0[1]:
%ip:ipv4%%-:whitespace%%ident:word%
liblognorm: delete 0x7f97c606f610[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%
liblognorm: delete 0x7f97c606f780[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word%
liblognorm: delete 0x7f97c606f820[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word% [
liblognorm: delete 0x7f97c606fc30[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word%
[%date:char-to%
liblognorm: delete 0x7f97c606fdc0[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word%
[%date:char-to%] "
liblognorm: delete 0x7f97c6070050[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word%
[%date:char-to%] "%method:word%
liblognorm: delete 0x7f97c60703c0[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word%
[%date:char-to%] "%method:word%%-:whitespace%
liblognorm: delete 0x7f97c6070530[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word%
[%date:char-to%] "%method:word%%-:whitespace%%request:char-to%
liblognorm: delete 0x7f97c6070bd0[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word%
[%date:char-to%] "%method:word%%-:whitespace%%request:char-to% HTTP/
liblognorm: delete 0x7f97c6070d50[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word%
[%date:char-to%] "%method:word%%-:whitespace%%request:char-to%
HTTP/%httpversion:float%
liblognorm: delete 0x7f97c6070f90[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word%
[%date:char-to%] "%method:word%%-:whitespace%%request:char-to%
HTTP/%httpversion:float%"
liblognorm: delete 0x7f97c6071200[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word%
[%date:char-to%] "%method:word%%-:whitespace%%request:char-to%
HTTP/%httpversion:float%"%-:whitespace%
liblognorm: delete 0x7f97c6071350[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word%
[%date:char-to%] "%method:word%%-:whitespace%%request:char-to%
HTTP/%httpversion:float%"%-:whitespace%%response:number%
liblognorm: delete 0x7f97c60716e0[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word%
[%date:char-to%] "%method:word%%-:whitespace%%request:char-to%
HTTP/%httpversion:float%"%-:whitespace%%response:number%%-:whitespace%
liblognorm: delete 0x7f97c6071a80[1]:
%ip:ipv4%%-:whitespace%%ident:word%%-:whitespace%%user:word%
[%date:char-to%] "%method:word%%-:whitespace%%request:char-to%
HTTP/%httpversion:float%"%-:whitespace%%response:number%%-:whitespace%%bytes:word%
liblognorm: delete 0x7f97c6070de0[1]:
liblognorm: delete 0x7f97c6070870[1]: -
liblognorm: delete 0x7f97c6070c80[1]: %-:whitespace%
liblognorm: delete 0x7f97c60713e0[1]:
%-:whitespace%%referrer:quoted-string%
liblognorm: delete 0x7f97c6071770[1]:
%-:whitespace%%referrer:quoted-string%%-:whitespace%
liblognorm: delete 0x7f97c606fb80[1]:
%-:whitespace%%referrer:quoted-string%%-:whitespace%%useragent:quoted-string%
this is the section to watch, you should see each step along the way
be tested and parsedTo march along the line. It looks like this
doesn't work the same for user defined types, try creating a ruleset
where you have the definition for the type as a rule instead of a type.
sorry, but I don't understand what are you talking about. This are the
liblognorm rules...how could I specify those on a ruleset???
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
