Simon was right in that the problem here is your use of
`read_message("raw")`, which should almost never be used in a decoder.
`read_message("raw")` returns the pack.MsgBytes value. Once a message
hits the router, this is guaranteed to have an accurate protobuf
encoding of the message, but before the router this is not the case.
If you want to use `print` in your Lua scripts you can do so by removing
`print` from the sandbox template's `remove_entries` value in the
lua_sandbox.go.in file:
https://github.com/mozilla-services/heka/blob/dev/sandbox/lua/lua_sandbox.go.in#L60
-r
On 10/12/2015 07:17 AM, Timur Batyrshin wrote:
Hi,
I’ve been writing a decoder for myself and have hit the following issue
which I can’t understand.
When I start Heka it produces the following error message in logs:
2015/10/12 13:54:47 SubDecoder
‘zerogw-zerogw_decoder-stdout-zerogw_rotate_fields’ error: FATAL:
process_message() /usr/share/heka/lua_decoders/rotate_fields.lua:30:
bad argument #0 to ‘decode_message’ (must have one string argument)
At the same time the code for decoder is the following:
|-- the only lines above are comments which are skipped metric_field =
read_config("metric_field") or "metric" value_field =
read_config("value_field") or "value" function process_message() local
fields = {} raw = read_message("raw”) # line 29 msg =
decode_message(raw) # line 30 -- other part of code is probably
irrelevant as crash is seen in the above line |
(I’ve tried writing that as |decode_message(read_message("raw”))| with
the same effect)
What’s really weird is the exactly the same decoder works fine on other
hosts.
I’m using the following Heka config:
|[zerogw] type = "ProcessInput" ticker_interval = 0 splitter =
"on_newline" decoder = "zerogw_decoder" stdout = true stderr = false
[zerogw.command.0] bin = "/usr/local/bin/zerogw_collector.py" args =
["-s", "tcp://127.0.0.1:5111 <http://127.0.0.1:5111>"] [on_newline] type
= "TokenSplitter" delimiter = "\n" [estp_decoder] type =
"PayloadRegexDecoder" match_regex = '^(?P<Name>[^\s]+)
(?P<Timestamp>\d+) (?P<Value>\d+)' timestamp_layout = "Epoch"
[estp_decoder.message_fields] Service = "Zerogw" Metric = "%Name%" Value
= "%Value%" [zerogw_decoder] type = "MultiDecoder" subs =
["estp_decoder", "zerogw_rotate_fields"] cascade_strategy = "all"
[zerogw_rotate_fields] type = "SandboxDecoder" filename =
"lua_decoders/rotate_fields.lua" [zerogw_rotate_fields.config]
metric_field = "Metric" value_field = "Value" |
zerogw_collector.py produces about a dozen of lines to stdout every 5
seconds in the format as seen in message payload (see below).
As MultiDecoder has |cascade_strategy = "all"| Heka dumps messages
processed by the first decoder in the chain to stdout which are the
following:
|2015/10/12 14:04:22 :Timestamp: 2015-10-12 14:04:22 +0000 UTC :Type:
ProcessInput :Hostname: t-eu-zgw :Pid: 5212 :Uuid:
2c7deb23-7961-49dc-8f57-da716d851439 :Logger: zerogw :Payload:
zerogw.connections.total 1444658662 4 :EnvVersion: :Severity: 7 :Fields:
| name:"ProcessInputName" type:string value:"zerogw.stdout" |
name:"ExitStatus" type:integer value:0 | name:"Value" type:string
value:"4" | name:"Service" type:string value:"Zerogw" | name:"Metric"
type:string value:"zerogw.connections.total" |
In plain Lua I’d dump the result of |read_message("raw")| to stdout, add
some prints everywhere and see what happens inside but don’t know how to do.
Any clues on how I should debug such cases?
Thanks,
Timur
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
