Hi Simon, I need to do some processing on Fields and that’s why I call read_message(“raw”). I use exactly the same construct in several plugins and it works fine. What’s strange here is I see this behaviour only on a single host.
Thanks for the clue! I’ll check if it returns nil. Why could it be like that? Thanks, Timur On 12 Oct 2015 at 17:56:17, Simon Pasquier ([email protected]) wrote: Hello Timur, I'm not sure why you're calling read_message("raw") in your decoder. Usually a Lua decoder calls read_message("Payload") which has been filled by the input plugin, processes what's in there to set fields in the Heka message's table and finally calls the inject_message() function. IIRC read_message("raw") returns the Protobuf encoding of the Heka message but the encoding is done after the decoder has done its job. In your case, read_message() probably returns nil which is why the decode_message() function complains. HTH Simon On Mon, Oct 12, 2015 at 4:17 PM, Timur Batyrshin <[email protected]> wrote: Hi, I’ve been writing a decoder for myself and have hit the following issue which I can’t understand. When I start Heka it produces the following error message in logs: 2015/10/12 13:54:47 SubDecoder ‘zerogw-zerogw_decoder-stdout-zerogw_rotate_fields’ error: FATAL: process_message() /usr/share/heka/lua_decoders/rotate_fields.lua:30: bad argument #0 to ‘decode_message’ (must have one string argument) At the same time the code for decoder is the following: -- the only lines above are comments which are skipped metric_field = read_config("metric_field") or "metric" value_field = read_config("value_field") or "value" function process_message() local fields = {} raw = read_message("raw”) # line 29 msg = decode_message(raw) # line 30 -- other part of code is probably irrelevant as crash is seen in the above line (I’ve tried writing that as decode_message(read_message("raw”)) with the same effect) What’s really weird is the exactly the same decoder works fine on other hosts. I’m using the following Heka config: [zerogw] type = "ProcessInput" ticker_interval = 0 splitter = "on_newline" decoder = "zerogw_decoder" stdout = true stderr = false [zerogw.command.0] bin = "/usr/local/bin/zerogw_collector.py" args = ["-s", "tcp://127.0.0.1:5111"] [on_newline] type = "TokenSplitter" delimiter = "\n" [estp_decoder] type = "PayloadRegexDecoder" match_regex = '^(?P<Name>[^\s]+) (?P<Timestamp>\d+) (?P<Value>\d+)' timestamp_layout = "Epoch" [estp_decoder.message_fields] Service = "Zerogw" Metric = "%Name%" Value = "%Value%" [zerogw_decoder] type = "MultiDecoder" subs = ["estp_decoder", "zerogw_rotate_fields"] cascade_strategy = "all" [zerogw_rotate_fields] type = "SandboxDecoder" filename = "lua_decoders/rotate_fields.lua" [zerogw_rotate_fields.config] metric_field = "Metric" value_field = "Value" zerogw_collector.py produces about a dozen of lines to stdout every 5 seconds in the format as seen in message payload (see below). As MultiDecoder has cascade_strategy = "all" Heka dumps messages processed by the first decoder in the chain to stdout which are the following: 2015/10/12 14:04:22 :Timestamp: 2015-10-12 14:04:22 +0000 UTC :Type: ProcessInput :Hostname: t-eu-zgw :Pid: 5212 :Uuid: 2c7deb23-7961-49dc-8f57-da716d851439 :Logger: zerogw :Payload: zerogw.connections.total 1444658662 4 :EnvVersion: :Severity: 7 :Fields: | name:"ProcessInputName" type:string value:"zerogw.stdout" | name:"ExitStatus" type:integer value:0 | name:"Value" type:string value:"4" | name:"Service" type:string value:"Zerogw" | name:"Metric" type:string value:"zerogw.connections.total" In plain Lua I’d dump the result of read_message("raw") to stdout, add some prints everywhere and see what happens inside but don’t know how to do. Any clues on how I should debug such cases? Thanks, Timur _______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
