Hi,
I’ve been trying to do some buffering and splitting of messages in an encoder
and I’ve run in a following issue.
Looks like calling inject_payload() several times in a single run of encoder’s
process_message() doesn’t work and only the last message is injected.
Is it so?
If yes, any ideas how can I solve the following task?
* I have incoming messages consisting of up to several hundreds of custom fields
* I’m encoding the fields into JSON array like [{“metric": “foobar.count",
"value": 1000}, {“metric": “bazqux.rate”, “value”: 10}] and sending them out
via HTTP to a different service
* I’d like to keep outgoing HTTP messages relatively small (no more than a few
dozens of fields) and for that reason I’m going to split messages with too many
fields into several smaller ones
For that I was going to do the splitting inside encoder attached to HTTPOutput
and calling inject_payload() several times but it seems that only the last
message is sent out.
I’ve tried setting max_process_inject = 1000 in [hekad] config section but this
didn’t help either.
I could try using a filter to split messages and reinject them back to Heka
(haven’t tested yet if it is going to work in this scenario) but this looks
like a bit overweight to me.
Any other ideas how to handle the task?
Thanks,
Timur_______________________________________________
Heka mailing list
Heka@mozilla.org
https://mail.mozilla.org/listinfo/heka
