No, I mean that SandboxEncoders don't at this point support multiple inject calls for each input message. You can only call inject once. If you want smaller input, you'll have to do the work in a filter.
-r On 01/19/2016 11:53 AM, Timur Batyrshin wrote:
My point was not accumulating strings in output buffer but injecting several messages instead to make them smaller. Or do you mean that inject_message() run several times in a single run should work? I’ll try that — haven’t done that yet. Timur On 19 Jan 2016 at 22:50:28, Rob Miller (rmil...@mozilla.com <mailto:rmil...@mozilla.com>) wrote: > Yes, that's right. A SandboxEncoder can only make a single inject > call. If you call `inject_message`, then you will be emitting a > protobuf encoded Heka message. If you call `inject_payload`, then you > will be emitting a UTF8 encoded set of bytes. If you want to > accumulate multiple string values in the output buffer, you can use > `add_to_payload` before a final `inject_payload` call. > > -r > > > On 01/16/2016 12:08 PM, Timur Batyrshin wrote: > > An encoder I was using for reference: > > > > function process_message() > > inject_payload("msg1", "", "msg1") > > inject_payload("msg2", "", "msg2") > > return 0 > > end > > > > This produces only “msg2”. > > > > > > Timur > > > > > > On 16 Jan 2016 at 22:09:35, Timur Batyrshin (timur.batyrs...@cinarra.com > > <mailto:timur.batyrs...@cinarra.com>) wrote: > > > > > Hi, > > > > > > > > > I’ve been trying to do some buffering and splitting of messages in an > > > encoder and I’ve run in a following issue. > > > > > > Looks like calling inject_payload() several times in a single run of > > > encoder’s process_message() doesn’t work and only the last message is > > > injected. > > > Is it so? > > > > > > If yes, any ideas how can I solve the following task? > > > * I have incoming messages consisting of up to several hundreds of > > > custom fields > > > * I’m encoding the fields into JSON array like [{“metric": > > > “foobar.count", "value": 1000}, {“metric": “bazqux.rate”, “value”: > > > 10}] and sending them out via HTTP to a different service > > > * I’d like to keep outgoing HTTP messages relatively small (no more > > > than a few dozens of fields) and for that reason I’m going to split > > > messages with too many fields into several smaller ones > > > > > > For that I was going to do the splitting inside encoder attached to > > > HTTPOutput and calling inject_payload() several times but it seems > > > that only the last message is sent out. > > > I’ve tried setting max_process_inject = 1000 in [hekad] config section > > > but this didn’t help either. > > > > > > I could try using a filter to split messages and reinject them back to > > > Heka (haven’t tested yet if it is going to work in this scenario) but > > > this looks like a bit overweight to me. > > > > > > Any other ideas how to handle the task? > > > > > > > > > Thanks, > > > Timur > > > > > > _______________________________________________ > > Heka mailing list > > Heka@mozilla.org > > https://mail.mozilla.org/listinfo/heka > > >
_______________________________________________ Heka mailing list Heka@mozilla.org https://mail.mozilla.org/listinfo/heka
