The default limit is 8MiB, it can be adjusted with the memory_limit
configuration parameter.
https://hekad.readthedocs.org/en/latest/config/common_sandbox_parameter.html#config-common-sandbox-parameters
I would recommend setting it to at least 10x the size of your largest
document. The memory is not reserved so even if you make it 100MiB you
will only pay for what you use (you can look at the sandbox stats for the
current and peak usage).
The sandbox is a very good choice: its JSON parser is very efficient and
multiple times faster than the one in Go.
Trink
On Wed, Apr 8, 2015 at 3:07 PM, M GS <mgs...@gmail.com> wrote:
> I am trying to create stats based about 2mb json response from
> elasticsearch. Our goal is to be able to graph and alert on shard state
> changes.
>
> I have an HTTPInput that queries /_cluster/state, but when im using my
> decoder to try to extract the state of each shard, my plugin encounters a
> fatal error:
>
> Decoder 'ESClusterStatusInput-ESClusterStatusDecoder' error: FATAL:
> process_message() not enough memory
>
> Below is my code:
>
> local resp = read_message("Payload")
> local data = cjson.decode(resp) -- it seems like here is where the
> error is thrown
>
> for _, index in pairs(data.routing_table.indices) do
> for _, shards in pairs(index.shards) do
> for _, shard in pairs(shards) do
> msg.Fields[shard.state] = msg.Fields[shard.state]+1
> end
> end
> end
>
> I tried with the same data outside of heka and it works, so I am assuming
> its a limit imposed on SandboxDecoders. My question is that since what im
> doing is probably pretty inefficient, is it even a good idea to be doing
> this kind of thing in general? Should I be doing this in go instead?
>
>
> Thanks!
>
> _______________________________________________
> Heka mailing list
> Heka@mozilla.org
> https://mail.mozilla.org/listinfo/heka
>
>
_______________________________________________
Heka mailing list
Heka@mozilla.org
https://mail.mozilla.org/listinfo/heka
