I can’t speak with completely certainty on every output plugin, but TcpOutputPlugin does buffer and re-try operations. Every pack that gets sent to to TcpOutputPlugin is written to disk before its sent off. If the connection fails for any reason, it will retry.
Internally this is achieved with BufferedOutput, and from a quick glance at the HttpOutput code in Heka 0.6, it doesn’t seem to use BufferedOutput. I’m not sure if this can be achieved without touching the Heka internals, but from what I’ve found its would be rather simple to have HttpOutputPlugin use BufferedOutput much like the TcpOutputPlugin does. Nimi On October 15, 2014 at 3:49:29 AM, Denis Shashkov ([email protected]) wrote: Hello! AFAIK, now heka doesn't guarantee message delivering in case of output plugin couldn't write message: - output plugins didn't buffer or re-try write operations, - there is no interior buffer between pipeline and output plugins (I know about channels, but they have finite length and they located in memory), - you cannot write a buffering filter (because you cannot write all messages back into pipeline). (Please, correct me if I wrong). I thought a lot about how not to lost messages if my storage (e.g. HTTP server) is unavailable. I decided it will be great if some output plugin will be special: if other outputs cannot write messages, this fallback output will write instead of them. Can I do this without touching pipeline code? WBR. _______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
