Nimi is right that the TcpOutput actually does buffer messages to disk.
Originally that functionality was built directly in to the TcpOutput,
but we later abstracted it out so it could be used by other output plugins.
What hasn't been mentioned so far is that we have plans to change the
interaction btn encoders and outputs and, as part of that, we plan on
making the output buffering automatically available as a configuration
option for *every* output. There are already a couple of issues open to
capture this:
https://github.com/mozilla-services/heka/issues/930
and:
https://github.com/mozilla-services/heka/issues/1103
which contains this relevant comment:
https://github.com/mozilla-services/heka/issues/1103#issuecomment-58548339
This will be a fair amount of work. It is all intended to land before we
release Heka 1.0, which is targeted for January 2015.
-r
On 10/15/2014 03:48 AM, Denis Shashkov wrote:
Hello!
AFAIK, now heka doesn't guarantee message delivering in case of output
plugin couldn't write message:
- output plugins didn't buffer or re-try write operations,
- there is no interior buffer between pipeline and output plugins (I
know about channels, but they have finite length and they located in
memory),
- you cannot write a buffering filter (because you cannot write all
messages back into pipeline).
(Please, correct me if I wrong).
I thought a lot about how not to lost messages if my storage (e.g. HTTP
server) is unavailable. I decided it will be great if some output plugin
will be special: if other outputs cannot write messages, this fallback
output will write instead of them.
Can I do this without touching pipeline code?
WBR.
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
