Hi there,
This is to let everyone know that soon (possibly tomorrow) the long
overdue work that I've been doing to support disk buffering for all
filter and output plugins will be getting merged back to the dev branch.
This is a significant breaking change, all custom Go filter and output
plugins will need to be changed in order to continue to work.
I've taken considerable pains to try to ease the transition. For the
entire 0.10.x series of Heka releases, we will be straddling two sets of
APIs for all filter and output plugins. One of the APIs is very similar
to what already exists. If you have a lot of custom plugins that you
need to update, you should be able to make minimal changes to them to
get them working with the next release. This is documented here:
http://hekad.readthedocs.org/en/router_buffering/developing/old_apis.html#older-apis
Updating your plugins to the transitional APIs will get them working,
and they'll support the new disk buffering features, but you may see
sub-optimal performance when disk buffering is in effect. (If you don't
turn on the buffering, performance will be unaffected.)
In order to get the best performance with disk buffering, and in order
to work with 0.11 versions of Heka and beyond, you'll need to make much
more deep changes to your custom plugins, to support the new APIs,
documented here for filters:
http://hekad.readthedocs.org/en/router_buffering/developing/plugin.html#filters
And here for outputs:
http://hekad.readthedocs.org/en/router_buffering/developing/plugin.html#outputs
You'll want to read those docs in detail, but the short version is that
going forward Go plugins will look a lot more like sandboxed Lua plugins
do now. Instead of interacting directly with channels, you'll be
implementing ProcessMessage and TimerEvent methods on your plugins that
will be called as needed.
I haven't been in the habit of making beta releases so far, but because
these changes are so substantial my plan is to make at least one 0.10.0
beta before actually putting out a 0.10.0 final. I expect there will be
a few issues, and I'm hoping that many of you will help us out by
testing and reporting the issues you find so we can get the new code
paths well baked before calling it a final release. In particular I'd
love for folks to exercise the TcpOutput and the ElasticSearchOutput,
both of which have been updated to the newer API style.
Thanks a bunch,
-r
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
