Hi all,
I'm happy to be able to say that Heka v0.9.0 has been officially
released, at last! This is a significant release, containing a
considerable number of new features and bug fixes. It also has a handful
of significant breaking changes, so upgrading from prior versions will
require tweaks to existing configuration, and should be done carefully
and tested thoroughly. I'll cover the issues briefly here, but you
should also check the changelog and the updated docs for the whole story.
The one issue that impacts everybody is the introduction of Splitter
plugins. Splitters are embedded within Inputs, and they're responsible
for watching the incoming data stream and deciding where it should be
sliced to generate discrete chunks of data that can be decoded and
injected into the Heka pipeline. In v0.8 and earlier, this job was
handled on an ad-hoc basis by the inputs themselves; now it's available
to all inputs and lots of duplicate code has been removed.
What this means in practice is that all of the 'parser_type',
'delimiter', and 'delimiter_location' config settings for various input
plugins are now gone. These are replaced by a single 'splitter' setting
that will refer to an separate section containing the splitter
configuration, much like the already existing 'decoder' and 'encoder'
behavior. Heka automatically registers some splitters for you, including
a NullSplitter, a HekaFramingSplitter, and a TokenSplitter (using the
default newline delimiter '\n'). All available splitters and their
configuration options are described in the docs:
http://hekad.readthedocs.org/en/v0.9.0/config/splitters/index.html
If you're just using existing input plugins, then converting your config
to use Splitter plugins should be quite straightforward. If you've
implemented your own inputs, they'll might still work, but you'll almost
certainly want to update them to take advantage of the new APIs we've
exposed that take the splitting and decoding burdens out of your hands.
How this works is more detailed than I want to cover here, but you
should find all of the info that you need in the updated developer docs
(http://hekad.readthedocs.org/en/v0.9.0/developing/plugin.html#inputs).
If you have any questions or problems, please feel free to ask on this
mailing list or in the #heka channel on irc.mozilla.org.
Beyond this big one, there are a lot of other exciting improvements,
especially with the Lua sandbox. The sandbox now supports loading
dynamic library loading, so you can pull in new Lua modules, implemented
in C, without having to recompile Heka. Also we've introduced
SandboxInput and SandboxOutput plugins, so it's now possible to do any
arbitrary I/O from within Lua code. We've also added the ability to
encode and decode Heka protobuf messages entirely within the sandbox,
which greatly expands the range of new behavior that can be introduced
without needing to recompile or redeploy the core Heka binary.
There's one unfortunate point to mention. Due to a bug in Go's
interfacing with C, the new version is currently not working on Windows.
We've reported the issue to the Go team
(https://github.com/golang/go/issues/9754), but as of yet there's been
no resolution. We did just get notice today that a patch has landed that
*may* resolve this issue (https://go-review.googlesource.com/#/c/5711/),
we'll be testing that soon to see if it helps. Fingers crossed!
Okay, I've blathered on long enough. Check the changelog
(https://github.com/mozilla-services/heka/blob/v0.9.0/CHANGES.txt) for
the complete list of what we've been up to. Check the release page
(https://github.com/mozilla-services/heka/releases/tag/v0.9.0) for the
available binary packages. Test carefully before deploying, and report
any issues or bugs that you discover in the issue tracker
(https://github.com/mozilla-services/heka/issues).
And, most importantly, thanks for using Heka, and have a great day.
Cheers!
-r
_______________________________________________
Heka mailing list
Heka@mozilla.org
https://mail.mozilla.org/listinfo/heka
