On 11/22/2014 09:19 AM, Rishi Dhupar wrote:
Hello,
I have a couple questions and a suggestion.
1. If I have about 20 Heka's running and all collecting data and
sending it to one central Heka instance which does the alerting and
database writes; do we have any benchmarks on what Heka can do
ops/second in this scenario? I did see
https://github.com/mozilla-services/heka/wiki/How-to-convert-a-PayloadRegex-MultiDecoder-to-a-SandboxDecoder-using-an-LPeg-Grammar#comparison
which seem very low but I am hoping they are not what heka can do with
my use case I described.
No, afraid there aren't any benchmarks for this. There are countless
unspecified variables in your use case that would impact performance, so
there's really no way to produce a benchmark for the general case.
I will say that the benchmark that you linked to is related to fairly
complicated text parsing, and the main point of that bench is to show that the
sandbox based LPEG approach is over 3 times faster than the MultiDecoder /
PayloadRegexDecoder combination, the absolute numbers aren't all that helpful.
Heka is certainly capable of much higher throughput than is shown there.
2. The Schema InfluxDB Encoder that was just added is a great addition,
are there any plans on bringing over a similar feature set to StatMetric
InfluxDB Encoder -- being able to choose the series name to match my
existing convention would be great i.e. servers.{host}.system.cpu and
the skip fields would be nice to haves.
These features aren't on our short-list, nope, but they'd be a pretty good
getting-your-feet-wet project if you (or anyone) wanted to tackle it and open a
pull request.
One suggestion I have on increasing users and visibility of the project,
the getting started guide is great and can get a new user up with
something running in 15 minutes, if we could extend that and maybe show
heka monitoring cpu, memory, disk usage and maybe even run an external
app for like ntp status. I think it would show the users the other
dimensions that heka provide them.
Again, I'm open to including more examples, but currently I have my head down
on some major under-the-hood coding.
Heka seems like it could become the swiss arm knife for system metrics,
app metrics, log collection and alerting in a very easy deployable form
and config format. It looks like it should be able to replace collectd,
statsd and logstash like apps on a lot of systems. Thanks for the great
work.
That's our hope, yes. We'd love help from the community in getting the word
out. Glad you're finding it useful.
-r
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
