Re: [heka] New "Getting Started" docs

2014-08-26 Thread Dieter Plaetinck
looks pretty great and complete. will give it a read - hopefully soon - and report back from a newbie perspective. On Tue, 26 Aug 2014 15:41:41 -0700 Rob Miller wrote: > Hi! > > It's been a long time coming, but I'm very happy to say that we've > finally added a "Getting Started" section to t

Re: [heka] Double Whammy!

2014-08-28 Thread Dieter Plaetinck
Thank you mozilla! On Thu, 28 Aug 2014 10:21:26 -0700 David Birdsong wrote: > Great job all! > > > On Thu, Aug 28, 2014 at 9:49 AM, Rob Miller wrote: > > > I'm here today to brighten up your Thursday with a double helping of new > > Heka releases. Heka v0.6.1 and v0.7.0 are available for you

Re: [heka] Change heka config during runtime

2014-10-06 Thread Dieter Plaetinck
On Mon, 06 Oct 2014 14:43:49 -0700 Rob Miller wrote: > That's not ready for use, though. Simply reloading the config is easy > enough. But in most cases Heka restarts so quickly, there's not much > benefit to reloading over restarting if the open connections are all > closed and need to be ree

[heka] heka statsd [multicore] performance

2014-10-18 Thread Dieter Plaetinck
Hi, with https://github.com/vimeo/statsdaemon, our statsd replacement in go (1.3) we are processing between 40~70k messages a second (10k timers/s, 500 gauges/s and the rest is counters), and: * cpu (Xeon E5-2697 v2 @ 2.70GHz) is typically 70~90% used with the occasional 100% (increasing GOMAXPR

Re: [heka] heka statsd [multicore] performance

2014-11-06 Thread Dieter Plaetinck
On Tue, 04 Nov 2014 11:22:00 -0800 Rob Miller wrote: > Getting caught up on backlog sorry for the delay. > > On 10/18/2014 01:06 PM, Dieter Plaetinck wrote: > > > > So I plan to do some experimenting with heka, but meanwhile I'ld like to > > hear from you.

Re: [heka] heka statsd [multicore] performance

2014-11-06 Thread Dieter Plaetinck
On Thu, 6 Nov 2014 09:30:48 -0500 Tom Cameron wrote: > But, I suspect the > cost of disassembling packets is higher than the math you're going to do. > This is made more true by the fact that you need shared (locked) structures > for your counts. i was thinking of having N workers, where every w

Re: [heka] heka statsd [multicore] performance

2014-11-07 Thread Dieter Plaetinck
On Thu, 6 Nov 2014 12:01:11 -0500 Tom Cameron wrote: > I suppose you could hash on the source address note that we would have to make sure that given keys are always processed together, i.e. if doing this, multiple sources can't send packets with the same key (typically people encode hostname

Re: [heka] heka statsd [multicore] performance

2014-11-07 Thread Dieter Plaetinck
On Fri, 07 Nov 2014 09:56:47 -0800 Rob Miller wrote: > On 11/07/2014 09:51 AM, Dieter Plaetinck wrote: > > anyway, i just did some crude testing. > > on an intel xeon E5-2690 @ 2.90GHz heka consumes 70~80% cpu typically, when > > i send around 75k timers/s and 40k counters/

Re: [heka] Any plans or thoughts on 1.0?

2015-04-27 Thread Dieter Plaetinck
On Fri, 24 Apr 2015 10:50:11 -0700 Rob Miller wrote: > My current thinking is that we'll let both of the APIs exist side by side for > a while. This would mean existing plugins could support buffering, albeit > with a performance hit, with only trivial code changes. Much improved > performance

[heka] interested in using heka for a reliable metrics pipeline. have some questions

2015-08-04 Thread Dieter Plaetinck
Hello everyone, I'm interested in using heka for our metrics pipeline. Basically, there would be 3 stages: 1. ingest: publicly exposed endpoints that authenticate sessions, take in metrics data, and pump it into rabbitmq 2. pull data out of rabbit as quickly as possible (i hear it doesn't do well

Re: [heka] interested in using heka for a reliable metrics pipeline. have some questions

2015-08-05 Thread Dieter Plaetinck
On Tue, 04 Aug 2015 11:49:38 -0700 Rob Miller wrote: > Answers in-line below, although I'm not sure you're going to like them... thanks as always. > > I have started work on a heka output and encoder plugin for kairosdb's rest > > endpoint. > Is a custom output really needed? Ideally you'd jus

Re: [heka] State and future of Heka

2016-05-06 Thread Dieter Plaetinck
On Fri, 6 May 2016 10:51:01 -0700 Rob Miller wrote: > Hi everyone, > > I'm lng overdue in sending out an update about the current state of > and plans for Heka. Unfortunately, what I have to share here will > probably be disappointing for many of you, and it might impact whether > or not