CodeHale doesn't handle extreme skew on measurements well last time I looked. For throughput, averages are great. For latency, you need very high percentiles to understand what is happening.
On Mon, Jun 16, 2014 at 6:00 PM, Michael Rose <mich...@fullcontact.com> wrote: > What kind of issues does Metrics have that leads you to recommend > HdrHistogram? > On Jun 16, 2014 6:57 PM, "Dan" <dcies...@hotmail.com> wrote: > >> Be careful when using Coda Hale's Metrics package when measuring latency. >> Consider using Gil Tene's >> High Dynamic Range Histogram instead: >> >> http://hdrhistogram.github.io/HdrHistogram/ >> >> -Dan >> >> ------------------------------ >> From: and...@parsely.com >> Date: Mon, 16 Jun 2014 18:20:11 -0400 >> Subject: Re: Extracting Performance Metrics >> To: user@storm.incubator.apache.org >> >> Also, I came across this presentation by Visible Measures which actually >> walks through a lot of great options covering most of what you want to know >> about: >> >> http://files.meetup.com/5809742/storm%20monitoring.pdf >> >> One other thing to be aware of is that in Storm 0.9.2 (forthcoming >> release), there is a new REST API used by the Storm UI for gathering some >> of these metrics: >> >> https://github.com/apache/incubator-storm/pull/101 >> https://issues.apache.org/jira/browse/STORM-205 >> >> >> On Mon, Jun 16, 2014 at 6:13 PM, Andrew Montalenti <and...@parsely.com> >> wrote: >> >> I haven't used it yet, but a lot of people get pointed to metrics_storm: >> >> https://github.com/ooyala/metrics_storm >> >> With this blog post that discusses it: >> >> http://engineering.ooyala.com/blog/open-sourcing-metrics-storm >> >> Michael Noll also has a nice blog post about streaming Storm 0.9 metrics >> to Graphite: >> >> >> http://www.michael-noll.com/blog/2013/11/06/sending-metrics-from-storm-to-graphite/ >> >> Currently, when we use Storm, we do a lot of custom metrics in Graphite >> using statsd, as described in this post (not about Storm, but about >> Graphite/statsd): >> >> http://codeascraft.com/2011/02/15/measure-anything-measure-everything/ >> >> >> >> >> On Mon, Jun 16, 2014 at 4:37 PM, Anis Nasir <aadi.a...@gmail.com> wrote: >> >> Dear all, >> >> I am running a cluster with 1 kafka + 1 nimbus + 10 supervisor + 1 >> zookeeper nodes. I am executing multiple topologies on the cluster and I >> want to extract different metrics that I am mentioning below. Can someone >> help me by recommending tools that I can use to extract this information. >> >> >> Per Topology >> - Throughput >> - Latency >> >> Per Spout or Bolt >> - Throughput >> - Latency >> - Execution Time >> - Queuing Time >> - Number of Messages Processed >> >> Regards >> Anis >> >> >> >>
