Re: Collecting Latency Metrics

2019-06-03 Thread shalom sagges
Thanks a lot for your comments. This mailing list is truly *the *definitive guide to Cassandra *. * The knowledge transferred here is invaluable. So just wanted to give a big shout out to anyone who is helping out here. Regards, On Thu, May 30, 2019 at 6:10 PM Jon Haddad wrote: > Yep. I would

Re: Collecting Latency Metrics

2019-05-30 Thread Jon Haddad
Yep. I would *never* use mean when it comes to performance to make any sort of decisions. I prefer to graph all the p99 latencies as well as the max. Some good reading on the topic: https://bravenewgeek.com/everything-you-know-about-latency-is-wrong/ On Thu, May 30, 2019 at 7:35 AM Chris

Re: Collecting Latency Metrics

2019-05-30 Thread Chris Lohfink
For what it is worth, generally I would recommend just using the mean vs calculating it yourself. It's a lot easier and averages are meaningless for anything besides trending anyway (which is really what this is useful for, finding issues on the larger scale), especially with high volume clusters

Re: Collecting Latency Metrics

2019-05-30 Thread Chris Lohfink
> > org.apache.cassandra.metrics.ClientRequest.Latency.Read these measure the > latency in milliseconds > Its actually in microseconds, unless calling the values() operation which gives the histogram in nanoseconds On Wed, May 29, 2019 at 4:34 PM Paul Chandler wrote: > There are various

Re: Collecting Latency Metrics

2019-05-30 Thread shalom sagges
Sorry for the duplicated emails but I just want to make sure I'm doing it correctly: To summarize, are both ways accurate or one is better than the other?

Re: Collecting Latency Metrics

2019-05-30 Thread shalom sagges
Thanks for your replies guys. I really appreciate it. @Alain, I use Graphite for backend on top of Grafana. But the goal is to move from Graphite to Prometheus eventually. I tried to find a direct way of getting a specific Latency metric in average and as Chris pointed out, then Mean value isn't

Re: Collecting Latency Metrics

2019-05-29 Thread shalom sagges
If I only send ReadTotalLatency to Graphite/Grafana, can I run an average on it and use "scale to seconds=1" ? Will that do the trick? Thanks! On Wed, May 29, 2019 at 5:31 PM shalom sagges wrote: > Hi All, > > I'm creating a dashboard that should collect read/write latency metrics on > C* 3.x.

Re: Collecting Latency Metrics

2019-05-29 Thread Paul Chandler
There are various attributes under org.apache.cassandra.metrics.ClientRequest.Latency.Read these measure the latency in milliseconds Thanks Paul www.redshots.com > On 29 May 2019, at 15:31, shalom sagges wrote: > > Hi All, > > I'm creating a dashboard that should collect read/write

Re: Collecting Latency Metrics

2019-05-29 Thread Alain RODRIGUEZ
Hello, This metric is available indeed: Most of the metrics available are documented here: http://cassandra.apache.org/doc/latest/operating/metrics.html For client requests (coordinator perspective latency): http://cassandra.apache.org/doc/latest/operating/metrics.html#client-request-metrics

Re: Collecting Latency Metrics

2019-05-29 Thread Chris Lohfink
To answer your question org.apache.cassandra.metrics:type=Table,name=ReadTotalLatency can give you the total local read latency in microseconds and you can get the count from the Latency read metric. If you are going to do that be sure to do it on the delta from previous query (new - last) for

Collecting Latency Metrics

2019-05-29 Thread shalom sagges
Hi All, I'm creating a dashboard that should collect read/write latency metrics on C* 3.x. In older versions (e.g. 2.0) I used to divide the total read latency in microseconds with the read count. Is there a metric attribute that shows read/write latency without the need to do the math, such as