Re: nodetool tablestats reporting local read count of 0, incorrectly
On 2017-04-03 12:42 (-0700), Voytek Jarnot wrote: > Continuing to grasp at straws... > > Is it possible that indexing is modifying the read path such that the > tablestats/tablehistograms output is no longer trustworthy? I notice more > realistic "local read count" numbers on tables which do not utilize SASI. > > Would greatly appreciate any thoughts, > Thanks. > Not the most ridiculous thought, though that would not be expected. Would be great if you could open a JIRA.
Re: nodetool tablestats reporting local read count of 0, incorrectly
Continuing to grasp at straws... Is it possible that indexing is modifying the read path such that the tablestats/tablehistograms output is no longer trustworthy? I notice more realistic "local read count" numbers on tables which do not utilize SASI. Would greatly appreciate any thoughts, Thanks. On Mon, Apr 3, 2017 at 9:56 AM, Voytek Jarnot wrote: > Further info - tablehistograms reports zeros for all percentiles for Read > Latency; tablestats also reports really low numbers for Bloom filter usage > (3-4 KiB, depending on node, whereas I'd expect orders of magnitude more > given other - less accessed - tables in this keyspace). This is the most > written-to and read-from table in the keyspace, seems to keep up with > tracking of writes, but not reads. > > Full repair on this table is the only thing I can think of; but that's a > guess and doesn't get me any closer to understanding what has happened. > > On Fri, Mar 31, 2017 at 11:11 PM, Voytek Jarnot > wrote: > >> Cassandra 3.9 >> >> Have a keyspace with 5 tables, one of which is exhibiting rather poor >> read performance. In starting an attempt to get to the bottom of the >> issues, I noticed that, when running nodetool tablestats against the >> keyspace, that particular table reports "Local read count: 0" on all nodes >> - which is incorrect. >> >> It tallies "Local write count", presumably correctly, as at least it's >> not 0. Other tables in the keyspace do not exhibit this behavior, as they >> provide non-zero numbers for both read and write values. >> >> Is this perhaps indicative of a deeper issue with this particular table? >> >> Thank you. >> > >
Re: nodetool tablestats reporting local read count of 0, incorrectly
Further info - tablehistograms reports zeros for all percentiles for Read Latency; tablestats also reports really low numbers for Bloom filter usage (3-4 KiB, depending on node, whereas I'd expect orders of magnitude more given other - less accessed - tables in this keyspace). This is the most written-to and read-from table in the keyspace, seems to keep up with tracking of writes, but not reads. Full repair on this table is the only thing I can think of; but that's a guess and doesn't get me any closer to understanding what has happened. On Fri, Mar 31, 2017 at 11:11 PM, Voytek Jarnot wrote: > Cassandra 3.9 > > Have a keyspace with 5 tables, one of which is exhibiting rather poor read > performance. In starting an attempt to get to the bottom of the issues, I > noticed that, when running nodetool tablestats against the keyspace, that > particular table reports "Local read count: 0" on all nodes - which is > incorrect. > > It tallies "Local write count", presumably correctly, as at least it's not > 0. Other tables in the keyspace do not exhibit this behavior, as they > provide non-zero numbers for both read and write values. > > Is this perhaps indicative of a deeper issue with this particular table? > > Thank you. >
nodetool tablestats reporting local read count of 0, incorrectly
Cassandra 3.9 Have a keyspace with 5 tables, one of which is exhibiting rather poor read performance. In starting an attempt to get to the bottom of the issues, I noticed that, when running nodetool tablestats against the keyspace, that particular table reports "Local read count: 0" on all nodes - which is incorrect. It tallies "Local write count", presumably correctly, as at least it's not 0. Other tables in the keyspace do not exhibit this behavior, as they provide non-zero numbers for both read and write values. Is this perhaps indicative of a deeper issue with this particular table? Thank you.