[
https://issues.apache.org/jira/browse/CASSANDRA-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15306185#comment-15306185
]
Jonathan Ellis commented on CASSANDRA-11738:
--------------------------------------------
bq. a measured latency can be influenced by a badly timed GC (e.g. G1 running
with a 500ms goal that sometimes has "valid" STW phases of up to 300/400ms).
True enough, but that's actually okay for our use case here. We prefer to use
*actual* latency, so we only need the *estimate* when there is no actual
available, i.e., when other coordinators stopped routing requests to us because
the actual was high. The job of the estimate is to let the other coordinators
know (when it gets low again) that they can resume sending us requests.
bq. Compactions and GCs can kick in every time anyway.
Right, but I see these as two different categories. GC STW lasts for fractions
of a second, while compaction can last minutes or even hours for a large STCS
job. So trying to route around GC is futile, but routing around compaction is
not.
bq. Just as an idea: a node can request a ping-response from a node it sends a
request to
If possible, I'd prefer to make this follow the existing "push" paradigm, via
gossip, for simplicity.
I had two ideas along those lines:
# Give up on computing a latency number in favor of other "load" metrics. The
coordinator can then extrapolate latency by comparing that number to other
nodes with similar load.
# Just brute force it: run SELECT * LIMIT 1 every 10s and report the latency
averaged across a sample of user tables.
> Re-think the use of Severity in the DynamicEndpointSnitch calculation
> ---------------------------------------------------------------------
>
> Key: CASSANDRA-11738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11738
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Jeremiah Jordan
> Assignee: Jonathan Ellis
> Fix For: 3.x
>
>
> CASSANDRA-11737 was opened to allow completely disabling the use of severity
> in the DynamicEndpointSnitch calculation, but that is a pretty big hammer.
> There is probably something we can do to better use the score.
> The issue seems to be that severity is given equal weight with latency in the
> current code, also that severity is only based on disk io. If you have a
> node that is CPU bound on something (say catching up on LCS compactions
> because of bootstrap/repair/replace) the IO wait can be low, but the latency
> to the node is high.
> Some ideas I had are:
> 1. Allowing a yaml parameter to tune how much impact the severity score has
> in the calculation.
> 2. Taking CPU load into account as well as IO Wait (this would probably help
> in the cases I have seen things go sideways)
> 3. Move the -D from CASSANDRA-11737 to being a yaml level setting
> 4. Go back to just relying on Latency and get rid of severity all together.
> Now that we have rapid read protection, maybe just using latency is enough,
> as it can help where the predictive nature of IO wait would have been useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
