[
https://issues.apache.org/jira/browse/CASSANDRA-13736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiangzhou xia updated CASSANDRA-13736:
--
Description:
When we testing atomic batch in production traffic, we found that p99 latency
in atomic batch write is 2x-3x worse than 2.2.
After debuging, we found that the regression is causing by CASSANDRA-9673. This
patch changed consistency level in batchlog store from ONE to TWO.
[~iamaleksey] think only block for one batchlog message is a bug in batchlog
and change it to block for two in CASSANDRA-9673, I think it's actually a very
good optimization to reduce latency.
Set the consistency to one will decrease the possibility of slow data node (GC,
long message queue, etc) affect the latency of atomic batch. In our shadow
cluster, when we change consistency from two to one, we notice a 2x-3x p99
latency drop in atomic batch.
was:we notice that changing consistency level from ONE to TWO dramatically
increased p99 latency in 3.0 atomic batch. [~iamaleksey] think only block for
one batchlog message is a bug in batchlog and change it to block for two in
CASSANDRA-9673, I think it's actually a very good optimization to reduce
latency. Set the consistency to one will decrease the possibility of slow data
node (GC, long message queue, etc) affect the latency of atomic batch. In our
shadow cluster, when we change consistency from two to one, we notice a 2x-3x
p99 latency drop in atomic batch.
Summary: CASSANDRA-9673 cause atomic batch p99 increase 3x (was:
consistency level change in batchlog send from CASSANDRA-9673 cause atomic
batch p99 increase 3x)
> CASSANDRA-9673 cause atomic batch p99 increase 3x
> -
>
> Key: CASSANDRA-13736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13736
> Project: Cassandra
> Issue Type: Bug
>Reporter: xiangzhou xia
>Assignee: xiangzhou xia
>
> When we testing atomic batch in production traffic, we found that p99 latency
> in atomic batch write is 2x-3x worse than 2.2.
> After debuging, we found that the regression is causing by CASSANDRA-9673.
> This patch changed consistency level in batchlog store from ONE to TWO.
> [~iamaleksey] think only block for one batchlog message is a bug in batchlog
> and change it to block for two in CASSANDRA-9673, I think it's actually a
> very good optimization to reduce latency.
> Set the consistency to one will decrease the possibility of slow data node
> (GC, long message queue, etc) affect the latency of atomic batch. In our
> shadow cluster, when we change consistency from two to one, we notice a 2x-3x
> p99 latency drop in atomic batch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
