hello all,
I have 1 DC of 3 nodes in which is running Cassandra 3.11.3 with
consistency level ONE and Java 1.8.0_191.
Every day, there are many nodejs programs that send data to the cassandra's
cluster via NodeJs cassandra-driver.
Every day I got like 600k requests. Each request makes the server to:
1_ READ some data in Cassandra (by an id, usually I get 3 records),
2_ DELETE one of those records
3_ WRITE the data into Cassandra.
So every day I make many deletes.
Every day I find errors like:
"All host(s) tried for query failed. First host tried, 10.8.0.10:9042: Host
considered as DOWN. See innerErrors...."
"Server timeout during write query at consistency LOCAL_ONE (0 peer(s)
acknowledged the write over 1 required)...."
"Server timeout during write query at consistency SERIAL (0 peer(s)
acknowledged the write over 1 required)...."
"Server timeout during read query at consistency LOCAL_ONE (0 peer(s)
acknowledged the read over 1 required)...."
nodetool tablehistograms tells me this:
Percentile SSTables Write Latency Read Latency Partition Size
Cell Count
(micros) (micros) (bytes)
50% 8.00 379.02 1955.67 379022
8
75% 10.00 785.94 155469.30 654949
17
95% 12.00 17436.92 268650.95 1629722
35
98% 12.00 25109.16 322381.14 2346799
42
99% 12.00 30130.99 386857.37 3379391
50
Min 0.00 6.87 88.15 104
0
Max 12.00 43388.63 386857.37 20924300
179
in the 99% I noted that write and read latency is pretty high, but I don't
know how to improve that.
I can provide more statistics if needed.
Is there any improvement I can make to the Cassandra's configuration in
order to not to lose any data?
Thanks
Regards
Marco
