READ Operation with CL=EACH_QUORUM succeed when a DC is down (RF=3)
-------------------------------------------------------------------
Key: CASSANDRA-3272
URL: https://issues.apache.org/jira/browse/CASSANDRA-3272
Project: Cassandra
Issue Type: Bug
Reporter: Jackson Chung
"READ EACH_QUORUM: Returns the record with the most recent timestamp once
a quorum of replicas in each data center of the cluster has responded."
In other words, if a DC is down and the QUORUM could not be reached on that DC,
read should fail.
test case:
- Cassandra version 0.8.6:
INFO [main] 2011-09-28 22:26:24,297 StorageService.java (line 371) Cassandra
version: 0.8.6
- 6-node cluster with 2 DC and 3 node each. RF=3 in each DC:
[default@Keyspace3] describe keyspace;
Keyspace: Keyspace3:
Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy
Durable Writes: true
Options: [DC2:3, DC1:3]
Column Families:
ColumnFamily: test
Key Validation Class: org.apache.cassandra.db.marshal.BytesType
Default column value validator: org.apache.cassandra.db.marshal.BytesType
Columns sorted by: org.apache.cassandra.db.marshal.BytesType
Row cache size / save period in seconds: 0.0/0
Key cache size / save period in seconds: 200000.0/14400
Memtable thresholds: 1.0875/1440/232 (millions of ops/minutes/MB)
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 1.0
Replicate on write: true
Built indexes: []
all nodes are up, insert a row:
$ nodetool -h localhost ring
Address DC Rack Status State Load Owns Token
141784319550391026443072753096570088106
10.34.79.179 DC1 RAC1 Up Normal 11.13 KB 16.67% 0
10.34.70.163 DC2 RAC1 Up Normal 11.14 KB 16.67%
28356863910078205288614550619314017621
10.35.81.147 DC1 RAC1 Up Normal 11.14 KB 16.67%
56713727820156410577229101238628035242
10.84.233.170 DC2 RAC1 Up Normal 11.14 KB 16.67%
85070591730234615865843651857942052864
10.195.201.236 DC1 RAC1 Up Normal 11.14 KB 16.67%
113427455640312821154458202477256070485
10.118.147.73 DC2 RAC1 Up Normal 11.14 KB 16.67%
141784319550391026443072753096570088106
- insert a value
[default@Keyspace3] set
test[utf8('test-key-1')][utf8('test-col')]=utf8('test-value');
Value inserted.
sanity check (cli connects to a node in DC1) :
[default@Keyspace3] consistencylevel as EACH_QUORUM;
Consistency level is set to 'EACH_QUORUM'.
[default@Keyspace3] get test[utf8('test-key-1')];
=> (column=746573742d636f6c, value=test-value, timestamp=1317249361722000)
Returned 1 results
shut down DC2:
$ nodetool -h localhost ring
Address DC Rack Status State Load Owns
Token
141784319550391026443072753096570088106
10.34.79.179 DC1 RAC1 Up Normal 51.86 KB 16.67%
0
10.34.70.163 DC2 RAC1 Down Normal 51.88 KB 16.67%
28356863910078205288614550619314017621
10.35.81.147 DC1 RAC1 Up Normal 47.5 KB 16.67%
56713727820156410577229101238628035242
10.84.233.170 DC2 RAC1 Down Normal 51.88 KB 16.67%
85070591730234615865843651857942052864
10.195.201.236 DC1 RAC1 Up Normal 47.5 KB 16.67%
113427455640312821154458202477256070485
10.118.147.73 DC2 RAC1 Down Normal 51.88 KB 16.67%
141784319550391026443072753096570088106
[default@Keyspace3] get test[utf8('test-key-1')];
=> (column=746573742d636f6c, value=746573742d76616c7565,
timestamp=1317249361722000)
Returned 1 results.
tried with pycassaShell:
>>> col_fam.get('test-key-1',read_consistency_level=pycassa.ConsistencyLevel.EACH_QUORUM)
OrderedDict([('test-col', 'test-value')])
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
