[ https://issues.apache.org/jira/browse/CASSANDRA-12947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Philip Thompson updated CASSANDRA-12947: ---------------------------------------- Fix Version/s: 3.x > Repair not replicating data > --------------------------- > > Key: CASSANDRA-12947 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12947 > Project: Cassandra > Issue Type: Bug > Environment: Cassandra 3.7 > Single DC > 7 Nodes > RF 3 > NetworkTopologyStrategy > OS: Ubuntu > Reporter: Malte Pickhan > Fix For: 3.x > > > We experienced strange behaviour of our C* cluster last week. > In the logs we've seen multiple requests in the logs where we the values read > from Cassandra actually have been 'null'. > When running the query on a node, we sometimes received the result and > sometimes not. > When setting the CL to LOCAL_QUORUM everything was fine. So far nothing > unusual, probably the dataset wasn't replicated to one of the nodes. > When turning on the tracing and running the query there was following > intersting line: > {quote} > Initiating read-repair [SharedPool-Worker-2] | 2016-11-18 10:17:47.528000 | > $PUBLIC_IP | 126 | 127.0.0.1 > Digest mismatch: org.apache.cassandra.service.DigestMismatchException: > Mismatch for key DecoratedKey(-5887526567589486157, > 3130333031303338383436303937) (db1e86d507513ff12ba95f0eff984b60 vs > d41d8cd98f00b204e9800998ecf8427e) [ReadRepairStage:1] > {quote} > This is probably related to CASSANDRA-12090? > The interesting part is, after that we've run a 'nodetool repair -pr', after > that the behaviour was still the same and the data randomly not available, > depending on which node was hit. > Only after running a 'nodetool repair -pr -full' the issue was gone. > Did we miss something here? The point that's bothering me is that the dataset > was not replicated. > Worth to note is probably that some weeks ago we've hit the bug > CASSANDRA-12694 and fixed it by scrubbing some tables. > Any hints/help are appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)