[ https://issues.apache.org/jira/browse/CASSANDRA-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15360322#comment-15360322 ]
Jeremiah Jordan commented on CASSANDRA-11740: --------------------------------------------- Make sure you remove the PropertyFileSnitch config file once all nodes are using GPFS. GPFS can fall back to the PFS config in some cases. > Nodes have wrong membership view of the cluster > ----------------------------------------------- > > Key: CASSANDRA-11740 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11740 > Project: Cassandra > Issue Type: Bug > Reporter: Dikang Gu > Assignee: Joel Knighton > Fix For: 2.2.x, 3.x > > > We have a few hundreds nodes across 3 data centers, and we are doing a few > millions writes per second into the cluster. > The problem we found is that there are some nodes (>10) have very wrong view > of the cluster. > For example, we have 3 data centers A, B and C. On the problem nodes, in the > output of the 'nodetool status', it shows that ~100 nodes are not in data > center A, B, or C. Instead, it shows nodes are in DC1, and rack r1, which is > very wrong. And as a result, the node will return wrong results to client > requests. > {code} > Datacenter: DC1 > =============== > Status=Up/Down > / State=Normal/Leaving/Joining/Moving > – Address Load Tokens Owns Host ID Rack > UN 2401:db00:11:6134:face:0:1:0 509.52 GB 256 ? > e24656ac-c3b2-4117-b933-a5b06852c993 r1 > UN 2401:db00:11:b218:face:0:5:0 510.01 GB 256 ? > 53da2104-b1b5-4fa5-a3dd-52c7557149f9 r1 > UN 2401:db00:2130:5133:face:0:4d:0 459.75 GB 256 ? > ef8311f0-f6b8-491c-904d-baa925cdd7c2 r1 > {code} > We are using GossipingPropertyFileSnitch. > Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)