julia zhang created CASSANDRA-10430:
---------------------------------------

             Summary: "Load" report from "nodetool status" is inaccurate
                 Key: CASSANDRA-10430
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10430
             Project: Cassandra
          Issue Type: Bug
          Components: Tools
         Environment: Cassandra v2.1.9 running on 6 node Amazon AWS, vnodes 
enabled. 
            Reporter: julia zhang


After running an incremental repair, nodetool status report unbalanced load 
among cluster. 

$ nodetool status mykeyspace
==========================
||Status|| Address         ||Load           ||Tokens  ||Owns (effective)  
||Host ID ||  Rack ||                         
|UN  |10.6.0.246  |1.13 TB       |256    |48.5%            
|a4477534-a5c6-4e3e-9108-17a69aebcfc0|  RAC1|
|UN  |10.6.0.188  |2.58 TB       |256     |50.5%             
|1a7c3864-879f-48c5-8dde-bc00cf4b23e6  |RAC2|
|UN  |10.6.0.189  |1.49 TB       |256     |51.5%             
|27df5b30-a5fc-44a5-9a2c-1cd65e1ba3f7  |RAC1|
|UN  |10.6.0.186  |250.97 GB  |256     |51.9%             
|9898a278-2fe6-4da2-b6dc-392e5fda51e6  |RAC3|
|UN  |10.6.0.187  |1.88 TB      |256     |49.5%             
|04aa9ce1-c1c3-4886-8d72-270b024b49b9  |RAC2|
|UN  |10.6.0.185  |1.3 TB        |256     |48.1%             
|6d5d48e6-d188-4f88-808d-dcdbb39fdca5  |RAC3|

There is no hints in the cluster and report remains the same after running 
"nodetool cleanup" on each node. "nodetool cfstats" shows number of keys are 
evenly distributed and Cassandra data physical disk on each node report about 
the same usage. 

"nodetool status" report this inaccurate large storage load until we restart 
each node, after the restart, it reports correct "Load". 

We did not see this behavior until upgrade to v2.1.9



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to