When I did the upgrade from 1.0.9 to 1.1.6, I had this same issue. And then I fixed it with the following steps below, in each of the nodes.
[default@unknown] use system; Authenticated to keyspace: system [default@system] list HintsColumnFamily; Using default limit of 100 Using default column limit of 100 ------------------- RowKey: 55555555555555555555555555555554 1 Row Returned. Elapsed time: 84 msec(s). [default@system] del HintsColumnFamily[55555555555555555555555555555554]; system keyspace is not user-modifiable. InvalidRequestException(why:system keyspace is not user-modifiable.) at org.apache.cassandra.thrift.Cassandra$remove_result.read(Cassandra.java:18213) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.cassandra.thrift.Cassandra$Client.recv_remove(Cassandra.java:861) at org.apache.cassandra.thrift.Cassandra$Client.remove(Cassandra.java:845) at org.apache.cassandra.cli.CliClient.executeDelete(CliClient.java:455) at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:225) at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219) at org.apache.cassandra.cli.CliMain.main(CliMain.java:346) [default@system] [root@ ~]# /usr/local/apache-cassandra/bin/nodetool -h 127.0.0.1 scrub system HintsColumnFamily [root@ ~]# /usr/local/apache-cassandra/bin/cassandra-cli -h 127.0.0.1 -p 9160 Connected to: "nosql-world" on 127.0.0.1/9160 Welcome to Cassandra CLI version 1.1.6 Type 'help;' or '?' for help. Type 'quit;' or 'exit;' to quit. [default@unknown] use system; Authenticated to keyspace: system [default@system] list HintsColumnFamily; Using default limit of 100 Using default column limit of 100 0 Row Returned. Elapsed time: 73 msec(s). [default@system] tail /var/log/cassandra/system.log INFO [CompactionExecutor:193] 2012-11-06 12:03:12,233 OutputHandler.java (line 42) Scrubbing SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hf-11-Data.db') INFO [CompactionExecutor:193] 2012-11-06 12:03:12,236 OutputHandler.java (line 42) Scrub of SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hf-11-Data.db') complete; looks like all 1 rows were tombstoned -- Marcus Both On Mon, 5 Nov 2012 19:21:40 +0000 Stephen Pierce <spie...@verifyle.com> wrote: > I did have a tombstone. I couldn't seem to delete it, so I truncated the > column family. > > Thanks for the help. > > Steve > > [default@system] list HintsColumnFamily; > Using default limit of 100 > Using default column limit of 100 > ------------------- > RowKey: 55555555555555555555555555555554 > > I wasn't able to delete the row: > > [default@system] del HintsColumnFamily[55555555555555555555555555555554]; > row removed. > [default@system] list HintsColumnFamily; > Using default limit of 100 > Using default column limit of 100 > ------------------- > RowKey: 55555555555555555555555555555554 > > But I finally truncated the column family: > > [default@system] truncate HintsColumnFamily; > HintsColumnFamily truncated. > > > -----Original Message----- > From: Radim Kolar [mailto:h...@filez.com] > Sent: Monday, October 29, 2012 3:46 PM > To: user@cassandra.apache.org > Subject: Re: Hinted Handoff runs every ten minutes > > Dne 29.10.2012 23:24, Stephen Pierce napsal(a): > > I'm running 1.1.5; the bug says it's fixed in 1.0.9/1.1.0. > > > > How can I check to see why it keeps running HintedHandoff? > you have tombstone is system.HintsColumnFamily use list command in > cassandra-cli to check > -- Marcus Both