No, it's not expected, but it's pretty obvious from reading the code what'll happen. Opened https://issues.apache.org/jira/browse/CASSANDRA-13526
On Thu, May 11, 2017 at 12:53 PM, Jai Bheemsen Rao Dhanwada < jaibheem...@gmail.com> wrote: > Yes I have many keyspaces which are not spread across all the data > centers(expected by design). > In this case, is this the expected behavior cleanup will not work for all > the keyspaces(nodetool cleanup)? is it going to be fixed in the latest > versions? > > P.S: Thanks for the tip, I can workaround this by "nodetool cleanup > keyspacename" > > On Thu, May 11, 2017 at 12:11 PM, Jeff Jirsa <jji...@gmail.com> wrote: > >> If you didn't explicitly remove a keyspace from one of your datacenters, >> the next most likely cause is that you have one keyspace that's NOT >> replicated to one of the datacenters. You can work around this by running >> 'nodetool cleanup <ks>' on all of your other keyspaces individually, >> skipping the one that isn't replicated to that datacenter. >> >> >> >> On Thu, May 11, 2017 at 11:19 AM, Jai Bheemsen Rao Dhanwada < >> jaibheem...@gmail.com> wrote: >> >>> Thanks Jeff, >>> >>> I have a C* cluster spread across multiple datacenter. >>> reason for cleanup : I added multiple nodes to cluster and need to run >>> cleanup on old nodes so that the redundant data is cleaned-up. >>> >>> On Thu, May 11, 2017 at 11:08 AM, Jeff Jirsa <jji...@apache.org> wrote: >>> >>>> >>>> >>>> On 2017-05-10 22:44 (-0700), Jai Bheemsen Rao Dhanwada < >>>> jaibheem...@gmail.com> wrote: >>>> > Hello, >>>> > >>>> > I am running into an issue where *nodetool cleanup *fails to cleanup >>>> data. >>>> > We are running 2.1.16 version of Cassandra. >>>> > >>>> > >>>> > [user@host ~]$ nodetool cleanup >>>> > Aborted cleaning up atleast one column family in keyspace user, check >>>> > server logs for more information. >>>> > Aborted cleaning up atleast one column family in keyspace org, check >>>> server >>>> > logs for more information. >>>> > error: nodetool failed, check server logs >>>> > -- StackTrace -- >>>> > java.lang.RuntimeException: nodetool failed, check server logs >>>> > at >>>> > org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool >>>> .java:294) >>>> > at org.apache.cassandra.tools.Nod >>>> eTool.main(NodeTool.java:206) >>>> > >>>> > *Logs:* >>>> > >>>> > INFO [RMI TCP Connection(17)-x.x.x.x] 2017-05-05 04:04:07,987 >>>> > CompactionManager.java:415 - Cleanup cannot run before a node has >>>> joined >>>> > the ring >>>> > INFO [RMI TCP Connection(17)-x.x.x.x] 2017-05-05 04:04:08,010 >>>> > CompactionManager.java:415 - Cleanup cannot run before a node has >>>> joined >>>> > the ring >>>> > >>>> > All the nodes in the cluster are up and running. We tried doing a >>>> rolling >>>> > restart of all nodes and no luck. >>>> > >>>> > After looking at the Cassandra JIRA : >>>> > https://issues.apache.org/jira/browse/CASSANDRA-10991 looks like the >>>> issue >>>> > is fixed with 2.2.6 and 3.0 version. >>>> > While we have plans to upgrade to the latest versions(which might take >>>> > longer time), does any know if there is any work around to mitigate >>>> the >>>> > issue? >>>> > >>>> >>>> Are you running multiple datacenters, and you just removed a specific >>>> datacenter from a keyspace (and that's why you want to run cleanup)? If >>>> that's the case, I fear the fix for 10991 isn't really going to fix it in >>>> the way you hope (we may need a follow-up jira). What you'll almost >>>> certainly need to do is remove the data on disk manually, which is quite >>>> unfortunate as it'll require you to >>>> stop+delete-data-for-that-keyspace+start >>>> each node in the datacenter for which you removed replication. >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org >>>> For additional commands, e-mail: user-h...@cassandra.apache.org >>>> >>>> >>> >> >
