> Currently I am copying all data files (thats all existing data) from one node > to the new nodes in hope that I could than manually assign them their new > tokenrange (nodetool move) and do cleanup.
Unless I'm misunderstanding you I believe you should be setting the initial token. nodetool move would be for a node already in the ring. And keep in mind that a nodetool move is currently a decommission+bootstrap - so if you're teetering on the edge of overload you will want to keep that in mind when moving a node to avoid ending up in a worse situation as another node temporarily receives more load than usual as a result of increased ring ownership. > Obviously I will try this tomorrow (it's been a long day) on a test system > but any advice would be highly appreciated. One possibility if you have additional hardware to spare temporarily, is to add more nodes than you actually need and then, once you are significantly over capacity, you have the flexibility to move nodes around to an optimum position and then decommission those machines that were only "borrowed". I.e., initial bootstrap of nodes takes a shorter amount of time because you're giving them less token space per new node. And once all are in the ring, you're free to move things around and then free up the hardware. (Another option may be to implement throttling of the anti-compaction so that it runs very slowly during peak hours, but that requires patching cassandra or else firewall/packet filtering fu and is probably likely to be more risky than it's worth.) -- / Peter Schuller
