My reading is it didn't forget the schema. It lost the data. My reading is decomissioning worked fine. Possibly when you changed the replication on a keyspace to include a second data center, the data didn't get replicated.
When you ADD a datacenter, you need to do a nodetool rebuild to get the data streamed to the new data center. When you alter a keyspace to include another datacenter in its replication schema, a nodetool repair is required -- was this done? http://www.datastax.com/documentation/cql/3.0/cql/cql_using/update_ks_rf_t.html When you use nodetool decomission, you're effectively deleting the parititioning token from the cluster. The node being decommissioned will stream its data to the new owners of its original token range. This streaming in no way should affect any other datacenter because you have not changed the tokens or data ownership for any datacenter but the one in which you are decomissioning a node. When you eventually decomission the last node in the datacenter, all data is gone as there are no tokens in that datacenter to own any data. If you had a keyspace that was only replicated within that datacenter, that data is gone (though you could probably add nodes back in and ressurect it). If you had a keyspace where you changed the replication to include another datacenter, if that datacenter had never received the data, then it may have the schema but would have none of the data (other than new data that was written AFTER you change the replication). On Thu, Aug 7, 2014 at 2:11 PM, srmore <comom...@gmail.com> wrote: > > > > On Thu, Aug 7, 2014 at 12:27 PM, Robert Coli <rc...@eventbrite.com> wrote: > >> On Thu, Aug 7, 2014 at 10:04 AM, srmore <comom...@gmail.com> wrote: >> >>> Sorry for being ambiguous. By "deletes" I mean that running >>> decommission I can no longer see any keyspaces owned by this node or >>> replicated by other nodes using the cfstats command. I am also seeing the >>> same behavior when I remove a single node from a cluster (without >>> datacenters). >>> >> >> I'm still not fully parsing you, but clusters should never "forget" >> schema as a result of decommission. >> >> Is that what you are saying is happening? >> > > Yes, this is what is happening. > > >> >> (In fact, even the decommissioned node itself does not forget its schema, >> which I personally consider a bug.) >> > > > Ok, so I am assuming this is not a normal behavior and possibly a bug - > is this correct ? > > >> >> =Rob >> >> > -- *Ken Hancock *| System Architect, Advanced Advertising SeaChange International 50 Nagog Park Acton, Massachusetts 01720 ken.hanc...@schange.com | www.schange.com | NASDAQ:SEAC <http://www.schange.com/en-US/Company/InvestorRelations.aspx> Office: +1 (978) 889-3329 | [image: Google Talk:] ken.hanc...@schange.com | [image: Skype:]hancockks | [image: Yahoo IM:]hancockks [image: LinkedIn] <http://www.linkedin.com/in/kenhancock> [image: SeaChange International] <http://www.schange.com/>This e-mail and any attachments may contain information which is SeaChange International confidential. The information enclosed is intended only for the addressees herein and may not be copied or forwarded without permission from SeaChange International.
