Re: Can't get data after building cluster
To elaborate: If you manage to screw things up to where it thinks a node has data, but it does not (adding a node without bootstrap would do this, for instance, which is probably what you did), at most data in the token range assigned to that node will be affected. On Tue, Jun 1, 2010 at 12:45 AM, David Boxenhorn da...@lookin2.com wrote: You say no, but that is exactly what I just observed. Can I have some more explanation? To recap: I added a server to my cluster. It had some junk in the system/LocationInfo files from previous, unsuccessful attempts to add the server to the cluster. (They were unsuccessful because I hadn't opened the port on that computer.) When I finally succeeded in adding the 2nd server, the 1st server started returning null when I tried to get data using the CLI. I stopped the 2nd server, deleted the files in system, restarted, and everything worked. I'm afraid that this, or some similar scenario will do the same, after I go live. How can I protect myself? On Mon, May 31, 2010 at 10:10 PM, Jonathan Ellis jbel...@gmail.com wrote: No. On Mon, May 31, 2010 at 10:47 AM, David Boxenhorn da...@lookin2.com wrote: So this means that I can take my entire cluster off line if I make a mistake adding a new server??? Yikes! On Mon, May 31, 2010 at 6:41 PM, David Boxenhorn da...@lookin2.com wrote: OK. Got it working. I had some data in the 2nd server from previous failed attempts at hooking up to the cluster. When I deleted that data and tried again, it said bootstrapping and my 1st server started working again. On Mon, May 31, 2010 at 4:50 PM, David Boxenhorn da...@lookin2.com wrote: I am trying to get a cluster up and working for the first time. I got one server up and running, with lots of data on it, which I can see with the CLI. I added my 2nd server, they seem to recognize each other. Now I can't see my data with the CLI. I do a get and it returns null. The data files seem to be intact. What happened??? How can I fix it? -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com
Re: Can't get data after building cluster
Depending on the key, the request would have been proxied to the first or second node. The CLI uses a consistency level of ONE, meaning that only a single node's data would have been considered when you get(). Also, the responsible nodes for a given key are mapped accordingly at request time, and proxy requests are made internally on your behalf. This allows the R+WN to hold, where N is the replication factor. It closes the subset of active nodes responsible for a key in a deterministic way. See http://www.slideshare.net/benjaminblack/introduction-to-cassandra-replication-and-consistency for more information. On Tue, Jun 1, 2010 at 1:43 AM, David Boxenhorn da...@lookin2.com wrote: I don't think it can be the case that at most data in the token range assigned to that node will be affected - the new node had no knowledge of any of our data. Any fake data that it might have had through some error on my part could not have been within the range of real data. I had 4.25 G of data on the 1st server, and as far as I could tell I couldn't access any of it. On Tue, Jun 1, 2010 at 9:10 AM, Jonathan Ellis jbel...@gmail.com wrote: To elaborate: If you manage to screw things up to where it thinks a node has data, but it does not (adding a node without bootstrap would do this, for instance, which is probably what you did), at most data in the token range assigned to that node will be affected. On Tue, Jun 1, 2010 at 12:45 AM, David Boxenhorn da...@lookin2.com wrote: You say no, but that is exactly what I just observed. Can I have some more explanation? To recap: I added a server to my cluster. It had some junk in the system/LocationInfo files from previous, unsuccessful attempts to add the server to the cluster. (They were unsuccessful because I hadn't opened the port on that computer.) When I finally succeeded in adding the 2nd server, the 1st server started returning null when I tried to get data using the CLI. I stopped the 2nd server, deleted the files in system, restarted, and everything worked. I'm afraid that this, or some similar scenario will do the same, after I go live. How can I protect myself? On Mon, May 31, 2010 at 10:10 PM, Jonathan Ellis jbel...@gmail.com wrote: No. On Mon, May 31, 2010 at 10:47 AM, David Boxenhorn da...@lookin2.com wrote: So this means that I can take my entire cluster off line if I make a mistake adding a new server??? Yikes! On Mon, May 31, 2010 at 6:41 PM, David Boxenhorn da...@lookin2.com wrote: OK. Got it working. I had some data in the 2nd server from previous failed attempts at hooking up to the cluster. When I deleted that data and tried again, it said bootstrapping and my 1st server started working again. On Mon, May 31, 2010 at 4:50 PM, David Boxenhorn da...@lookin2.com wrote: I am trying to get a cluster up and working for the first time. I got one server up and running, with lots of data on it, which I can see with the CLI. I added my 2nd server, they seem to recognize each other. Now I can't see my data with the CLI. I do a get and it returns null. The data files seem to be intact. What happened??? How can I fix it? -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com
Can't get data after building cluster
I am trying to get a cluster up and working for the first time. I got one server up and running, with lots of data on it, which I can see with the CLI. I added my 2nd server, they seem to recognize each other. Now I can't see my data with the CLI. I do a get and it returns null. The data files seem to be intact. What happened??? How can I fix it?
Re: Can't get data after building cluster
OK. Got it working. I had some data in the 2nd server from previous failed attempts at hooking up to the cluster. When I deleted that data and tried again, it said bootstrapping and my 1st server started working again. On Mon, May 31, 2010 at 4:50 PM, David Boxenhorn da...@lookin2.com wrote: I am trying to get a cluster up and working for the first time. I got one server up and running, with lots of data on it, which I can see with the CLI. I added my 2nd server, they seem to recognize each other. Now I can't see my data with the CLI. I do a get and it returns null. The data files seem to be intact. What happened??? How can I fix it?
Re: Can't get data after building cluster
So this means that I can take my entire cluster off line if I make a mistake adding a new server??? Yikes! On Mon, May 31, 2010 at 6:41 PM, David Boxenhorn da...@lookin2.com wrote: OK. Got it working. I had some data in the 2nd server from previous failed attempts at hooking up to the cluster. When I deleted that data and tried again, it said bootstrapping and my 1st server started working again. On Mon, May 31, 2010 at 4:50 PM, David Boxenhorn da...@lookin2.comwrote: I am trying to get a cluster up and working for the first time. I got one server up and running, with lots of data on it, which I can see with the CLI. I added my 2nd server, they seem to recognize each other. Now I can't see my data with the CLI. I do a get and it returns null. The data files seem to be intact. What happened??? How can I fix it?
Re: Can't get data after building cluster
No. On Mon, May 31, 2010 at 10:47 AM, David Boxenhorn da...@lookin2.com wrote: So this means that I can take my entire cluster off line if I make a mistake adding a new server??? Yikes! On Mon, May 31, 2010 at 6:41 PM, David Boxenhorn da...@lookin2.com wrote: OK. Got it working. I had some data in the 2nd server from previous failed attempts at hooking up to the cluster. When I deleted that data and tried again, it said bootstrapping and my 1st server started working again. On Mon, May 31, 2010 at 4:50 PM, David Boxenhorn da...@lookin2.com wrote: I am trying to get a cluster up and working for the first time. I got one server up and running, with lots of data on it, which I can see with the CLI. I added my 2nd server, they seem to recognize each other. Now I can't see my data with the CLI. I do a get and it returns null. The data files seem to be intact. What happened??? How can I fix it? -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com
Re: Can't get data after building cluster
You say no, but that is exactly what I just observed. Can I have some more explanation? To recap: I added a server to my cluster. It had some junk in the system/LocationInfo files from previous, unsuccessful attempts to add the server to the cluster. (They were unsuccessful because I hadn't opened the port on that computer.) When I finally succeeded in adding the 2nd server, the 1st server started returning null when I tried to get data using the CLI. I stopped the 2nd server, deleted the files in system, restarted, and everything worked. I'm afraid that this, or some similar scenario will do the same, after I go live. How can I protect myself? On Mon, May 31, 2010 at 10:10 PM, Jonathan Ellis jbel...@gmail.com wrote: No. On Mon, May 31, 2010 at 10:47 AM, David Boxenhorn da...@lookin2.com wrote: So this means that I can take my entire cluster off line if I make a mistake adding a new server??? Yikes! On Mon, May 31, 2010 at 6:41 PM, David Boxenhorn da...@lookin2.com wrote: OK. Got it working. I had some data in the 2nd server from previous failed attempts at hooking up to the cluster. When I deleted that data and tried again, it said bootstrapping and my 1st server started working again. On Mon, May 31, 2010 at 4:50 PM, David Boxenhorn da...@lookin2.com wrote: I am trying to get a cluster up and working for the first time. I got one server up and running, with lots of data on it, which I can see with the CLI. I added my 2nd server, they seem to recognize each other. Now I can't see my data with the CLI. I do a get and it returns null. The data files seem to be intact. What happened??? How can I fix it? -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com
