Re: UnavailableException on QUORUM write
On Jul 27, 2010, at 12:23 AM, Jonathan Ellis wrote: > Can you turn on debug logging and try this patch? Yes, but..I am on vacation now, so it will be about 3 weeks from now.
Re: UnavailableException on QUORUM write
Can you turn on debug logging and try this patch?
On Wed, Jul 21, 2010 at 1:23 AM, Per Olesen wrote:
>>> And when one of my non-seed nodes in my 3 node cluster is down, I do NOT
>>> get the exception.
>>> Anyway, guess I need to try and reproduce it in small scale.
>>
>>Does it return w/ UE immediately, or does it wait for RPCTimeout first?
>
> It returns with UE immediately.
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com
Index: src/java/org/apache/cassandra/service/StorageProxy.java
===
--- src/java/org/apache/cassandra/service/StorageProxy.java (revision
979441)
+++ src/java/org/apache/cassandra/service/StorageProxy.java (working copy)
@@ -204,6 +204,7 @@
Collection writeEndpoints =
rs.getWriteEndpoints(StorageService.getPartitioner().getToken(rm.key()), table,
naturalEndpoints);
Multimap hintedEndpoints =
rs.getHintedEndpoints(table, writeEndpoints);
int blockFor = determineBlockFor(writeEndpoints.size(),
consistency_level);
+logger.debug("blocking for " + blockFor + " endpoints out of "
+ writeEndpoints.size());
// avoid starting a write we know can't achieve the required
consistency
assureSufficientLiveNodes(blockFor, writeEndpoints,
hintedEndpoints, consistency_level);
@@ -296,6 +297,7 @@
}
if (liveNodes < blockFor)
{
+logger.debug("only " + liveNodes + " seen out of " + blockFor + "
required; throwing UE");
throw new UnavailableException();
}
}
SV: UnavailableException on QUORUM write
>> And when one of my non-seed nodes in my 3 node cluster is down, I do NOT get >> the exception. >> Anyway, guess I need to try and reproduce it in small scale. > >Does it return w/ UE immediately, or does it wait for RPCTimeout first? It returns with UE immediately.
Re: UnavailableException on QUORUM write
On Tue, Jul 20, 2010 at 6:40 AM, Per Olesen wrote: >>Seed should only be important when joining the cluster. You're using >>the Thrift API, right? > > Yep! > > And when one of my non-seed nodes in my 3 node cluster is down, I do NOT get > the exception. > Anyway, guess I need to try and reproduce it in small scale. Does it return w/ UE immediately, or does it wait for RPCTimeout first? -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com
SV: UnavailableException on QUORUM write
>Seed should only be important when joining the cluster. You're using >the Thrift API, right? Yep! And when one of my non-seed nodes in my 3 node cluster is down, I do NOT get the exception. Anyway, guess I need to try and reproduce it in small scale. On Tue, Jul 20, 2010 at 5:34 AM, Per Olesen wrote: > Hi, > > Think I might have found out the problem. > I had only one seed node, and when that node is down, they all give > UnavailableException. Guess at least one seed needs to be up then? Sounds > fair. > > > /Per > > Fra: Per Olesen [...@trifork.com] > Sendt: 9. juli 2010 11:19 > Til: user@cassandra.apache.org > Emne: Re: UnavailableException on QUORUM write > > On Jul 9, 2010, at 11:11 AM, ChingShen wrote: > >> Which client library do you use? > > Direct on thrift api using thrift.jar, in version 917130. > > -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com
Re: UnavailableException on QUORUM write
Seed should only be important when joining the cluster. You're using the Thrift API, right? On Tue, Jul 20, 2010 at 5:34 AM, Per Olesen wrote: > Hi, > > Think I might have found out the problem. > I had only one seed node, and when that node is down, they all give > UnavailableException. Guess at least one seed needs to be up then? Sounds > fair. > > > /Per > > Fra: Per Olesen [...@trifork.com] > Sendt: 9. juli 2010 11:19 > Til: user@cassandra.apache.org > Emne: Re: UnavailableException on QUORUM write > > On Jul 9, 2010, at 11:11 AM, ChingShen wrote: > >> Which client library do you use? > > Direct on thrift api using thrift.jar, in version 917130. > > -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com
SV: UnavailableException on QUORUM write
Hi, Think I might have found out the problem. I had only one seed node, and when that node is down, they all give UnavailableException. Guess at least one seed needs to be up then? Sounds fair. /Per Fra: Per Olesen [...@trifork.com] Sendt: 9. juli 2010 11:19 Til: user@cassandra.apache.org Emne: Re: UnavailableException on QUORUM write On Jul 9, 2010, at 11:11 AM, ChingShen wrote: > Which client library do you use? Direct on thrift api using thrift.jar, in version 917130.
Re: UnavailableException on QUORUM write
this sounds like a bug, although if you've attempted any node movement or bootstrapping, that could cause the required quorum to be larger than just the number of nodes. On Fri, Jul 9, 2010 at 3:53 AM, Per Olesen wrote: > Hi, > > I am a bit confused about getting an UnavailableException when doing a QUORUM > write. > > I have a 3 node cluster, with RF=3. When all 3 nodes are up, the QUORUM write > succeeds. When 1 of the 3 nodes are down, the QUORUM write fails with > UnavailableException. Shouldn't it be enough with 2 nodes up for RF=3? Won't > N/2+1 be 2 in my case? > > /Per -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com
Re: UnavailableException on QUORUM write
On Jul 9, 2010, at 11:11 AM, ChingShen wrote: > Which client library do you use? Direct on thrift api using thrift.jar, in version 917130.
Re: UnavailableException on QUORUM write
Which client library do you use? Shen On Fri, Jul 9, 2010 at 4:53 PM, Per Olesen wrote: > Hi, > > I am a bit confused about getting an UnavailableException when doing a > QUORUM write. > > I have a 3 node cluster, with RF=3. When all 3 nodes are up, the QUORUM > write succeeds. When 1 of the 3 nodes are down, the QUORUM write fails with > UnavailableException. Shouldn't it be enough with 2 nodes up for RF=3? Won't > N/2+1 be 2 in my case? > > /Per
UnavailableException on QUORUM write
Hi, I am a bit confused about getting an UnavailableException when doing a QUORUM write. I have a 3 node cluster, with RF=3. When all 3 nodes are up, the QUORUM write succeeds. When 1 of the 3 nodes are down, the QUORUM write fails with UnavailableException. Shouldn't it be enough with 2 nodes up for RF=3? Won't N/2+1 be 2 in my case? /Per
