Re: Pelops - a new Java client library paradigm

2010-06-11 Thread Ian Soboroff
Sounds nice. Can you say something about the scales at which you've used this library? Both write and read load? Size of clusters and size of data? Ian On Fri, Jun 11, 2010 at 9:41 AM, Dominic Williams < thedwilli...@googlemail.com> wrote: > Pelops is a new high quality Java client library fo

Re: Handling disk-full scenarios

2010-06-08 Thread Ian Soboroff
t 12:50 PM, Ian Soboroff wrote: > Story continued, in hopes this experience is useful to someone... > > I shut down the node, removed the huge file, restarted the node, and told > everybody to repair. Two days later, AE stages are still running. > > Ian > > > On Thu, J

Re: Tree Search in Cassandra

2010-06-07 Thread Ian Soboroff
I was going to say, if ordered trees are your problem, Cassandra is not your solution. Try building something with Berkeley DB. Ian On Jun 7, 2010, at 17:30, Tatu Saloranta wrote: On Mon, Jun 7, 2010 at 12:06 AM, David Boxenhorn wrote: I wonder if there is a robust algorithm for maintain

Re: Handling disk-full scenarios

2010-06-04 Thread Ian Soboroff
BOD configuration is contraindicated for cassandra. > http://wiki.apache.org/cassandra/CassandraHardware > > On Tue, Jun 1, 2010 at 1:08 PM, Ian Soboroff wrote: > > My nodes have 5 disks and are using them separately as data disks. The > > usage on the disks is not uniform, and one i

Capacity planning and Re: Handling disk-full scenarios

2010-06-02 Thread Ian Soboroff
ata directories using du(1) I get around 23TB already used. On Wed, Jun 2, 2010 at 11:29 AM, Ian Soboroff wrote: > Ok, answered part of this myself. You can stop a node, move files around > on the data disks, as long as they stay in the right keyspace directories, > and all is fine.

Re: Handling disk-full scenarios

2010-06-02 Thread Ian Soboroff
oadbalance, or ... what? It would be great to be able to fix a storage quota on a per-data-directory basis, to ensure that enough capacity is retained for anticompaction. Default 45% quota, adjustable for the brave. Ian On Tue, Jun 1, 2010 at 4:08 PM, Ian Soboroff wrote: > My nodes have 5

Handling disk-full scenarios

2010-06-01 Thread Ian Soboroff
My nodes have 5 disks and are using them separately as data disks. The usage on the disks is not uniform, and one is nearly full. Is there some way to manually balance the files across the disks? Pretty much anything done via nodetool incurs an anticompaction with obviously fails. system/ is no

Re: Monitoring compaction

2010-06-01 Thread Ian Soboroff
code a patch if anyone's interested. Ian On Tue, Jun 1, 2010 at 12:41 PM, Ian Soboroff wrote: > Thanks. Are folks open to exposing this via nodetool? I've been trying to > figure out a decent way to aggregate and expose all this information that is > easier than nodetool and les

Re: Monitoring compaction

2010-06-01 Thread Ian Soboroff
se is private, so running a pile of jconsoles is not even possible...) Ian On Tue, Jun 1, 2010 at 12:33 PM, Dylan Egan / WildfireApp.com < dylan.e...@wildfireapp.com> wrote: > Hi Ian, > > On Tue, Jun 1, 2010 at 9:27 AM, Ian Soboroff wrote: > > Are stats exposed over JMX for compact

Monitoring compaction

2010-06-01 Thread Ian Soboroff
Are stats exposed over JMX for compaction? I'm trying to see when a node is in compaction, and guess when it will complete. tpstats doesn't show anything but the process is using lots of CPU time... I was wondering if there's a better view on compaction besides looking backwards in the system.log

Re: Continuously increasing RAM usage

2010-05-30 Thread Ian Soboroff
I'm also seeing the same thing. I need to do more log-staring to understand what's up... incompatible with holiday weekends ;-). tpstats are all 0. Ian On Sat, May 29, 2010 at 5:54 PM, Jonathan Ellis wrote: > Did you manually invoke a GC? It doesn't look like you're using > enough of the hea

Re: Continuously increasing RAM usage

2010-05-27 Thread Ian Soboroff
A lot of folks have reported this issue, and there are a few JIRAs related to it. Post the output of nodetool tpstats. Also, are there lots of GCs in the system.log? If so, are they something besides ParNew? Ian On Thu, May 27, 2010 at 2:32 AM, James Golick wrote: > We're seeing RAM usage co

Re: Ideal configuration for given hardware

2010-05-24 Thread Ian Soboroff
My data disks on two of my nodes are RAID-5, just because of circumstances. My other nodes are JBOD. I don't notice any real difference, but I haven't strongly benched it. Ian On Mon, May 24, 2010 at 2:45 PM, Jonathan Ellis wrote: > I can think of at least 2 clusters running 32GB boxes with sin

Re: Scaling problems

2010-05-22 Thread Ian Soboroff
s waiting to be sent. Is there some way to tell the system "Just stop caring, I'm just writing, let's worry about leveling out when I get around to wanting to read?" Ian On Fri, May 21, 2010 at 9:06 PM, Jonathan Ellis wrote: > On Fri, May 21, 2010 at 9:09 AM, I

Re: Scaling problems

2010-05-21 Thread Ian Soboroff
Ok, spending some time slogging through the cassandra-user archives. Seems lots of folks have this problem. Starting with a JVM upgrade, then skimming through JIRA looking for patches. Ian On Fri, May 21, 2010 at 12:09 PM, Ian Soboroff wrote: > So at the moment, I'm not running m

Re: Scaling problems

2010-05-21 Thread Ian Soboroff
0 0 MEMTABLE-POST-FLUSHER 0 0206 FLUSH-WRITER-POOL 0 0206 AE-SERVICE-STAGE 0 0 0 HINTED-HANDOFF-POOL 1 158 23 On Fri, May 21, 2010 at 10:37 AM,

Re: Scaling problems

2010-05-21 Thread Ian Soboroff
t 7:05 AM, Ian Soboroff wrote: > > Just an update. I rolled the memtable size back to 128MB. I am still > > seeing that the daemon runs for a while with reasonable heap usage, but > then > > the heap climbs up to the max (6GB in this case, should be plenty) and it > >

Re: Scaling problems

2010-05-21 Thread Ian Soboroff
, it's over and done with for this application. I'm just trying to get stuff loaded. Is there a limit to how much on-disk data a Cassandra daemon can manage? Is there runtime overhead associated with stuff on disk? Ian On Thu, May 20, 2010 at 9:31 PM, Ian Soboroff wrote: > Excellen

Re: Scaling problems

2010-05-20 Thread Ian Soboroff
-- large = more than 2 or 3) > > On Tue, May 18, 2010 at 6:24 AM, Ian Soboroff wrote: > > I hope this isn't too much of a newbie question. I am using Cassandra > 0.6.1 > > on a small cluster of Linux boxes - 14 nodes, each with 8GB RAM and 5 > data > > drives. T

Scaling problems

2010-05-18 Thread Ian Soboroff
I hope this isn't too much of a newbie question. I am using Cassandra 0.6.1 on a small cluster of Linux boxes - 14 nodes, each with 8GB RAM and 5 data drives. The nodes are running HDFS to serve files within the cluster, but at the moment the rest of Hadoop is shut down. I'm trying to load a lar

Re: Handling failures in Thrift

2010-05-13 Thread Ian Soboroff
has not yet realized it": > altered the phi constant in o.a.c.gsm.FailureDetector ( > phiConvictThreshold_, > default == 8)? > > // Roger Schildmeijer > > On 13 maj 2010, at 19.53em, Ian Soboroff wrote: > > I searched the Wiki and the mailing list archives a b

Handling failures in Thrift

2010-05-13 Thread Ian Soboroff
I searched the Wiki and the mailing list archives a bit but couldn't find the answer. If I catch an exception from a Cassandra.Client method, in my case batch_mutate, what's the proper course of action? Ignoring InvalidRequestException, we have Unavailable, TimedOut, and generic Thrift exception