Re: Ant error in Eclipse when building Cassandra

2011-05-07 Thread Stephen Connolly
if you can give me (an intellij user) enough details to reproduce on my MBP I'll try and fix it. things like, download this eclipse distro, add these update centers, set these env variables, then clicks through this horrible UI as follows... presto crash! - Stephen --- Sent from my Android

Re: Running Cassandra across different Amazon EC2 regions

2011-05-07 Thread Stephen Connolly
vpn on ubuntu should be easy if you ask your good friend google... you should not have to pay for it (but paying might get you a fancy GUI, or perhaps very optimized performance that could squeeze a few more %) - Stephen --- Sent from my Android phone, so random spelling mistakes, random

Re: GC for ParNew (cassandra under performance ) 0.6.3

2011-05-07 Thread Ali Ahsan
Hi That sloved my issue of GC,Now i am facing with new one i have no swap but this process kswapd0 take up 100% cpu when there is load on cassandra any idea why this is happening On 05/05/2011 09:06 PM, Ali Ahsan wrote: Thanks replying,let me disable my swap memory. On 05/05/2011 09:01 PM,

Re: Migrating all rows from 0.6.13 to 0.7.5 over thrift?

2011-05-07 Thread aaron morton
get_range_slices() does read repair if enabled (checked DoConsistencyChecksBoolean in the config, it's on by default) so you should be getting good reads. If you want belt-and-braces run nodetool repair first. Hope that helps. On 7 May 2011, at 11:46, Jeremy Hanna wrote: Great! I just

compaction strategy

2011-05-07 Thread Terje Marthinussen
Even with the current concurrent compactions, given a high speed datafeed, compactions will obviously start lagging at some stage, and once it does, things can turn bad in terms of disk usage and read performance. I have not read the compaction code well, but if

Re: GC for ParNew (cassandra under performance ) 0.6.3

2011-05-07 Thread Jonathan Ellis
I'll quote the top google hit for [kswapd0 100%]: How much memory did you have in cached when you looked with top (and no swap enabled) ? If the amount of cached memory is very low, it could mean that your shared libraries are being pushed out of memory, instead of the kernel swapping out some

Re: Migrating all rows from 0.6.13 to 0.7.5 over thrift?

2011-05-07 Thread Jonathan Ellis
range_slices respects consistencylevel, but only single-row reads and multiget do the *repair* part of RR. On Sat, May 7, 2011 at 1:44 AM, aaron morton aa...@thelastpickle.com wrote: get_range_slices() does read repair if enabled (checked DoConsistencyChecksBoolean in the config, it's on by

Re: compaction strategy

2011-05-07 Thread Jonathan Ellis
On Sat, May 7, 2011 at 2:01 AM, Terje Marthinussen tmarthinus...@gmail.com wrote: 1. Would it make sense to make full compactions occur a bit more aggressive. I'd rather reduce the performance impact of being behind, than do more full compactions:

Performance thoughts on CQL

2011-05-07 Thread Edward Capriolo
Having used mysql as key value store before I have noticed that when doing low-latency queries the query planning time can actually end up being larger then the execution time. Even if you use a prepared statement mechanism with clients connecting and disconnecting this can be a lot of overhead.

Re: Ant error in Eclipse when building Cassandra

2011-05-07 Thread Ed Anuff
Yes, it is, those last two lines repeat forever: at org.apache.tools.ant.PropertyHelper.getPropertyHook(PropertyHelper.java:189) at org.apache.maven.artifact.ant.POMPropertyHelper.getPropertyHook(POMPropertyHelper.java:50) It's trying to look up a property value somewhere and getting

Re: compaction strategy

2011-05-07 Thread Edward Capriolo
On Sat, May 7, 2011 at 8:54 AM, Jonathan Ellis jbel...@gmail.com wrote: On Sat, May 7, 2011 at 2:01 AM, Terje Marthinussen tmarthinus...@gmail.com wrote: 1. Would it make sense to make full compactions occur a bit more aggressive. I'd rather reduce the performance impact of being behind, than

Re: compaction strategy

2011-05-07 Thread Peter Schuller
If you are seeing 600 pending compaction tasks regularly you almost definitely need more hardware. Note that pending compactions is pretty misleading and you can't really draw conclusions just based on the pending compactions number/graph. For example, standard behavior during e.g.a long

Re: compaction strategy

2011-05-07 Thread Terje Marthinussen
This is an all ssd system. I have no problems with read/write performance due to I/O. I do have a potential with the crazy explosion you can get in terms of disk use if compaction cannot keep up. As things falls behind and you get many generations of data, yes, read performance gets a problem due

Re: GC for ParNew (cassandra under performance ) 0.6.3

2011-05-07 Thread Ali Ahsan
This what free -m and vmstat show now my CPU is on 100% [root@cassandra2 ~]# free -m total used free sharedbuffers cached Mem: 16043 15966 76 0 4900 -/+ buffers/cache: 15061982 Swap:0

Re: compaction strategy

2011-05-07 Thread Peter Schuller
It does not really make sense to me to go through all these minor merges when a full compaction will do a much faster and better job. In a system heavily reliant on caching (platter drives, large data sizes, much larger than RAM) major compactions can be very detrimental to performance due to

Re: Performance thoughts on CQL

2011-05-07 Thread Jonathan Ellis
The only query planning Cassandra does is deciding which index to use when there are multiple options. That is handled the same way whether the request is from CQL or classic Thrift. On Sat, May 7, 2011 at 8:17 AM, Edward Capriolo edlinuxg...@gmail.com wrote: Having used mysql as key value store

Re: GC for ParNew (cassandra under performance ) 0.6.3

2011-05-07 Thread Ali Ahsan
Hi All some on suggested about linux scheduler cat /sys/block/sda/queue/scheduler there are quite few of them noop [anticipatory] deadline cfq.I have changed from cfq default to anticipatory,Now problem is less for me kswapd0 take 10-50% of cpu in very short bust less then milliseconds.Also

Re: Migrating all rows from 0.6.13 to 0.7.5 over thrift?

2011-05-07 Thread aaron morton
I've been able to reproduce the fault using python on my mac book see https://github.com/amorton/cassandra-unicode-bug When we try to find the unicode key in the index in 0.7 it fails because the tokens are different. The readme in the github project has more info. Any thoughts? Will try to

Re: CFHistograms?

2011-05-07 Thread Josep Blanquer
I believe the offset value of Writes and Reads are in *micro*seconds right? (that page talks about *milli*seconds) Also, are any timeouts or errors reflected in those times or just successful operations? if not, is there any JMX or other tool to keep track of them? Josep M. On Fri, May 6, 2011

Re: Migrating all rows from 0.6.13 to 0.7.5 over thrift?

2011-05-07 Thread Jake Luciani
I know thrift and python and Unicode don't mix. On May 7, 2011, at 4:21 PM, aaron morton aa...@thelastpickle.com wrote: I've been able to reproduce the fault using python on my mac book see https://github.com/amorton/cassandra-unicode-bug When we try to find the unicode key in the

Re: Memory Usage During Read

2011-05-07 Thread Serediuk, Adam
How much memory should a single hot cf with a 128mb memtable take with row and key caching disabled during read? Because I'm seeing heap go from 3.5gb skyrocketing straight to max (regardless of the size, 8gb and 24gb both do the same) at which time the jvm will do nothing but full gc and is

Re: Migrating all rows from 0.6.13 to 0.7.5 over thrift?

2011-05-07 Thread aaron morton
I remembered something like that so had a look at RangeSliceResponseResolver.resolve() in 0.6.12 and it looks like it schedules the repairs... protected Row getReduced() { ColumnFamily resolved = ReadResponseResolver.resolveSuperset(versions);

Re: Memory Usage During Read

2011-05-07 Thread Jonathan Ellis
The live:serialized size ratio depends on what your data looks like (small columns will be less efficient than large blobs) but using the rule of thumb of 10x, around 1G * (1 + memtable_flush_writers + memtable_flush_queue_size). So first thing I would do is drop writers and queue to 1 and 1.

Re: Migrating all rows from 0.6.13 to 0.7.5 over thrift?

2011-05-07 Thread Jonathan Ellis
Right, that's sort of a half-repair: it will repair differences in replies it got, but it won't doublecheck md5s on the rest in the background. So if you're doing CL.ONE reads this is a no-op. On Sat, May 7, 2011 at 4:25 PM, aaron morton aa...@thelastpickle.com wrote: I remembered something like