I didn't know you use actual key instead its md5 (for random patitioner) in
KCF. It's good point that I'll watch hit ratio of KCF to determine whether
it needs to be increased.
Thanks,
-Weijun
On Tue, Feb 16, 2010 at 5:34 PM, Jonathan Ellis wrote:
> On Tue, Feb 16, 2010 at 7:27 PM, Weijun Li
On Tue, Feb 16, 2010 at 7:27 PM, Weijun Li wrote:
> Yes my KeysCachedFraction is already 0.3 but it doesn't relief the disk i/o.
> I compacted the data to be a 60GB (took quite a while to finish and it
> increased latency as expected) one but doesn't help much either.
>
> If I set KCF to 1 (meanin
rameter in your config file. In 0.6 you can
> >> additionally cache rows. You don't want to use up all of the memory on
> your
> >> box for those caches though: you'll want to leave at least 50% for your
> OS's
> >> disk cache, which will st
on't want to use up all of the memory on your
>> box for those caches though: you'll want to leave at least 50% for your OS's
>> disk cache, which will store the full row content.
>>
>>
>> -Original Message-----
>> From: "Weij
l Message-
> From: "Weijun Li"
> Sent: Tuesday, February 16, 2010 12:16pm
> To: cassandra-user@incubator.apache.org
> Subject: Re: Cassandra benchmark shows OK throughput but high read latency
> (> 100ms)?
>
> Thanks for for DataFileDirectory trick and I'll
f the memory on your box for those
caches though: you'll want to leave at least 50% for your OS's disk cache,
which will store the full row content.
-Original Message-
From: "Weijun Li"
Sent: Tuesday, February 16, 2010 12:16pm
To: cassandra-user@incubator.apache.org
Su
On Tue, Feb 16, 2010 at 12:16 PM, Weijun Li wrote:
> Thanks for for DataFileDirectory trick and I'll give a try.
>
> Just noticed the impact of number of data files: node A has 13 data files
> with read latency of 20ms and node B has 27 files with read latency of 60ms.
> After I ran "nodeprobe co
Thanks for for DataFileDirectory trick and I'll give a try.
Just noticed the impact of number of data files: node A has 13 data files
with read latency of 20ms and node B has 27 files with read latency of 60ms.
After I ran "nodeprobe compact" on node B its read latency went up to 150ms.
The read l
On Tue, Feb 16, 2010 at 11:56 AM, Weijun Li wrote:
> One more thoughts about Martin's suggestion: is it possible to put the data
> files into multiple directories that are located in different physical
> disks? This should help to improve the i/o bottleneck issue.
>
>
Yes, you can already do this
On Tue, Feb 16, 2010 at 11:50 AM, Weijun Li wrote:
> Dumped 50mil records into my 2-node cluster overnight, made sure that
> there's not many data files (around 30 only) per Martin's suggestion. The
> size of the data directory is 63GB. Now when I read records from the cluster
> the read latency
One more thoughts about Martin's suggestion: is it possible to put the data
files into multiple directories that are located in different physical
disks? This should help to improve the i/o bottleneck issue.
Has anybody tested the row-caching feature in trunk (shoot for 0.6?)?
-Weijun
On Tue, Fe
Dumped 50mil records into my 2-node cluster overnight, made sure that
there's not many data files (around 30 only) per Martin's suggestion. The
size of the data directory is 63GB. Now when I read records from the cluster
the read latency is still ~44ms, --there's no write happening during the
read.
On Tue, Feb 16, 2010 at 2:32 AM, Dr. Martin Grabmüller <
martin.grabmuel...@eleven.de> wrote:
> In my tests I have observed that good read latency depends on keeping
> the number of data files low. In my current test setup, I have stored
> 1.9 TB of data on a single node, which is in 21 data file
> The other problem is: if I keep mixed write and read (e.g, 8
> write threads
> plus 7 read threads) against the 2-nodes cluster
> continuously, the read
> latency will go up gradually (along with the size of
> Cassandra data file),
> and at the end it will become ~40ms (up from ~20ms) even wit
Thanks,
-Weijun
-Original Message-
From: Jonathan Ellis [mailto:jbel...@gmail.com]
Sent: Sunday, February 14, 2010 6:22 PM
To: cassandra-user@incubator.apache.org
Subject: Re: Cassandra benchmark shows OK throughput but high read latency
(> 100ms)?
are you i/o bound? what is your on-disk da
are you i/o bound? what is your on-disk data set size? what does
iostats tell you?
http://spyced.blogspot.com/2010/01/linux-performance-basics.html
do you have a lot of pending compactions? (tpstats will tell you)
have you increased KeysCachedFraction?
On Sun, Feb 14, 2010 at 8:18 PM, Weijun
Hello,
I saw some Cassandra benchmark reports mentioning read latency that is less
than 50ms or even 30ms. But my benchmark with 0.5 doesn’t seem to support that.
Here’s my settings:
Nodes: 2 machines. 2x2.5GHZ Xeon Quad Core (thus 8 cores), 8GB RAM
ReplicationFactor=2 Partitioner=Random
17 matches
Mail list logo
