Re: Commitlog files not getting deleted

2013-08-22 Thread Jay Svc
its DSE 3.1 Cassandra 2.1 On Thu, Aug 22, 2013 at 10:28 AM, Robert Coli wrote: > On Thu, Aug 22, 2013 at 10:24 AM, Jay Svc wrote: > >> In our cluster, the commit log is getting filled up as write progresses. >> It is noticed that once the commit log is flushed to SSTa

Commitlog files not getting deleted

2013-08-22 Thread Jay Svc
Hi Users, In our cluster, the commit log is getting filled up as write progresses. It is noticed that once the commit log is flushed to SSTable the commit log files are not removed/deleted. The result of that the commit log volume is getting filled with commit log files. Any reason why commit log

Re: How to make compaction run faster?

2013-04-23 Thread Jay Svc
that works and then make one change at a time until it does not. > You should then be able to say "The system can handle X writes of Y size > per second, but after that compaction cannot keep up." > > Cheers > > - > Aaron Morton > Freelance Cassandra

Re: How to make compaction run faster?

2013-04-18 Thread Jay Svc
ction > in a test machine, solved by changing from openjdk 1.6 to sun-jdk 1.6. > > > > > On Thu, Apr 18, 2013 at 2:08 PM, Jay Svc wrote: > >> By the way the compaction and commit log disk latency, these are two >> seperate problems I see. >> >> The impo

Re: How to make compaction run faster?

2013-04-18 Thread Jay Svc
the throughput > until compaction cannot keep up. Then increase the throughout and see how > that works. Then increase throughput again and see what happens. > > Cheers > > > - > Aaron Morton > Freelance Cassandra Consultant > New Zealand > > @aaro

Re: How to make compaction run faster?

2013-04-18 Thread Jay Svc
oblems with compaction > in a test machine, solved by changing from openjdk 1.6 to sun-jdk 1.6. > > > > > On Thu, Apr 18, 2013 at 2:08 PM, Jay Svc wrote: > >> By the way the compaction and commit log disk latency, these are two >> seperate problems I see. >>

Re: How to make compaction run faster?

2013-04-18 Thread Jay Svc
By the way the compaction and commit log disk latency, these are two seperate problems I see. The important one is compaction problem, How I can speed that up? Thanks, Jay On Thu, Apr 18, 2013 at 12:07 PM, Jay Svc wrote: > Looks like formatting is bit messed up. Please let me know if

Re: How to make compaction run faster?

2013-04-18 Thread Jay Svc
Looks like formatting is bit messed up. Please let me know if you want the same in clean format. Thanks, Jay On Thu, Apr 18, 2013 at 12:05 PM, Jay Svc wrote: > Hi Aaron, Alexis, > > Thanks for reply, Please find some more details below. > > *Core problems:* Compaction is taking

Re: How to make compaction run faster?

2013-04-18 Thread Jay Svc
rectories and not in the > commitlog. > > > > http://wiki.apache.org/cassandra/MemtableSSTable > > > > > > > > > > On Wed, Apr 17, 2013 at 7:58 PM, Jay Svc wrote: > > Hi Alexis, > > > > Thank you for your response. > > > &g

Re: How to make compaction run faster?

2013-04-18 Thread Jay Svc
; >>> >>> On Wed, Apr 17, 2013 at 5:26 PM, Edward Capriolo >>> wrote: >>> >>>> three things: >>>> 1) compaction throughput is fairly low (yaml nodetool) >>>> 2) concurrent compactions is fairly low (yaml) >>>> 3) mu

Re: How to make compaction run faster?

2013-04-17 Thread Jay Svc
your version >> >> Try raising these things. Otherwise consider option 4. >> >> 4)$$$ RAID,RAM> >> >> On Wed, Apr 17, 2013 at 4:01 PM, Jay Svc wrote: >> >>> Hi Team, >>> >>> >>> >>> I have a

Re: How to make compaction run faster?

2013-04-17 Thread Jay Svc
RAID,RAM > > On Wed, Apr 17, 2013 at 4:01 PM, Jay Svc wrote: > >> Hi Team, >> >> >> >> I have a high write traffic to my Cassandra cluster. I experience a very >> high number of pending compactions. As I expect higher writes, The pending >&g

How to make compaction run faster?

2013-04-17 Thread Jay Svc
Hi Team, I have a high write traffic to my Cassandra cluster. I experience a very high number of pending compactions. As I expect higher writes, The pending compactions keep increasing. Even when I stop my writes it takes several hours to finishing pending compactions. My CF is configured with

Re: Does Memtable resides in Heap?

2013-04-16 Thread Jay Svc
...@adform.com> wrote: > >> Memtables resides in heap, write rate impacts GC, more writes - more >> frequent and longer ParNew GC pauses. >> >> >> From: Jay Svc [mailto:jaytechg...@gmail.com] >> Sent: Friday, April 12, 2013 01:03 >> To: user@cassandr

Re: Does Memtable resides in Heap?

2013-04-15 Thread Jay Svc
On Thu, Apr 11, 2013 at 10:35 PM, Viktor Jevdokimov < viktor.jevdoki...@adform.com> wrote: > Memtables resides in heap, write rate impacts GC, more writes - more > frequent and longer ParNew GC pauses. > > > From: Jay Svc [mailto:jaytechg...@gmail.com] > Sent: Friday, A

Does Memtable resides in Heap?

2013-04-11 Thread Jay Svc
Hi Team, I have got this 8GB of RAM out of that 4GB allocated to Java Heap. My question is the size of Memtable does it contribute to heap size? or they are part of off-heap? Does bigger Memtable would have impact on GC and overall memory management? I am using DSE 3.0 / Cassandra 1.1.9. Thanks

Re: Insert v/s Update performance

2013-04-02 Thread Jay Svc
u > system is running. > > Slowing down compaction will reduce it's impact. > > Cheers > > - > Aaron Morton > Freelance Cassandra Consultant > New Zealand > > @aaronmorton > http://www.thelastpickle.com > > On 30/03/2013, at 12:58 AM,

Re: Insert v/s Update performance

2013-03-29 Thread Jay Svc
heers > > - > Aaron Morton > Freelance Cassandra Consultant > New Zealand > > @aaronmorton > http://www.thelastpickle.com > > On 27/03/2013, at 1:43 PM, Jay Svc wrote: > > Thanks Dean again! > > My use case is high number of reads and writ

Re: Insert v/s Update performance

2013-03-26 Thread Jay Svc
etc. and see how backlogged you are in > pending tasks….how many pending? > > Later, > Dean > > From: Jay Svc mailto:jaytechg...@gmail.com>> > Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" < > user@cassandra.apache.or

Re: Insert v/s Update performance

2013-03-26 Thread Jay Svc
r handle the load. > > Later, > Dean > > From: Jay Svc mailto:jaytechg...@gmail.com>> > Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" < > user@cassandra.apache.org<mailto:user@cassandra.apache.org>> > Date: Tuesda

Insert v/s Update performance

2013-03-26 Thread Jay Svc
Hi Team, I have this 3 node cluster. I am writing data to these node at the rate of 2,000 records/second. What I observed that if I do inserts. (Means records for those keys does not exist, my column family has 0 records to start with) then I have better write performacne, low SSTable count, low p

conditional update or insert

2013-02-01 Thread Jay Svc
Hi All, On each row I have a column which maintains the timestamp like "lastUpdated" etc. While inserting such row I want to make sure that the row should be only updated if the lastUpdated is older than the new one I am inserting. One way to do this is - Read the record first check the timesta

High Read and write through put

2013-01-21 Thread Jay Svc
Folks, For given situation I am expecting multiple read and write request to a same cluster. What are primary design or configuration consideration we should make? Any thoughts or links to such documentation is appreciated. Thanks, Jay

Concurrent write performance

2013-01-21 Thread Jay Svc
Folks, I would like to write(insert or update) to a single row in a column family. I have concurrent requests which will write to a single row. Do we see any performance implications because of concurrent writes to a single row where comparator has to sort the columns at the same time? Please s