Re: GCInspector info messages in cassandra log

2012-08-16 Thread Tamar Fraenkel
Thank you very much! *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 On Thu, Aug 16, 2012 at 12:11 AM, aaron morton wrote: > Is there anything to do before that? like drain or

Re: GCInspector info messages in cassandra log

2012-08-15 Thread aaron morton
> Is there anything to do before that? like drain or flush? For a clean shutdown I do nodetool -h localhost disablethrift nodetool -h localhost disablegossip && sleep 10 nodetool -h localhost drain then kill > Would you recommend that? If I do it, how often should I do a full snapshot, > and h

Re: GCInspector info messages in cassandra log

2012-08-14 Thread Tamar Fraenkel
Aaron, Thank you very much. I will do as you suggested. One last question regarding restart: I assume, I should do it node by node. Is there anything to do before that? like drain or flush? I am also considering enabling incremental backups on my cluster. Currently I take a daily full snapshot of

Re: GCInspector info messages in cassandra log

2012-08-14 Thread aaron morton
> According to cfstats there are the some CF with high Comacted row maximum > sizes (1131752, 4866323 and 25109160). Others max sizes are < 100. Are > these considered to be problematic, what can I do to solve that? They are only 1, 4 and 25 MB. Not too big. > What should be the values of

Re: GCInspector info messages in cassandra log

2012-08-14 Thread Tamar Fraenkel
Hi! It helps, but before I do more actions I want to give you some more info, and ask some questions: *Related Info* 1. According to my yaml file (where do I see these parameters in the jmx? I couldn't find them): in_memory_compaction_limit_in_mb: 64 concurrent_compactors: 1, but it i

Re: GCInspector info messages in cassandra log

2012-08-14 Thread aaron morton
There are a couple of steps you can take if compaction is causing GC. - if you have a lot of wide rows consider reducing the in_memory_compaction_limit_in_mb yaml setting. This will slow down compaction but will reduce the memory usage. - reduce concurrent_compactors Both of these may slow

GCInspector info messages in cassandra log

2012-08-13 Thread Tamar Fraenkel
Hi! I have 3 nodes ring running on Amazon EC2. About once a week I see in the logs compaction messages and around the same time info messages about GC (see below) that I think means it is taking too long and happening too often. Does it mean I have to reduce my cache size? Thanks, Tamar INFO [Sc