Re: C* consumes all RAM

2013-04-16 Thread aaron morton
You are probably seeing this http://wiki.apache.org/cassandra/FAQ#mmap

Cheers

-
Aaron Morton
Freelance Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 16/04/2013, at 8:43 PM, Mikhail Mazursky ash...@gmail.com wrote:

 More details:
 
 USER   PID %CPU %MEMVSZ   RSS TTY  STAT START   TIME COMMAND 
 219   3801  0.7 92.7 6561116 3567016 ? SLl  Mar11 372:44 
 /usr/java/latest/bin/java
 
 Linux XXX@YYY 3.2.30-49.59.amzn1.x86_64 #1 SMP Wed Oct 3 19:54:33 UTC 2012 
 x86_64 x86_64 x86_64 GNU/Linux
 
 total   used   free sharedbuffers cached 
 Mem:  3754   3695 59  0 37 75 
 -/+ buffers/cache:   3582171 
 Swap: 2055165   1890
 
 
 
 2013/4/16 Mikhail Mazursky ash...@gmail.com
 Hello.
 
 C* have been running without any problem for some weeks but now it started to 
 consume all available ram.
 
 The cluster have very little data in it. There are no errors in logs, CPU is 
 not loaded at all, jstack shows no deadlocks, there are 83 threads. 
 Read/write latency is 1-4ms.
 
 The question is: how to find out where is the RAM used?
 
 # nodetool ring keyspace name
 Address DC  RackStatus State   Load
 Effective-Ownership Token
   
  114967934986393518082067045023985443411  
 10.0.1.220  datacenter1 rack1   Up Normal  889.71 KB   
 100.00% 0
 10.0.1.222  datacenter1 rack1   Up Normal  918.92 KB   
 100.00% 56662868904138943096229641828625627278   
 10.0.1.221  datacenter1 rack1   Up Normal  1.04 MB 
 100.00% 114967934986393518082067045023985443411
 
 We use AWS m1.medium - 3.75 RAM, 1 Core 
 
 java version 1.7.0_10 
 Java(TM) SE Runtime Environment (build 1.7.0_10-b18) 
 Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode)
 
 C* is started with the following command
 /usr/java/latest/bin/java -ea 
 -javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar -XX:+UseThreadPriorities 
 -XX:ThreadPriorityPolicy=42 -Xms1G -Xmx1G -Xmn512M 
 -XX:+HeapDumpOnOutOfMemoryError -Xss256k -XX:+UseParNewGC
 
 Thanks in advance.
 
 Mikhail.
 



Re: C* consumes all RAM

2013-04-16 Thread Mikhail Mazursky
Thank you, Aaron.

p.s. we're on 1.1.9 - i forgot to mention that.


2013/4/17 aaron morton aa...@thelastpickle.com

 You are probably seeing this http://wiki.apache.org/cassandra/FAQ#mmap

 Cheers

-
 Aaron Morton
 Freelance Cassandra Consultant
 New Zealand

 @aaronmorton
 http://www.thelastpickle.com

 On 16/04/2013, at 8:43 PM, Mikhail Mazursky ash...@gmail.com wrote:

 More details:

 USER   PID %CPU %MEMVSZ   RSS TTY  STAT START   TIME COMMAND
 219   3801  0.7 92.7 6561116 3567016 ? SLl  Mar11 372:44
 /usr/java/latest/bin/java

 Linux XXX@YYY 3.2.30-49.59.amzn1.x86_64 #1 SMP Wed Oct 3 19:54:33 UTC
 2012 x86_64 x86_64 x86_64 GNU/Linux

 total   used   free sharedbuffers cached
 Mem:  3754   3695 59  0 37 75
 -/+ buffers/cache:   3582171
 Swap: 2055165   1890



 2013/4/16 Mikhail Mazursky ash...@gmail.com

 Hello.

 C* have been running without any problem for some weeks but now it
 started to consume all available ram.

 The cluster have very little data in it. There are no errors in logs, CPU
 is not loaded at all, jstack shows no deadlocks, there are 83 threads.
 Read/write latency is 1-4ms.

 The question is: how to find out where is the RAM used?

 # nodetool ring keyspace name
 Address DC  RackStatus State   Load
 Effective-Ownership Token

 114967934986393518082067045023985443411
 10.0.1.220  datacenter1 rack1   Up Normal  889.71 KB
 100.00% 0
 10.0.1.222  datacenter1 rack1   Up Normal  918.92 KB
 100.00% 56662868904138943096229641828625627278
 10.0.1.221  datacenter1 rack1   Up Normal  1.04 MB
 100.00% 114967934986393518082067045023985443411

 We use AWS m1.medium - 3.75 RAM, 1 Core

 java version 1.7.0_10
 Java(TM) SE Runtime Environment (build 1.7.0_10-b18)
 Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode)

 C* is started with the following command
 /usr/java/latest/bin/java -ea
 -javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar -XX:+UseThreadPriorities
 -XX:ThreadPriorityPolicy=42 -Xms1G -Xmx1G -Xmn512M
 -XX:+HeapDumpOnOutOfMemoryError -Xss256k -XX:+UseParNewGC

 Thanks in advance.

 Mikhail.