What is your Native heap size? And how are you measuring memory usage? It woudl also hep to see the commandlien you are using to launch the JVM On Mon, Jun 11, 2012 at 9:14 PM, Jason Tang <ares.t...@gmail.com> wrote:
> Hi > > We have some problem with Cassandra memory usage, we configure the JVM > HEAP 6G, but after runing Cassandra for several hours (insert, update, > delete). The total memory used by Cassandra go up to 15G, which cause the > OS low memory. > So I wonder if it is normal to have so many memory used by cassandra? > > And how to limit the native memory used by Cassandra? > > > =========================== > Cassandra 1.0.3, 64 bit jdk. > > Memory ocupied by Cassandra 15G > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 9567 casadm 20 0 28.3g 15g 9.1g S 269 65.1 385:57.65 java > > ============================= > -Xms6G -Xmx6G -Xmn1600M > > # ps -ef | grep 9567 > casadm 9567 1 55 Jun11 ? 05:59:44 /opt/jdk1.6.0_29/bin/java > -ea -javaagent:/opt/dve/cassandra/bin/../lib/jamm-0.2.5.jar > -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms6G -Xmx6G > -Xmn1600M -XX:+HeapDumpOnOutOfMemoryError -Xss128k -XX:+UseParNewGC > -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 > -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 > -XX:+UseCMSInitiatingOccupancyOnly -Djava.net.preferIPv4Stack=true > -Dcom.sun.management.jmxremote.port=6080 > -Dcom.sun.management.jmxremote.ssl=false > -Dcom.sun.management.jmxremote.authenticate=false > -Daccess.properties=/opt/dve/cassandra/conf/access.properties > -Dpasswd.properties=/opt/dve/cassandra/conf/passwd.properties > -Dpasswd.mode=MD5 -Dlog4j.configuration=log4j-server.properties > -Dlog4j.defaultInitOverride=true -cp > /opt/dve/cassandra/bin/../conf:/opt/dve/cassandra/bin/../build/classes/main:/opt/dve/cassandra/bin/../build/classes/thrift:/opt/dve/cassandra/bin/../lib/Cassandra-Extensions-1.0.0.jar:/opt/dve/cassandra/bin/../lib/antlr-3.2.jar:/opt/dve/cassandra/bin/../lib/apache-cassandra-1.0.3.jar:/opt/dve/cassandra/bin/../lib/apache-cassandra-clientutil-1.0.3.jar:/opt/dve/cassandra/bin/../lib/apache-cassandra-thrift-1.0.3.jar:/opt/dve/cassandra/bin/../lib/avro-1.4.0-fixes.jar:/opt/dve/cassandra/bin/../lib/avro-1.4.0-sources-fixes.jar:/opt/dve/cassandra/bin/../lib/commons-cli-1.1.jar:/opt/dve/cassandra/bin/../lib/commons-codec-1.2.jar:/opt/dve/cassandra/bin/../lib/commons-lang-2.4.jar:/opt/dve/cassandra/bin/../lib/compress-lzf-0.8.4.jar:/opt/dve/cassandra/bin/../lib/concurrentlinkedhashmap-lru-1.2.jar:/opt/dve/cassandra/bin/../lib/guava-r08.jar:/opt/dve/cassandra/bin/../lib/high-scale-lib-1.1.2.jar:/opt/dve/cassandra/bin/../lib/jackson-core-asl-1.4.0.jar:/opt/dve/cassandra/bin/../lib/jackson-mapper-asl-1.4.0.jar:/opt/dve/cassandra/bin/../lib/jamm-0.2.5.jar:/opt/dve/cassandra/bin/../lib/jline-0.9.94.jar:/opt/dve/cassandra/bin/../lib/json-simple-1.1.jar:/opt/dve/cassandra/bin/../lib/libthrift-0.6.jar:/opt/dve/cassandra/bin/../lib/log4j-1.2.16.jar:/opt/dve/cassandra/bin/../lib/servlet-api-2.5-20081211.jar:/opt/dve/cassandra/bin/../lib/slf4j-api-1.6.1.jar:/opt/dve/cassandra/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/dve/cassandra/bin/../lib/snakeyaml-1.6.jar:/opt/dve/cassandra/bin/../lib/snappy-java-1.0.4.1.jar > org.apache.cassandra.thrift.CassandraDaemon > > ============================== > # nodetool -h 127.0.0.1 -p 6080 info > Token : 85070591730234615865843651857942052864 > Gossip active : true > Load : 20.59 GB > Generation No : 1339423322 > Uptime (seconds) : 39626 > Heap Memory (MB) : 3418.42 / 5984.00 > Data Center : datacenter1 > Rack : rack1 > Exceptions : 0 > > ============================= > All row cache and key cache are disabled by default > > Key cache: disabled > Row cache: disabled > > > ============================== > > # pmap 9567 > 9567: java > START SIZE RSS PSS DIRTY SWAP PERM MAPPING > 0000000040000000 36K 36K 36K 0K 0K r-xp > /opt/jdk1.6.0_29/bin/java > 0000000040108000 8K 8K 8K 8K 0K rwxp > /opt/jdk1.6.0_29/bin/java > 000000004010a000 18040K 17988K 17988K 17988K 0K rwxp [heap] > 000000067ae00000 6326700K 6258664K 6258664K 6258664K 0K rwxp [anon] > 00000007fd06b000 48724K 0K 0K 0K 0K rwxp [anon] > 00007fbed1530000 1331104K 0K 0K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-219-Data.db > 00007fbf22918000 2097152K 0K 0K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-219-Data.db > 00007fbfa2918000 2097148K 1124464K 1124462K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-219-Data.db > 00007fc022917000 2097156K 2096496K 2096492K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-219-Data.db > 00007fc0a2918000 2097148K 2097148K 2097146K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-219-Data.db > 00007fc1a2917000 733584K 6444K 6444K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-109-Data.db > 00007fc1cf57b000 2097148K 20980K 20980K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-109-Data.db > 00007fc24f57a000 2097152K 456480K 456478K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-109-Data.db > 00007fc2cf57a000 2097156K 1168320K 1168318K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-109-Data.db > 00007fc34f57b000 2097148K 1177520K 1177520K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-109-Data.db > 00007fc405629000 618708K 338248K 338248K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-230-Data.db > 00007fc42b25e000 620388K 289024K 289024K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-224-Data.db > 00007fc451037000 619160K 342108K 342108K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-216-Data.db > 00007fc62b7df000 132696K 0K 0K 0K 0K r-xs > /var/cassandra/data/drc/queue.idxInQueueTime-hb-175-Data.db > 00007fc6de8e0000 132696K 0K 0K 0K 0K r-xs > /var/cassandra/data/drc/queue.idxRecvTime-hb-175-Data.db > 00007fc6f2bcc000 52492K 0K 0K 0K 0K r-xs > /var/cassandra/data/drc/queue.idxPartitionId-hb-211-Data.db > 00007fc6f64dc000 43784K 40840K 40840K 0K 0K r-xs > /var/cassandra/data/drc/fpr_index-hb-91-Data.db > 00007fc707ca6000 68968K 37724K 37724K 0K 0K r-xs > /var/cassandra/data/drc/queue-hb-219-Index.db > 00007fc70c000000 2468K 2436K 2436K 2436K 0K rwxp [anon] > 00007fc70c269000 63068K 0K 0K 0K 0K ---p [anon] > 00007fc710b9e000 52888K 0K 0K 0K 0K r-xs > /var/cassandra/data/drc/queue.idxInQueueTime-hb-216-Data.db > 00007fc713f44000 52952K 0K 0K 0K 0K r-xs > /var/cassandra/data/drc/queue.idxFireTimeRange-hb-140-Data.db > 00007fc7172fa000 52952K 0K 0K 0K 0K r-xs > /var/cassandra/data/drc/queue.idxFireTime-hb-140-Data.db > 00007fc71bd13000 162992K 162984K 162984K 0K 0K r-xs > /var/cassandra/data/drc/fpr_index-hb-80-Data.db > 00007fc725c3f000 52952K 28712K 28712K 0K 0K r-xs > /var/cassandra/data/drc/queue.idxInQueueTimeRange-hb-140-Data.db > 00007fc728ff5000 52952K 0K 0K 0K 0K r-xs > /var/cassandra/data/drc/queue.idxRecvTimeRange-hb-140-Data.db > 00007fc72d026000 52480K 0K 0K 0K 0K r-xs > /var/cassandra/data/drc/queue.idxRecvTimeRange-hb-211-Data.db > 00007fc730366000 52564K 0K 0K 0K 0K r-xs > /var/cassandra/data/drc/queue.idxStatus-hb-196-Data.db > 00007fc7336bb000 22348K 0K 0K 0K 0K r-xs > /var/cassandra/data/drc/queue.idxInQueueTime-hb-175-Index.db > > > BRs > //Ares Tang > -- It's always darkest just before you are eaten by a grue.
