If bloom filters are taking too much memory, you can adjust bloom filters: https://docs.datastax.com/en/cassandra/2.1/cassandra/operations/ops_tuning_bloom_filters_c.html <https://docs.datastax.com/en/cassandra/2.1/cassandra/operations/ops_tuning_bloom_filters_c.html>
Hannu > On 4 Mar 2017, at 22:54, Thakrar, Jayesh <jthak...@conversantmedia.com> wrote: > > I have a situation where the off-heap memory is bloating the jvm process > memory, making it a candidate to be killed by the oom_killer. > My server has 256 GB RAM and Cassandra heap memory of 16 GB > > Below is the output of "nodetool info" and nodetool compactionstats for a > culprit table which causes bloom filter bloat. > Ofcourse one option is to turnoff bloom filter, but I need to look into > application access pattern, etc. > > > xss = -ea -Dorg.xerial.snappy.tempdir=/home/vchadoop/var/tmp > -javaagent:/home/vchadoop/apps/apache-cassandra-2.2.5//lib/jamm-0.3.0.jar > -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms16G -Xmx16G > -Xmn4800M -XX:+HeapDumpOnOutOfMemoryError -Xss256k > ID : 2b9b4252-0760-49c1-8d14-544be0183271 > Gossip active : true > Thrift active : false > Native Transport active: true > Load : 953.19 GB > Generation No : 1488641545 > Uptime (seconds) : 15706 > Heap Memory (MB) : 7692.93 / 16309.00 > Off Heap Memory (MB) : 175115.07 > Data Center : ord > Rack : rack3 > Exceptions : 0 > Key Cache : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0 > requests, NaN recent hit rate, 14400 save period in seconds > Row Cache : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0 > requests, NaN recent hit rate, 0 save period in seconds > Counter Cache : entries 0, size 0 bytes, capacity 50 MB, 0 hits, 0 > requests, NaN recent hit rate, 7200 save period in seconds > Token : (invoke with -T/--tokens to see all 256 tokens) > > > Table: logs_by_user > SSTable count: 622 > SSTables in each level: [174/4, 447/10, 0, 0, 0, 0, 0, 0, 0] > Space used (live): 313156769247 > Space used (total): 313156769247 > Space used by snapshots (total): 0 > Off heap memory used (total): 180354511884 > SSTable Compression Ratio: 0.25016314078395613 > Number of keys (estimate): 147261312 > Memtable cell count: 44796 > Memtable data size: 57578717 > Memtable off heap memory used: 0 > Memtable switch count: 21 > Local read count: 0 > Local read latency: NaN ms > Local write count: 1148687 > Local write latency: 0.123 ms > Pending flushes: 0 > Bloom filter false positives: 0 > Bloom filter false ratio: 0.00000 > Bloom filter space used: 180269125192 > Bloom filter off heap memory used: 180269120216 > Index summary off heap memory used: 24335340 > Compression metadata off heap memory used: 61056328 > Compacted partition minimum bytes: 150 > Compacted partition maximum bytes: 668489532 > Compacted partition mean bytes: 3539 > Average live cells per slice (last five minutes): NaN > Maximum live cells per slice (last five minutes): 0 > Average tombstones per slice (last five minutes): NaN > Maximum tombstones per slice (last five minutes): 0 > > > From: Conversant <jthak <mailto:jthak...@conversantmedia.com>
