[
https://issues.apache.org/jira/browse/CASSANDRA-14495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641911#comment-16641911
]
Chris Lohfink commented on CASSANDRA-14495:
-------------------------------------------
long GCs or frequent GCs caused by large partitions would cause problems, yes
makes sense. The heap usage can be at 5% everytime you look and still have that
though in that scenario (as heap fills up, goes down, fills up, goes down
depending on when you look). Its not an indicator of a problem in itself, even
on a perfectly healthy system with no issues you can check and see the heap
usage high. Look at the GC logs or GCInspector reports to see issues. If real
curious use the swiss java knive (sjk) and {{java -jar sjk.jar -p
CASSANDRA_PID}} and you can see the allocation rate and which threads are doing
the allocations. When your pushing 1gb/s objects on heap you will see issues
(hardware dependent on where its a problem).
> Memory Leak /High Memory usage post 3.11.2 upgrade
> --------------------------------------------------
>
> Key: CASSANDRA-14495
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14495
> Project: Cassandra
> Issue Type: Bug
> Components: Metrics
> Reporter: Abdul Patel
> Priority: Major
> Attachments: cas_heap.txt
>
>
> Hi All,
>
> I recently upgraded my non prod cassandra cluster( 4 nodes single DC) from
> 3.10 to 3.11.2 version.
> No issues reported apart from only nodetool info reporting 80% usage .
> I intially had 16GB memory on each node, later i bumped up to 20GB, and
> rebooted all nodes.
> Waited for an week and now again i have seen memory usage more than 80% ,
> 16GB + .
> this means some memory leaks are happening over the time.
> Any one has faced such issue or do we have any workaround ? my 3.11.2 version
> upgrade rollout has been halted because of this bug.
> ===================================================================
> ID : 65b64f5a-7fe6-4036-94c8-8da9c57718cc
> Gossip active : true
> Thrift active : true
> Native Transport active: true
> Load : 985.24 MiB
> Generation No : 1526923117
> Uptime (seconds) : 1097684
> Heap Memory (MB) : 16875.64 / 20480.00
> Off Heap Memory (MB) : 20.42
> Data Center : DC7
> Rack : rac1
> Exceptions : 0
> Key Cache : entries 3569, size 421.44 KiB, capacity 100 MiB,
> 7931933 hits, 8098632 requests, 0.979 recent hit rate, 14400 save period in
> seconds
> Row Cache : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0
> requests, NaN recent hit rate, 0 save period in seconds
> Counter Cache : entries 0, size 0 bytes, capacity 50 MiB, 0 hits, 0
> requests, NaN recent hit rate, 7200 save period in seconds
> Chunk Cache : entries 2361, size 147.56 MiB, capacity 3.97 GiB,
> 2412803 misses, 72594047 requests, 0.967 recent hit rate, NaN microseconds
> miss latency
> Percent Repaired : 99.88086234106282%
> Token : (invoke with -T/--tokens to see all 256 tokens)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
