[
https://issues.apache.org/jira/browse/CASSANDRA-8860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342032#comment-14342032
]
Phil Yang edited comment on CASSANDRA-8860 at 3/1/15 7:42 AM:
--------------------------------------------------------------
Furthermore, if my algorithm is correct, my patch-v2 still need a 2-d loop to
get the undirected graph. The space complexity is still O(N^2) if all the
SSTables overlap each other and the graph is a complete graph. So I think there
is a way to optimize it:
Use a 2-d bitmap matrix with the boolean which means whether coldSSTables[i]
overlaps coldSSTables[j] to replace the 2-d collection of SSTableReader. Each
HashMap$Entry need 32*8=256 bits but each bitmap boolean needs only 1 bit. This
method may need more memory in some case, but need much less in the worst case.
If my algorithm is not wrong, we can discuss if this optimization is needed or
not :)
was (Author: yangzhe1991):
Furthermore, if my algorithm is correct, my patch-v2 still need a 2-d loop to
get the undirected graph. The space complexity is still O(N^2) if all the
SSTables overlap each other. So I think there is a way to optimize it:
Use a 2-d bitmap matrix with the boolean which means whether coldSSTables[i]
overlaps coldSSTables[j] to replace the 2-d collection of SSTableReader. Each
HashMap$Entry need 32*8=256 bits but each bitmap boolean needs only 1 bit. This
method may need more memory in some case, but need much less in the worst case.
If my algorithm is not wrong, we can discuss if this optimization is needed or
not :)
> Too many java.util.HashMap$Entry objects in heap
> ------------------------------------------------
>
> Key: CASSANDRA-8860
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8860
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 2.1.3, jdk 1.7u51
> Reporter: Phil Yang
> Assignee: Phil Yang
> Fix For: 2.1.4
>
> Attachments: 8860-v2.txt, 8860.txt, cassandra-env.sh, cassandra.yaml,
> jmap.txt, jstack.txt, jstat-afterv1.txt, jstat-afterv2.txt, jstat-before.txt
>
>
> While I upgrading my cluster to 2.1.3, I find some nodes (not all) may have
> GC issue after the node restarting successfully. Old gen grows very fast and
> most of the space can not be recycled after setting its status to normal
> immediately. The qps of both reading and writing are very low and there is no
> heavy compaction.
> Jmap result seems strange that there are too many java.util.HashMap$Entry
> objects in heap, where in my experience the "[B" is usually the No1.
> If I downgrade it to 2.1.1, this issue will not appear.
> I uploaded conf files and jstack/jmap outputs. I'll upload heap dump if
> someone need it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
