[ 
https://issues.apache.org/jira/browse/CASSANDRA-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882740#action_12882740
 ] 

Jonathan Ellis commented on CASSANDRA-1014:
-------------------------------------------

This is sounding more and more to me like a bug in CMS, since stop-the-world 
collections make it go away.  (We're not allocating *that* much during a sweep!)

> GC storming, possible memory leak
> ---------------------------------
>
>                 Key: CASSANDRA-1014
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1014
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.6
>         Environment: debian lenny amd64 OpenJDK 64-Bit Server VM (build 
> 1.6.0_0-b11, mixed mode)
>            Reporter: Brandon Williams
>            Assignee: Jonathan Ellis
>             Fix For: 0.6.4
>
>         Attachments: 1014-2Gheap.png, 1014-commitlog-v2.tar.gz, 
> 1014-table.diff, 724-0001.png, gc2.png
>
>
> There appears to be a GC issue due to memory pressure in the 0.6 branch.  You 
> can see this by starting the server and performing many inserts.  Quickly the 
> jvm will consume most of its heap, and pauses for stop-the-world GC will 
> begin.  With verbose GC turned on, this can be observed as follows:
> [GC [ParNew (promotion failed): 79703K->79703K(84544K), 0.0622980 
> secs][CMS[CMS-concurrent-mark: 3.678/5.031 secs] [Times: user=10.35 sys=4.22, 
> real=5.03 secs]
>  (concurrent mode failure): 944529K->492222K(963392K), 2.8264480 secs] 
> 990745K->492222K(1047936K), 2.8890500 secs] [Times: user=2.90 sys=0.04, 
> real=2.90 secs]
> After enough inserts (around 75-100 million) the server will GC storm and 
> then OOM.
> jbellis and I narrowed this down to patch 0001 in CASSANDRA-724.  Switching 
> LBQ with ABQ made no difference, however using batch mode instead of periodic 
> for the commitlog does prevent the issue from occurring.  The attached 
> screenshot shows the heap usage in jconsole first when the issue is 
> exhibiting, a restart, and then the same amount of inserts when it does not.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to