[ 
https://issues.apache.org/jira/browse/CASSANDRA-9914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14645037#comment-14645037
 ] 

Robbie Strickland edited comment on CASSANDRA-9914 at 7/28/15 8:57 PM:
-----------------------------------------------------------------------

[~yukim] Possibly, but I don't think so.  The symptoms in CASSANDRA-9662 are 
momentary, whereas this persists indefinitely (and gets worse over time) with 
much higher task counts.


was (Author: rstrickland):
[~yukim] Possibly, but I don't think so.  The symptoms in CASSANDRA-9662 are 
momentary, whereas this is persists indefinitely (and gets worse over time) 
with much higher task counts.

> Millions of fake pending compaction tasks + high CPU
> ----------------------------------------------------
>
>                 Key: CASSANDRA-9914
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9914
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: CentOS
>            Reporter: Robbie Strickland
>            Assignee: Marcus Eriksson
>             Fix For: 2.1.x
>
>         Attachments: cass_high_cpu.png, high_pending_compactions.txt
>
>
> We have a 3-node test cluster (initially running 2.1.8) with *zero traffic* 
> and about 10GB of data on each node.  It's showing millions of pending 
> compaction tasks (but no actual work in progress), and the CPUs are pegged on 
> all three nodes.  The task count goes down rapidly, but then jumps back up 
> again seconds later.  All tables are set to STCS.  The issue persists after 
> restart, but takes a few minutes before it becomes a problem.  SSTable counts 
> are below 10 for every table.  We're also seeing 20s Old Gen GC pauses about 
> every 2-3 mins.
> This started happening after bulk loading some old data.  We started seeing 
> very long GC pauses (sometimes 30 min or more) that would bring down the 
> nodes.  We then truncated this table, which resulted in the current behavior. 
>  We attempted to roll back our cluster to 2.1.7 patched with CASSANDRA-9637, 
> but we observed the same behavior.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to