[
https://issues.apache.org/jira/browse/CASSANDRA-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14614276#comment-14614276
]
Benedict commented on CASSANDRA-9720:
-------------------------------------
If you could grab a bunch of {{top -Hbn 1}} and {{jstack}} outputs while the
node is exhibiting this behaviour, it might help us narrow down candidates.
> half open tcp connections to cassandra cluster nodes cause 100% cpu load
> ------------------------------------------------------------------------
>
> Key: CASSANDRA-9720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9720
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Alexander Piavlo
>
> cassandra 2.1.5
> We spotted that few of the nodes in our cluster got sudden cpu 100% spike
> which never ended. It's not a GC and not increased reads/writes nodes.
> What we saw is that those nodes that have 100% cpu load
> all have some connections (file descriptios) with "can't identify protocol"
> which indicate those must be unprolery handled abrupt connections by
> cassandra process.
> http://stackoverflow.com/questions/7911840/seeing-too-many-lsof-cant-identify-protocol
> We are pretty sure what triggered this is the spark cassandra connector
> which sudenly started to get stuck in early discovery of cassandra nodes
> before running any stages
> We had to restart the affected cassandra processes to get the cpu back to
> normal
> ps. we had similar issues some time ago with earlier version, of 2.1.x
> cassandra branch, and ended up solving the problerm by upgrading from
> spark1.2.1 to spark1.3.1 and also upgrading spark datastax connecor
> accordingly. Now looks like the problem is back with 99.9% same symptoms
> ps2. We have observed previously several java/cassandra unrelated processes
> (mainly in php-cli) go crazy with cpu then they had "can't identify protocol"
> symphoms
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
