[ https://issues.apache.org/jira/browse/CASSANDRA-2357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams resolved CASSANDRA-2357. ----------------------------------------- Resolution: Not A Problem > Load spikes on coordinators since upgrade from 0.6.8 to 0.7 > ----------------------------------------------------------- > > Key: CASSANDRA-2357 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2357 > Project: Cassandra > Issue Type: Bug > Affects Versions: 0.7.4 > Reporter: Jason Harvey > Attachments: thread_dump.txt > > > Since our move from 0.6.8 to 0.7, all of the nodes which speak with clients > have been having periodic, abrupt load spikes going into the hundreds. We > have been seeing these load spikes 1 to 2 times per hour on every node which > clients are speaking with. The load graph for a typical spike: > http://i.imgur.com/jY8AV.png > I have verified that client connections are not spiking at the same time via > TCP statistics. I have also verified that we aren't seeing any spikes in > reads/mutations/etc. > We were using the DynamicSnitch, but I turned that off as a troubleshooting > step. The issue was unchanged. > When the spikes occur, the box's responsiveness slows to a crawl so I am > unable to do much in terms of real-time diagnostics. I was able to get a > thread dump a few seconds after a spike, which I have attached to this > ticket. Not sure if it will show anything since I couldn't capture it > immediately during the spike. > I should note that David King noticed a similar problem (#2058) when he tried > moving us from 0.6.8 to 0.6.10. The main issue at the time was a long-lasting > load spike, but he also saw occasional abrupt load spikes like we are seeing > now. When we moved back to 0.6.8, we didn't see the problem again, until the > move to 0.7. > I realize this information is somewhat nebulous. If there is any further info > I can provide, please let me know. The spikes are causing quite a bit of > instability, so we are considering retreating back to 0.6.8. I'd like to > investigate every possible solution before we resort to that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira