Thanks for the details. I don't know what happened on that node. It's a long time ago I think. I wasn't aware of it earlier.
Will a node in hibernating state that failed joining and subsequently was discarded get removed from gossip at some point? On Wed, Oct 12, 2016 at 5:23 PM, Joel Knighton <joel.knigh...@datastax.com> wrote: > 1. A hibernating node is participating in gossip but intentionally hasn't > yet joined the ring. The two cases where a node would set a hibernating > status are when the node was started with "-Dcassandra.join_ring=False" and > has tokens or when the node was started to replace another node (using > "-Dcassandra.replace_address" or "-Dcassandra.replace_address_ > first_boot"). > > 2. A rolling restart is probably your best bet. You may have more luck > with an assassinate in the case that you connect to a node that is not > continuously removing/adding the state. I suspect that this node will have > an alive status for this endpoint state. As usual, you should wield > assassinate with lots of caution. > > This issue sounds most similar to CASSANDRA-10371. If you provide > debugging information similar to that requested on the above ticket as well > as what operation you were performing on the node (was it a failed attempt > at replacing? etc) on a JIRA ticket, someone might have a chance to look > into this further. > > On Wed, Oct 12, 2016 at 9:48 AM, Kasper Petersen <kas...@sybogames.com> > wrote: > >> Hi, >> >> I've recently upgraded our Cassandra cluster from 2.1 to 3.9. By >> default(?) 3.9 creates a debug.log file containing a ton of lines (a new >> one every second) with: >> >> DEBUG [GossipTasks:1] 2016-10-12 14:43:38,761 Gossiper.java:337 - >>> Convicting /172.31.137.65 with status hibernate - alive false >> >> >> That node has not been around for a very long time now. >> >> It does not show up in nodetool status and nodetool gossipinfo returns >> the following output about that node: >> >> /172.31.137.65 >>> generation:1433571405 >>> heartbeat:232 >>> STATUS:3:hibernate,true >>> LOAD:225:96445.0 >>> SCHEMA:53:e2d1a288-581c-3f35-b492-1b9d5a803631 >>> DC:9:us-east >>> RACK:11:1b >>> RELEASE_VERSION:7:2.1.5 >>> RPC_ADDRESS:6:172.31.137.65 >>> SEVERITY:231:0.2512562870979309 >>> NET_VERSION:4:8 >>> HOST_ID:5:7988d3c9-dec8-4b71-b5a9-0b962aad0680 >>> TOKENS:2:<hidden> >> >> >> nodetool removenode 7988d3c9-dec8-4b71-b5a9-0b962aad0680 resulted in: >> >> error: Host ID not found. >>> >> >> Now my questions are: >> >> 1. What does it mean for a node to be "hibernating"? How does it end >> up in that state? >> 2. How do I get rid of it? Its not coming back. >> >> >> >> -- >> Best regards, >> Kasper Middelboe Petersen >> >> *Lead Backend Developer* >> >> *SYBO Games ApS* >> Jorcks Passage 1A, 4th. >> 1162 Copenhagen K >> > > -- Best regards, Kasper Middelboe Petersen *Lead Backend Developer* *SYBO Games ApS* Jorcks Passage 1A, 4th. 1162 Copenhagen K
