[
https://issues.apache.org/jira/browse/CASSANDRA-16847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Ramirez updated CASSANDRA-16847:
--------------------------------------
Description:
User [Yorick|https://the-asf.slack.com/team/U026YMYS2QN] reported on ASF Slack
that a node which failed to bootstrap is stuck in {{hibernate}} status and
persists in {{gossipinfo}}:
{noformat}
/10.x.x.108
generation:1625493756
heartbeat:86
STATUS:2:hibernate,true
LOAD:79:96124.0
SCHEMA:14:59adb24e-f3cd-3e02-97f0-5b395827453f
DC:10:DC1
RACK:12:RAC3
RELEASE_VERSION:6:3.11.10
INTERNAL_IP:8:10.x.x.108
RPC_ADDRESS:5:10.x.x.108
NET_VERSION:3:11
HOST_ID:4:5b254d51-fc58-4ca2-856f-fe7878752131
TOKENS:1:<hidden> {noformat}
Attempts to assassinate the node returns {{IndexOutOfBoundsException}}:
{noformat}
ERROR [GossipStage:1] 2021-08-11 09:10:03,440 CassandraDaemon.java:244 -
Exception in thread Thread[GossipStage:1,5,main]
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:659) ~[na:1.8.0_292]
at java.util.ArrayList.get(ArrayList.java:435) ~[na:1.8.0_292]
at com.google.common.collect.Iterables.get(Iterables.java:728)
~[guava-18.0.jar:na]
at
org.apache.cassandra.gms.VersionedValue$VersionedValueFactory.makeTokenString(VersionedValue.java:156)
~[apache-cassandra-3.11.10.jar:3.11.10]
at
org.apache.cassandra.gms.VersionedValue$VersionedValueFactory.left(VersionedValue.java:178)
~[apache-cassandra-3.11.10.jar:3.11.10]
at
org.apache.cassandra.gms.Gossiper.lambda$assassinateEndpoint$1(Gossiper.java:695)
~[apache-cassandra-3.11.10.jar:3.11.10]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_292]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_292]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[na:1.8.0_292]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[na:1.8.0_292]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84)
[apache-cassandra-3.11.10.jar:3.11.10]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_292] {noformat}
was:
User Yorick reported on ASF Slack that a node which failed to bootstrap is
stuck in {{hibernate}} status and persists in {{gossipinfo}}:
{noformat}
/10.x.x.108
generation:1625493756
heartbeat:86
STATUS:2:hibernate,true
LOAD:79:96124.0
SCHEMA:14:59adb24e-f3cd-3e02-97f0-5b395827453f
DC:10:DC1
RACK:12:RAC3
RELEASE_VERSION:6:3.11.10
INTERNAL_IP:8:10.x.x.108
RPC_ADDRESS:5:10.x.x.108
NET_VERSION:3:11
HOST_ID:4:5b254d51-fc58-4ca2-856f-fe7878752131
TOKENS:1:<hidden> {noformat}
Attempts to assassinate the node returns {{IndexOutOfBoundsException}}:
{noformat}
ERROR [GossipStage:1] 2021-08-11 09:10:03,440 CassandraDaemon.java:244 -
Exception in thread Thread[GossipStage:1,5,main]
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:659) ~[na:1.8.0_292]
at java.util.ArrayList.get(ArrayList.java:435) ~[na:1.8.0_292]
at com.google.common.collect.Iterables.get(Iterables.java:728)
~[guava-18.0.jar:na]
at
org.apache.cassandra.gms.VersionedValue$VersionedValueFactory.makeTokenString(VersionedValue.java:156)
~[apache-cassandra-3.11.10.jar:3.11.10]
at
org.apache.cassandra.gms.VersionedValue$VersionedValueFactory.left(VersionedValue.java:178)
~[apache-cassandra-3.11.10.jar:3.11.10]
at
org.apache.cassandra.gms.Gossiper.lambda$assassinateEndpoint$1(Gossiper.java:695)
~[apache-cassandra-3.11.10.jar:3.11.10]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_292]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_292]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[na:1.8.0_292]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[na:1.8.0_292]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84)
[apache-cassandra-3.11.10.jar:3.11.10]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_292] {noformat}
> nodetool assassinate of a node that failed bootstrap may lead to
> IndexOutOfBoundsException
> ------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-16847
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16847
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip, Cluster/Membership,
> Consistency/Bootstrap and Decommission
> Reporter: Erick Ramirez
> Priority: Normal
>
> User [Yorick|https://the-asf.slack.com/team/U026YMYS2QN] reported on ASF
> Slack that a node which failed to bootstrap is stuck in {{hibernate}} status
> and persists in {{gossipinfo}}:
> {noformat}
> /10.x.x.108
> generation:1625493756
> heartbeat:86
> STATUS:2:hibernate,true
> LOAD:79:96124.0
> SCHEMA:14:59adb24e-f3cd-3e02-97f0-5b395827453f
> DC:10:DC1
> RACK:12:RAC3
> RELEASE_VERSION:6:3.11.10
> INTERNAL_IP:8:10.x.x.108
> RPC_ADDRESS:5:10.x.x.108
> NET_VERSION:3:11
> HOST_ID:4:5b254d51-fc58-4ca2-856f-fe7878752131
> TOKENS:1:<hidden> {noformat}
> Attempts to assassinate the node returns {{IndexOutOfBoundsException}}:
> {noformat}
> ERROR [GossipStage:1] 2021-08-11 09:10:03,440 CassandraDaemon.java:244 -
> Exception in thread Thread[GossipStage:1,5,main]
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
> at java.util.ArrayList.rangeCheck(ArrayList.java:659) ~[na:1.8.0_292]
> at java.util.ArrayList.get(ArrayList.java:435) ~[na:1.8.0_292]
> at com.google.common.collect.Iterables.get(Iterables.java:728)
> ~[guava-18.0.jar:na]
> at
> org.apache.cassandra.gms.VersionedValue$VersionedValueFactory.makeTokenString(VersionedValue.java:156)
> ~[apache-cassandra-3.11.10.jar:3.11.10]
> at
> org.apache.cassandra.gms.VersionedValue$VersionedValueFactory.left(VersionedValue.java:178)
> ~[apache-cassandra-3.11.10.jar:3.11.10]
> at
> org.apache.cassandra.gms.Gossiper.lambda$assassinateEndpoint$1(Gossiper.java:695)
> ~[apache-cassandra-3.11.10.jar:3.11.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_292]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_292]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> ~[na:1.8.0_292]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_292]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84)
> [apache-cassandra-3.11.10.jar:3.11.10]
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_292] {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
