[
https://issues.apache.org/jira/browse/CASSANDRA-15263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
feroz shaik updated CASSANDRA-15263:
------------------------------------
Description:
We have also hit a problem today while upgrading from 2.1.16 to 3.11.4.
we encountered this as soon as the first node started up with 3.11.4
The full error stack is attached - [^stack_trace.txt]
The below errors continued in the log file as long as the process was up.
ERROR [Native-Transport-Requests-12] 2019-08-06 03:00:47,135
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:48,778
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-13] 2019-08-06 03:00:57,454
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-11] 2019-08-06 03:00:57,482
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-2] 2019-08-06 03:00:58,543
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:58,899
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-17] 2019-08-06 03:00:59,074
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-12] 2019-08-06 03:01:08,123
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-17] 2019-08-06 03:01:19,055
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-4] 2019-08-06 03:01:20,880
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
WARN [ReadStage-13] 2019-08-06 03:01:29,983
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-13,5,main]: {}
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-2] 2019-08-06 03:01:31,119
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-6] 2019-08-06 03:01:46,262
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-15] 2019-08-06 03:01:46,520
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
WARN [ReadStage-2] 2019-08-06 03:01:48,842
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-2,5,main]: {}
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-1] 2019-08-06 03:01:50,351
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-5] 2019-08-06 03:02:06,061
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
WARN [ReadStage-8] 2019-08-06 03:02:07,616
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-8,5,main]: {}
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-17] 2019-08-06 03:02:08,384
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-5] 2019-08-06 03:02:10,244
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
The nodetool version says 3.11.4 and the no of connections on native por t-
9042 was similar to other nodes. The exceptions were scary that we had to call
off the change. Any help and insights to this problem from the community is
appreciated.
was:
We have also hit a problem today while upgrading from 2.1.16 to 3.11.4.
we encountered this as soon as the first node started up with 3.11.4
The full error stack is as below:
WARN [ReadStage-4] 2019-08-06 02:57:57,408
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-4,5,main]: {}
java.lang.NullPointerException: null
at
org.apache.cassandra.db.LegacyLayout$LegacyRangeTombstoneList.updateDigest(LegacyLayout.java:2433)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.db.LegacyLayout$LegacyUnfilteredPartition.digest(LegacyLayout.java:1479)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.db.rows.UnfilteredRowIterators.digest(UnfilteredRowIterators.java:182)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators.digest(UnfilteredPartitionIterators.java:263)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.db.ReadResponse.makeDigest(ReadResponse.java:140)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.db.ReadResponse.createDigestResponse(ReadResponse.java:87)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:352)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:50)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_131]
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
[apache-cassandra-3.11.4.jar:3.11.4]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:114)
[apache-cassandra-3.11.4.jar:3.11.4]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
The below errors continued in the log file as long as the process was up.
ERROR [Native-Transport-Requests-12] 2019-08-06 03:00:47,135
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:48,778
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-13] 2019-08-06 03:00:57,454
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-11] 2019-08-06 03:00:57,482
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-2] 2019-08-06 03:00:58,543
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:58,899
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-17] 2019-08-06 03:00:59,074
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-12] 2019-08-06 03:01:08,123
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-17] 2019-08-06 03:01:19,055
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-4] 2019-08-06 03:01:20,880
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
WARN [ReadStage-13] 2019-08-06 03:01:29,983
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-13,5,main]: {}
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-2] 2019-08-06 03:01:31,119
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-6] 2019-08-06 03:01:46,262
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-15] 2019-08-06 03:01:46,520
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
WARN [ReadStage-2] 2019-08-06 03:01:48,842
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-2,5,main]: {}
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-1] 2019-08-06 03:01:50,351
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-5] 2019-08-06 03:02:06,061
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
WARN [ReadStage-8] 2019-08-06 03:02:07,616
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-8,5,main]: {}
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-17] 2019-08-06 03:02:08,384
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-5] 2019-08-06 03:02:10,244
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
The nodetool version says 3.11.4 and the no of connections on native por t-
9042 was similar to other nodes. The exceptions were scary that we had to call
off the change. Any help and insights to this problem from the community is
appreciated.
> LegacyLayout RangeTombstoneList throws java.lang.NullPointerException: null
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-15263
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15263
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
> Reporter: feroz shaik
> Priority: Normal
> Labels: 2.1.16, 3.11.4
> Attachments: stack_trace.txt
>
>
> We have also hit a problem today while upgrading from 2.1.16 to 3.11.4.
> we encountered this as soon as the first node started up with 3.11.4
> The full error stack is attached - [^stack_trace.txt]
>
> The below errors continued in the log file as long as the process was up.
> ERROR [Native-Transport-Requests-12] 2019-08-06 03:00:47,135
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:48,778
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-13] 2019-08-06 03:00:57,454
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-11] 2019-08-06 03:00:57,482
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-2] 2019-08-06 03:00:58,543
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:58,899
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-17] 2019-08-06 03:00:59,074
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-12] 2019-08-06 03:01:08,123
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-17] 2019-08-06 03:01:19,055
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-4] 2019-08-06 03:01:20,880
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> WARN [ReadStage-13] 2019-08-06 03:01:29,983
> AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
> Thread[ReadStage-13,5,main]: {}
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-2] 2019-08-06 03:01:31,119
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-6] 2019-08-06 03:01:46,262
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-15] 2019-08-06 03:01:46,520
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> WARN [ReadStage-2] 2019-08-06 03:01:48,842
> AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
> Thread[ReadStage-2,5,main]: {}
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-1] 2019-08-06 03:01:50,351
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-5] 2019-08-06 03:02:06,061
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> WARN [ReadStage-8] 2019-08-06 03:02:07,616
> AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
> Thread[ReadStage-8,5,main]: {}
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-17] 2019-08-06 03:02:08,384
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
> ERROR [Native-Transport-Requests-5] 2019-08-06 03:02:10,244
> ErrorMessage.java:384 - Unexpected exception during request
> java.lang.NullPointerException: null
>
> The nodetool version says 3.11.4 and the no of connections on native por t-
> 9042 was similar to other nodes. The exceptions were scary that we had to
> call off the change. Any help and insights to this problem from the community
> is appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
