[ https://issues.apache.org/jira/browse/CASSANDRA-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sanjay resolved CASSANDRA-5673. ------------------------------- Resolution: Fixed Fix Version/s: 1.2.6 After Upgrading the problem seems to be fixed. > NullPointerException on running instances > ----------------------------------------- > > Key: CASSANDRA-5673 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5673 > Project: Cassandra > Issue Type: Bug > Components: Core > Affects Versions: 1.2.4 > Reporter: Sanjay > Fix For: 1.2.6 > > > Hallo, > We are having sporadic NullPointerException in some of the cassandra nodes in > cluster (See stacktrace). > We are having two Datacenter, each having 15 nodes with RF = 2, OS is SLES > with java-1_6_0-ibm-1.6.0_sr12.0-0.5.1. > At present only workaround is to stop the application running on same node > and run repair tool on cassandra. We are unable to identify the cause of > error. > 1) > INFO|ScheduledTasks:1|org.apache.cassandra.service.GCInspector|GC for > MarkSweepCompact: 347 ms for 1 collections, 138398568 used; ma > x is 1051721728 > 2013-06-19T16:25:50:843|ERROR|ReplicateOnWriteStage:115|org.apache.cassandra.service.CassandraDaemon|Exception > in thread Thread[ReplicateOnWriteStage:115,5,m > ain] > java.lang.RuntimeException: java.lang.NullPointerException > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1582) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > at java.lang.Thread.run(Thread.java:738) > Caused by: java.lang.NullPointerException > at java.util.TreeSet.iterator(TreeSet.java:230) > at > org.apache.cassandra.db.columniterator.SSTableNamesIterator.read(SSTableNamesIterator.java:163) > at > org.apache.cassandra.db.columniterator.SSTableNamesIterator.<init>(SSTableNamesIterator.java:64) > at > org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:81) > at > org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:68) > at > org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:274) > at > org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65) > at > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1357) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1214) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1126) > at org.apache.cassandra.db.Table.getRow(Table.java:347) > at > org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:64) > at > org.apache.cassandra.db.CounterMutation.makeReplicationMutation(CounterMutation.java:90) > at > org.apache.cassandra.service.StorageProxy$7$1.runMayThrow(StorageProxy.java:796) > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1578) > ... 3 more > 2013-06-19T16:26:01:001|ERROR|ReadStage:4833|org.apache.cassandra.service.CassandraDaemon|Exception > in thread Thread[ReadStage:4833,5,main] > java.lang.RuntimeException: java.lang.NullPointerException > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1582) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > at java.lang.Thread.run(Thread.java:738) > Caused by: java.lang.NullPointerException > at java.util.TreeSet.iterator(TreeSet.java:230) > at > org.apache.cassandra.db.columniterator.SSTableNamesIterator.read(SSTableNamesIterator.java:163) > at org. > 2) > 2013-06-19T08:38:23:436| > INFO|Thread-2447|org.apache.cassandra.service.StorageService|Starting repair > command #2, repairing 1 ranges for keyspace system_auth > 2013-06-19T08:58:25:685|ERROR|ReadStage:9270|org.apache.cassandra.service.CassandraDaemon|Exception > in thread Thread[ReadStage:9270,5,main] > java.lang.NullPointerException > at java.util.TreeSet.iterator(TreeSet.java:230) > at > org.apache.cassandra.db.columniterator.SSTableNamesIterator.read(SSTableNamesIterator.java:163) > at > org.apache.cassandra.db.columniterator.SSTableNamesIterator.<init>(SSTableNamesIterator.java:64) > at > org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:81) > at > org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:68) > at > org.apache.cassandra.db.CollationController.collectTimeOrderedData(CollationController.java:133) > at > org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65) > at > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1357) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1214) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1126) > at org.apache.cassandra.db.Table.getRow(Table.java:347) > at > org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:64) > at > org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:44) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:908) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:931) > at java.lang.Thread.run(Thread.java:738) > Best Regards > Sanjay -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira