Andre Campeau created CASSANDRA-6669: ----------------------------------------
Summary: NullPointerException(s) on startup if StorageServiceShutdownHook fails Key: CASSANDRA-6669 URL: https://issues.apache.org/jira/browse/CASSANDRA-6669 Project: Cassandra Issue Type: Bug Components: Core Environment: RHEL 6.5 (likely not hardware specific) Reporter: Andre Campeau Priority: Minor Fix For: 2.0.4 For example, when starting a newly installed Cassandra node for the first time, if the seed is down (and the current node is not a seed) a RuntimeException gets thrown by the Gossiper with "Unable to gossip with any seeds", and Cassandra subsequently shuts down. The issue is with the StorageServiceShutdownHook thread which, if any exception is thrown as it's executing, will miss the remaining steps in the shutdown process. Here's a log fragment. INFO [MemoryMeter:1] 2014-01-29 09:07:04,126 Memtable.java (line 451) CFS(Keyspace='system', ColumnFamily='schema_columnfamilies') liveRatio is 2.8177450396801733 (just-counted was 2.5754099604780003). calculation took 459ms for 26900 cells INFO [ScheduledTasks:1] 2014-01-29 09:07:04,168 GCInspector.java (line 116) GC for ParNew: 293 ms for 1 collections, 318930288 used; max is 6215958528 INFO [MemoryMeter:1] 2014-01-29 09:07:04,246 Memtable.java (line 451) CFS(Keyspace='system', ColumnFamily='schema_columns') liveRatio is 2.8940509915872097 (just-counted was 2.6458758011978563). calculation took 119ms for 23968 cells INFO [MemoryMeter:1] 2014-01-29 09:07:07,005 Memtable.java (line 451) CFS(Keyspace='system', ColumnFamily='schema_columnfamilies') liveRatio is 2.6841006146942297 (just-counted was 2.550456189708286). calculation took 297ms for 54325 cells INFO [ScheduledTasks:1] 2014-01-29 09:07:07,326 GCInspector.java (line 116) GC for ParNew: 289 ms for 1 collections, 291162792 used; max is 6215958528 INFO [MemoryMeter:1] 2014-01-29 09:07:07,607 Memtable.java (line 451) CFS(Keyspace='system', ColumnFamily='schema_columns') liveRatio is 2.770798679512837 (just-counted was 2.6475463674384643). calculation took 601ms for 47908 cells INFO [main] 2014-01-29 09:07:09,914 StorageService.java (line 490) Cassandra version: 2.0.4-SNAPSHOT INFO [main] 2014-01-29 09:07:09,914 StorageService.java (line 491) Thrift API version: 19.39.0 INFO [main] 2014-01-29 09:07:09,918 StorageService.java (line 492) CQL supported versions: 2.0.0,3.1.3 (default: 3.1.3) INFO [main] 2014-01-29 09:07:09,933 StorageService.java (line 515) Loading persisted ring state INFO [main] 2014-01-29 09:07:09,974 MessagingService.java (line 458) Starting Messaging Service on port 7000 ERROR [main] 2014-01-29 09:07:40,993 CassandraDaemon.java (line 473) Exception encountered during startup java.lang.RuntimeException: Unable to gossip with any seeds at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1160) at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:426) at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:618) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:586) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:485) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:341) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:456) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:499) ERROR [StorageServiceShutdownHook] 2014-01-29 09:07:41,150 CassandraDaemon.java (line 188) Exception in thread Thread[StorageServiceShutdownHook,5,main] java.lang.NullPointerException at org.apache.cassandra.service.StorageService.stopNativeTransport(StorageService.java:349) at org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:364) at org.apache.cassandra.service.StorageService.access$000(StorageService.java:97) at org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:551) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at java.lang.Thread.run(Thread.java:744) -- This message was sent by Atlassian JIRA (v6.1.5#6160)