I am using Random Partitioner. The other 2 nodes are working fine. There are no Errors in the log files for the 2 good nodes.
There were no log messages within 30 minutes before the exception occurs. Here is the last log statement before the exception occurred. INFO [COMPACTION-POOL:1] 2010-07-15 03:07:58,217 CompactionManager.java (line 246) Compacting [org. apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Musubi/Conn-22-Data.db'),org.apache. cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Musubi/Conn-23-Data.db'),org.apache.cassand ra.io.SSTableReader(path='/var/lib/cassandra/data/Musubi/Conn-24-Data.db'),org.apache.cassandra.io.S STableReader(path='/var/lib/cassandra/data/Musubi/Conn-25-Data.db')] INFO [COMPACTION-POOL:1] 2010-07-15 03:07:59,884 CompactionManager.java (line 320) Compacted to /var/lib/cassandra/data/Musubi/Conn-26-Data.db. 21687831/21675414 bytes for 72901 keys. Time: 1666ms. Any help is MUCH appreciated! claire On Jul 15, 2010, at 10:43 AM, Claire Chang wrote: > I am trying to set up a 3 node cluster. RF=3 and CL=1 for most of the > request. The initial seeding took about 1 hour to complete which loaded each > node with 2G of data. After the seeding completed, one node started having > this exception and hung. Read/Write with CL=ALL timed out but CL=QUORUM was > fine. I restarted the node and the problem went away. But after 3 hours, same > scenario reappeared. Any idea? Any workaround? Thanks. -Claire > > > ERROR 22:59:00,329 Error in ThreadPoolExecutor > java.lang.NullPointerException > at org.apache.cassandra.db.Table.apply(Table.java:407) > at > org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:68) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:40) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:636) > ... > > > ERROR 03:30:59,862 Uncaught exception in thread > Thread[ROW-MUTATION-STAGE:15,5,main] > java.lang.NullPointerException > at org.apache.cassandra.db.Table.apply(Table.java:407) > at > org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:68) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:40) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:636) > INFO 06:11:22,965 InetAddress /192.168.11.29 is now dead. > INFO 06:11:22,966 InetAddress /192.168.11.27 is now dead. >
