Hello. I am testing out Cassandra and ran out of file descriptors while attempting to scan through the key space. I have ~180 million unique keys, 10 nodes, and a replication factor of 3.
Cassandra reported the following: WARN [GMFD:1] 2009-11-12 16:07:24,961 MessagingService.java (line 393) Exception was generated at : 11/12/2009 16:07:24 on thread GMFD:1 Too many open files java.net.SocketException: Too many open files at sun.nio.ch.Net.socket0(Native Method) at sun.nio.ch.Net.socket(Unknown Source) at sun.nio.ch.DatagramChannelImpl.<init>(Unknown Source) at sun.nio.ch.SelectorProviderImpl.openDatagramChannel(Unknown Source) at java.nio.channels.DatagramChannel.open(Unknown Source) at org.apache.cassandra.net.UdpConnection.init(UdpConnection.java:49) at org.apache.cassandra.net.MessagingService.sendUdpOneWay(MessagingService.java:388) at org.apache.cassandra.gms.GossipDigestSynVerbHandler.doVerb(Gossiper.java:889) at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:38) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Lsof reports that the java process has 65486 files open (I have ulimit -n 65535 set in cassandra.in.sh). Many of the lsof entries include a trailing '(deleted)' comment after the file path. This appears to be similar to CASSANDRA-283. Anyone have a work around for this? Would a forced GC take care of the ones marked deleted? Here is my sample code to count the number of keys: public class CClient { public static void main(String[] args) throws TException, InvalidRequestException, UnavailableException, UnsupportedEncodingException, NotFoundException { TTransport tr = new TSocket("localhost", 9160); TProtocol proto = new TBinaryProtocol(tr); Cassandra.Client client = new Cassandra.Client(proto); tr.open(); int count = 0; int block = 1000; String key = " "; while (true) { List<String> list = client.get_key_range("Keyspace1", "Standard1", key, "~", block, ConsistencyLevel.ONE); int size = list.size(); if (size == 0) break; count += size; key = list.get(size - 1) + '~'; System.out.println("Count: " + Integer.toString(count)); } tr.close(); } }