I had similar OutOfMemory issues, see "commit logs are not deleted" thread, but it was fixed/commited to trunk. What I don't get is how your commit log segment grows beyond threshold.
>Is it really nescessary to have as much RAM as you have data?!?!? No, I was able to insert 50Gb worth of data with 3Gb cap On Wed, Oct 7, 2009 at 2:16 PM, Dan Larsen <d...@techba.se> wrote: > Sorry about that... running 0.4 taken from trunk.I didn't > change CommitLogRotationThresholdInMB - it's still 128 > > Thanks! > > On 07/10/2009, at 20.12, Igor Katkov wrote: > > What version are you running? Take 0.4 from trunk. > Default config has > <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB> > Did you change it allow your commit log chunks be 1Gb? > > On Wed, Oct 7, 2009 at 1:38 PM, Dan Larsen <d...@techba.se> wrote: > >> Hi guys, >> >> I have a small setup with 3 macs running os x snow leopard and the native >> java 1.6.0.something.something ;-) >> I am doing a lot of inserts, without any problems and I can read the data >> again. >> >> But.... >> At some point, around 1G Commitlog size, cassandra starts a rise in ram >> usage, untill it just stops responding - if I restart the server, with or >> without -b, it will just usae 100% cpu for 10-15 mins and then stop with >> this message: >> Another problem, is that it seems to kill other servers too, when I try to >> restart :-( >> >> java.lang.OutOfMemoryError: Java heap space >> Dumping heap to java_pid53353.hprof ... >> Heap dump file created [1164687070 bytes in 89.912 secs] >> >> ERROR - Fatal exception in thread Thread[RMI TCP Connection(idle),5,RMI >> Runtime] >> java.lang.OutOfMemoryError: Java heap space >> at java.io.BufferedInputStream.<init>(BufferedInputStream.java:178) >> at java.io.BufferedInputStream.<init>(BufferedInputStream.java:158) >> at >> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:679) >> at >> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >> at java.lang.Thread.run(Thread.java:637) >> ERROR - Fatal exception in thread Thread[main,5,main] >> java.lang.OutOfMemoryError: Java heap space >> ERROR - Fatal exception in thread Thread[RMI TCP Connection(idle),5,RMI >> Runtime] >> java.lang.OutOfMemoryError: Java heap space >> at java.lang.Class.getDeclaredMethods0(Native Method) >> at java.lang.Class.privateGetDeclaredMethods(Class.java:2427) >> at java.lang.Class.getDeclaredMethod(Class.java:1935) >> at >> java.io.ObjectStreamClass.getPrivateMethod(ObjectStreamClass.java:1382) >> at java.io.ObjectStreamClass.access$1700(ObjectStreamClass.java:52) >> at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:438) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:413) >> at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:310) >> at >> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1106) >> at >> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509) >> at >> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474) >> at >> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392) >> at >> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150) >> at >> java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326) >> at java.util.ArrayList.writeObject(ArrayList.java:570) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at >> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:945) >> at >> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1461) >> at >> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392) >> at >> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150) >> at >> java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326) >> at sun.rmi.server.UnicastRef.marshalValue(UnicastRef.java:274) >> at >> sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:315) >> at sun.rmi.transport.Transport$1.run(Transport.java:159) >> at java.security.AccessController.doPrivileged(Native Method) >> at sun.rmi.transport.Transport.serviceCall(Transport.java:155) >> at >> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) >> at >> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) >> ERROR - Fatal exception in thread Thread[RMI TCP Connection(idle),5,RMI >> Runtime] >> >> [ABOVE REPEATS A COUPLE OF TIMES] >> >> If I try to connect with nodeprobe, after it halts, I will just get this: >> >> Error connecting to remote JMX agent! >> java.io.IOException: Failed to retrieve RMIServer stub: >> javax.naming.ServiceUnavailableException [Root exception is >> java.rmi.ConnectException: Connection refused to host: 192.168.100.203; >> nested exception is: >> java.net.ConnectException: Connection refused] >> at >> javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:338) >> at >> javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:248) >> at org.apache.cassandra.tools.NodeProbe.connect(NodeProbe.java:152) >> at org.apache.cassandra.tools.NodeProbe.<init>(NodeProbe.java:114) >> at org.apache.cassandra.tools.NodeProbe.main(NodeProbe.java:545) >> Caused by: javax.naming.ServiceUnavailableException [Root exception is >> java.rmi.ConnectException: Connection refused to host: 192.168.100.203; >> nested exception is: >> java.net.ConnectException: Connection refused] >> at >> com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:101) >> at >> com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:185) >> at javax.naming.InitialContext.lookup(InitialContext.java:392) >> at >> javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1886) >> at >> javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1856) >> at >> javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:257) >> ... 4 more >> Caused by: java.rmi.ConnectException: Connection refused to host: >> 192.168.100.203; nested exception is: >> java.net.ConnectException: Connection refused >> at >> sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:601) >> at >> sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:198) >> at >> sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184) >> at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322) >> at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source) >> at >> com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:97) >> ... 9 more >> Caused by: java.net.ConnectException: Connection refused >> at java.net.PlainSocketImpl.socketConnect(Native Method) >> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) >> at >> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) >> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) >> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:432) >> at java.net.Socket.connect(Socket.java:525) >> at java.net.Socket.connect(Socket.java:475) >> at java.net.Socket.<init>(Socket.java:372) >> at java.net.Socket.<init>(Socket.java:186) >> at >> sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:22) >> at >> sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:128) >> at >> sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:595) >> ... 14 more >> >> >> I know i responds as supposed, before the crash. >> >> Hope somebody can help! :-) >> >> Best regards >> Dan Larsen >> > > >
