I haven't had a chance to play with that yet. I'm just trying to get a bunch of data loaded so I can run some tests. Once the tests are done I will look at starting and stopping servers again.
The tokens thing helped out quite a lot. brian -----Original Message----- From: Jonathan Ellis [mailto:jbel...@gmail.com] Sent: Thursday, August 20, 2009 10:50 AM To: cassandra-user@incubator.apache.org Subject: Re: Server cannot startup after shutdown I saw that you're testing with different tokens now -- how did the replay OOM work out? On Wed, Aug 19, 2009 at 10:54 PM, Brian Frank Cooper<coop...@yahoo-inc.com> wrote: > Thanks for the reply. I'll try playing with the memory settings. > > brian > ________________________________________ > From: Jonathan Ellis [jbel...@gmail.com] > Sent: Wednesday, August 19, 2009 7:46 PM > To: cassandra-user@incubator.apache.org > Subject: Re: Server cannot startup after shutdown > > The malformed input bug was fixed after beta1 and should be in a > nightly build by now. (I introduced a regression where it couldn't > handle the last entry in the commitlog being incomplete. So upgrading > should be able to restart on the existing commitlogs.) > > The OOM puzzles me a little; I'm not sure how it could be unable to > replay a mutation that it was able to write to the commitlog in the > first place. You could try setting the memtable object and memory > thresholds lower temporarily and see if that leaves enough extra free > to do the replay. > > -Jonathan > > On Wed, Aug 19, 2009 at 7:12 PM, Brian Frank > Cooper<coop...@yahoo-inc.com> wrote: >> Hi folks, >> >> I'm using 0.4 beta1 and had six servers loaded with 20 GB of data per >> server. (In this test, 10 KB per record, and 2 GB heap space allocated to >> the JVM.) I stopped the servers (using what I think is the recommended >> method, the kill command). Upon trying to restart, some servers threw a >> UTFDataFormatException, while others threw an OutOfMemoryError exception. >> None of them started. >> >> Is this a known issue? >> >> ERROR - Fatal exception in thread Thread[main,5,main] >> java.lang.OutOfMemoryError: Java heap space >> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:274) >> at >> org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63) >> at >> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:96) >> at >> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:171) >> >> ERROR - Exception encountered during startup. >> java.io.UTFDataFormatException: malformed input around byte 5497 >> at java.io.DataInputStream.readUTF(DataInputStream.java:639) >> at java.io.DataInputStream.readUTF(DataInputStream.java:547) >> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:218) >> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:285) >> at >> org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63) >> at >> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:96) >> at >> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:171) >> Exception encountered during startup. >> java.io.UTFDataFormatException: malformed input around byte 5497 >> at java.io.DataInputStream.readUTF(DataInputStream.java:639) >> at java.io.DataInputStream.readUTF(DataInputStream.java:547) >> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:218) >> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:285) >> at >> org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63) >> at >> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:96) >> at >> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:171) >> >> Thanks for the help! >> >> Brian >
