Hi Li, all,
> When load testing the write operation against Zookeeper 3.7.0, I observed a
> couple of times that the server crashed because the txn log was too large
> and it was not able to load it.
Difficult to say without more details, but I suspect ZOOKEEPER-4306 to
be the culprit:
https://issues.apache.org/jira/browse/ZOOKEEPER-4306
Would it be possible for you to share the transaction log ZooKeeper
fails to load?
HTH, -D
--8<---------------original message------------->8---
Li Wang <li4w...@gmail.com> writes:
> Hi,
>
>
>
> When load testing the write operation against Zookeeper 3.7.0, I observed a
> couple of times that the server crashed because the txn log was too large
> and it was not able to load it. However the data size of write is only 4
> bytes in the load test and the *jute.maxbuffer *was set to default (i.e.
> 1M). The error doesn't always happen.
>
>
> I wonder if anyone has also seen this error or has any idea on what may
> cause the issue?
>
>
> StackTrace
>
> =========
>
>
> 2021-07-01 16:02:00,837 [myid:3] - ERROR [main:QuorumPeerMain@114] -
> Unexpected exception, exiting abnormally
>
> java.lang.RuntimeException: Unable to run quorum server
>
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:1200)
>
> at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:1131)
>
> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:229)
>
> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:137)
>
> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:91)
>
> Caused by: java.io.IOException: Unreasonable length = 3175014
>
> at
> org.apache.jute.BinaryInputArchive.checkLength(BinaryInputArchive.java:166)
>
> at
> org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:127)
>
> at org.apache.zookeeper.server.persistence.Util.readTxnBytes(Util.java:159)
>
> at
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:749)
>
> at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.fastForwardFromEdits(FileTxnSnapLog.java:361)
>
> at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.lambda$restore$0(FileTxnSnapLog.java:267)
>
> at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:312)
>
> at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:287)
>
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:1145)
>
>
> Thanks,
>
>
> Li
