[jira] [Commented] (ZOOKEEPER-2973) "Unreasonable length" exception
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349931#comment-16349931 ] wuyiyun commented on ZOOKEEPER-2973: Yes, that is one reason and easy to resolve. But i know that even if i send length of data just no more than one hundred byte, once enabled ssl capability, several hours later, just zookeeper leader node will get this exception. I met OutOfMemory error cause zookeeper shutdown. and restart not ok. 1. Maybe the client write too much too the ZooKeeper, which bigger than the jute.maxbuffer > "Unreasonable length" exception > > > Key: ZOOKEEPER-2973 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2973 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.6 >Reporter: wanggang_123 >Priority: Blocker > > I am running a three node ZooKeeper cluster. At 2018-01-28 17:56:30,leader > node has error log: > 2018-01-28 17:56:30 > [UTC:20180128T175630+0800]|ERROR||LearnerHandler-/118.123.180.23:44836hread|Coordination > > Unexpected exception causing shutdown while sock still open > (LearnerHandler.java:633) > java.io.IOException: Unreasonable length = 1885430131 > at org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:95) > at > org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:85) > at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:103) > at > org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:546) > 2018-01-28 17:56:30 [UTC:20180128T175630+0800]|WARN > ||LearnerHandler-/118.123.180.23:44836hread|Coordination > *** GOODBYE > /118.123.180.23:44836 (LearnerHandler.java:646) > 2018-01-28 17:56:30 [UTC:20180128T175630+0800]|INFO ||ProcessThread(sid:2 > cport:-1):hread|Coordination > Got user-level KeeperException when processing > sessionid:0x16138593ad43cf9 type:delete cxid:0x5 zxid:0xc104b59e9 txntype:-1 > reqpath:n/a Error > Path:/VSP/Leader/syncScore-0/_c_9101a3d6-f431-4792-b71d-a493e938895d-latch-093037 > Error:KeeperErrorCode = NoNode for > /VSP/Leader/syncScore-0/_c_9101a3d6-f431-4792-b71d-a493e938895d-latch-093037 > (PrepRequestProcessor.java:645) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-2973) "Unreasonable length" exception
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349900#comment-16349900 ] caixiaofeng commented on ZOOKEEPER-2973: I met OutOfMemory error cause zookeeper shutdown. and restart not ok. 1. Maybe the client write too much too the ZooKeeper, which bigger than the jute.maxbuffer https://issues.apache.org/jira/browse/ZOOKEEPER-2101 Transaction larger than max buffer of jute makes zookeeper unavailable > "Unreasonable length" exception > > > Key: ZOOKEEPER-2973 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2973 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.6 >Reporter: wanggang_123 >Priority: Blocker > > I am running a three node ZooKeeper cluster. At 2018-01-28 17:56:30,leader > node has error log: > 2018-01-28 17:56:30 > [UTC:20180128T175630+0800]|ERROR||LearnerHandler-/118.123.180.23:44836hread|Coordination > > Unexpected exception causing shutdown while sock still open > (LearnerHandler.java:633) > java.io.IOException: Unreasonable length = 1885430131 > at org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:95) > at > org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:85) > at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:103) > at > org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:546) > 2018-01-28 17:56:30 [UTC:20180128T175630+0800]|WARN > ||LearnerHandler-/118.123.180.23:44836hread|Coordination > *** GOODBYE > /118.123.180.23:44836 (LearnerHandler.java:646) > 2018-01-28 17:56:30 [UTC:20180128T175630+0800]|INFO ||ProcessThread(sid:2 > cport:-1):hread|Coordination > Got user-level KeeperException when processing > sessionid:0x16138593ad43cf9 type:delete cxid:0x5 zxid:0xc104b59e9 txntype:-1 > reqpath:n/a Error > Path:/VSP/Leader/syncScore-0/_c_9101a3d6-f431-4792-b71d-a493e938895d-latch-093037 > Error:KeeperErrorCode = NoNode for > /VSP/Leader/syncScore-0/_c_9101a3d6-f431-4792-b71d-a493e938895d-latch-093037 > (PrepRequestProcessor.java:645) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-2973) "Unreasonable length" exception
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347842#comment-16347842 ] wuyiyun commented on ZOOKEEPER-2973: Me too!. I met OutOfMemory error cause zookeeper shutdown. but you can add crontab task to restart zookeeper to resolve this issue. Of course, you should insure that zookeeper client jar has same version with zookeeper server firstly. > "Unreasonable length" exception > > > Key: ZOOKEEPER-2973 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2973 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.6 >Reporter: wanggang_123 >Priority: Blocker > > I am running a three node ZooKeeper cluster. At 2018-01-28 17:56:30,leader > node has error log: > 2018-01-28 17:56:30 > [UTC:20180128T175630+0800]|ERROR||LearnerHandler-/118.123.180.23:44836hread|Coordination > > Unexpected exception causing shutdown while sock still open > (LearnerHandler.java:633) > java.io.IOException: Unreasonable length = 1885430131 > at org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:95) > at > org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:85) > at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:103) > at > org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:546) > 2018-01-28 17:56:30 [UTC:20180128T175630+0800]|WARN > ||LearnerHandler-/118.123.180.23:44836hread|Coordination > *** GOODBYE > /118.123.180.23:44836 (LearnerHandler.java:646) > 2018-01-28 17:56:30 [UTC:20180128T175630+0800]|INFO ||ProcessThread(sid:2 > cport:-1):hread|Coordination > Got user-level KeeperException when processing > sessionid:0x16138593ad43cf9 type:delete cxid:0x5 zxid:0xc104b59e9 txntype:-1 > reqpath:n/a Error > Path:/VSP/Leader/syncScore-0/_c_9101a3d6-f431-4792-b71d-a493e938895d-latch-093037 > Error:KeeperErrorCode = NoNode for > /VSP/Leader/syncScore-0/_c_9101a3d6-f431-4792-b71d-a493e938895d-latch-093037 > (PrepRequestProcessor.java:645) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-2973) "Unreasonable length" exception
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346549#comment-16346549 ] wanggang_123 commented on ZOOKEEPER-2973: - 你好，我这边的zookeeper，是偶现shutdown，你遇到的也是吗？ 这个问题的触发条件是什么啊？如何能够复现？ > "Unreasonable length" exception > > > Key: ZOOKEEPER-2973 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2973 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.6 >Reporter: wanggang_123 >Priority: Blocker > > I am running a three node ZooKeeper cluster. At 2018-01-28 17:56:30,leader > node has error log: > 2018-01-28 17:56:30 > [UTC:20180128T175630+0800]|ERROR||LearnerHandler-/118.123.180.23:44836hread|Coordination > > Unexpected exception causing shutdown while sock still open > (LearnerHandler.java:633) > java.io.IOException: Unreasonable length = 1885430131 > at org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:95) > at > org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:85) > at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:103) > at > org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:546) > 2018-01-28 17:56:30 [UTC:20180128T175630+0800]|WARN > ||LearnerHandler-/118.123.180.23:44836hread|Coordination > *** GOODBYE > /118.123.180.23:44836 (LearnerHandler.java:646) > 2018-01-28 17:56:30 [UTC:20180128T175630+0800]|INFO ||ProcessThread(sid:2 > cport:-1):hread|Coordination > Got user-level KeeperException when processing > sessionid:0x16138593ad43cf9 type:delete cxid:0x5 zxid:0xc104b59e9 txntype:-1 > reqpath:n/a Error > Path:/VSP/Leader/syncScore-0/_c_9101a3d6-f431-4792-b71d-a493e938895d-latch-093037 > Error:KeeperErrorCode = NoNode for > /VSP/Leader/syncScore-0/_c_9101a3d6-f431-4792-b71d-a493e938895d-latch-093037 > (PrepRequestProcessor.java:645) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-2973) "Unreasonable length" exception
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346522#comment-16346522
]
wuyiyun commented on ZOOKEEPER-2973:
I met this problem too, while i use open ssl capability on zookeeper server and
zookeeper client. while i close ssl capability and will not cause this problem.
you can modify source of BinaryInputArchive like below and find what
public String readString(String tag) throws IOException {
int len = in.readInt();
if (len == -1) return null;
checkLength(len);
byte b[] = new byte[len];
in.readFully(b);
String tobeReturnString = new String(b, "UTF8");
LOG.info("Just readString String's lengh="+len+" and this
string is:"+tobeReturnString);
return tobeReturnString;
}
static public final int maxBuffer = Integer.getInteger("jute.maxbuffer",
0xf);
public byte[] readBuffer(String tag) throws IOException {
int len = readInt(tag);
if (len == -1) return null;
checkLength(len);
byte[] arr = new byte[len];
in.readFully(arr);
String tobeReturnString = new String(arr, "UTF8");
LOG.info("Just readBuffer String's lengh="+len+" and this
string is:"+tobeReturnString);
return arr;
}
> "Unreasonable length" exception
>
>
> Key: ZOOKEEPER-2973
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2973
> Project: ZooKeeper
> Issue Type: Bug
>Affects Versions: 3.4.6
>Reporter: wanggang_123
>Priority: Blocker
>
> I am running a three node ZooKeeper cluster. At 2018-01-28 17:56:30,leader
> node has error log:
> 2018-01-28 17:56:30
> [UTC:20180128T175630+0800]|ERROR||LearnerHandler-/118.123.180.23:44836hread|Coordination
> > Unexpected exception causing shutdown while sock still open
> (LearnerHandler.java:633)
> java.io.IOException: Unreasonable length = 1885430131
> at org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:95)
> at
> org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:85)
> at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:103)
> at
> org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:546)
> 2018-01-28 17:56:30 [UTC:20180128T175630+0800]|WARN
> ||LearnerHandler-/118.123.180.23:44836hread|Coordination > *** GOODBYE
> /118.123.180.23:44836 (LearnerHandler.java:646)
> 2018-01-28 17:56:30 [UTC:20180128T175630+0800]|INFO ||ProcessThread(sid:2
> cport:-1):hread|Coordination > Got user-level KeeperException when processing
> sessionid:0x16138593ad43cf9 type:delete cxid:0x5 zxid:0xc104b59e9 txntype:-1
> reqpath:n/a Error
> Path:/VSP/Leader/syncScore-0/_c_9101a3d6-f431-4792-b71d-a493e938895d-latch-093037
> Error:KeeperErrorCode = NoNode for
> /VSP/Leader/syncScore-0/_c_9101a3d6-f431-4792-b71d-a493e938895d-latch-093037
> (PrepRequestProcessor.java:645)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
