[
https://issues.apache.org/jira/browse/ZOOKEEPER-3496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923151#comment-16923151
]
Mohammad Arshad commented on ZOOKEEPER-3496:
--------------------------------------------
PR Created
> Transaction larger than jute.maxbuffer makes ZooKeeper unavailable
> ------------------------------------------------------------------
>
> Key: ZOOKEEPER-3496
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3496
> Project: ZooKeeper
> Issue Type: Bug
> Affects Versions: 3.5.5, 3.4.14
> Reporter: Mohammad Arshad
> Assignee: Mohammad Arshad
> Priority: Critical
> Labels: pull-request-available
> Fix For: 3.6.0, 3.4.15, 3.5.6
>
> Attachments: ZOOKEEPER-3496-001.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *Problem:*
> ZooKeeper server fails to start, logs following error
> {code:java}
> Exception in thread "main" java.io.IOException: Unreasonable length = 1001025
> at
> org.apache.jute.BinaryInputArchive.checkLength(BinaryInputArchive.java:127)
> at
> org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:92)
> {code}
> This indicates that one of the transactions size is more than the configured
> jute.maxbuffer values. But how transaction more than jute.maxbuffer size is
> allowed to write?
> *Analysis:*
> At ZooKeeper server jute.maxbuffer specifies the maximum size of a
> transaction. By default it is 1 MB at the server
> jute.maxbuffer is used for following:
> # Size sanity check of incoming request. Incoming requests size must not be
> more than jute.maxbuffer
> # Size sanity check of the transaction while reading from transaction or
> snapshot file. Transaction size must not be more than jute.maxbuffer+1024
> # Size sanity check of transaction while reading data from the leader.
> Transaction size must not be more than jute.maxbuffer+1024
> Request size sanity check is done in the beginning of a request processing
> but later request processing adds additional information into request then
> writes to transaction file. This additional information size is not
> considered in sanity check. This is how transaction larger than
> jute.maxbuffer are accepted into ZooKeeper.
> If this additional information size is less than 1024 Bytes then it is OK as
> ZooKeeper already takes care of it.
> But if this additional information size is more than 1024 bytes it allows the
> request, But while reading from transaction/snapshot file and while reading
> from leader it fails and make the ZooKeeper service unavailable
> +Example:+
> Suppose incoming request size is 1000000 Bytes
> Configured jute.maxbuffer is 1000000
> After processing the request ZooKeeper server adds 1025 more bytes
> In this case, request will be processed successfully, and 1000000+1025 bytes
> will be written to transaction file
> But while reading from the transaction log 1000000+1025 bytes cannot be read
> as max allowed length is 1000000(effectively 1000000+1024).
> *Solutions:*
> If incoming request size sanity check is done after populating all additional
> information then this problem is solved. But doing sanity check in the later
> stage of request processing will defeat the purpose of sanity check itself.
> So this we can not do
> Currently additional information size is constant 1024 Bytes [Code
> Reference|https://github.com/apache/zookeeper/blob/branch-3.5/zookeeper-jute/src/main/java/org/apache/jute/BinaryInputArchive.java#L126].
> We should increase this value and make it more reasonable. I propose to make
> this additional information size to same as the jute.maxbuffer. Also make
> additional information size configurable.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
