Take a look in the code repo. should be a simple pull. S On Fri, Jul 14, 2023 at 3:23 PM Ruel, Ryan <[email protected]> wrote:
> We have an application where the size of individual ZNodes is small (a few > KB typically), however our data is distributed in the tree such that we can > have many sub nodes (10s of thousands, in some cases). > > When running the ZK CLI tool to view our data, I was surprised to see that > we started to get IOExceptions for exceeding the 1MB jute.maxbuffer. > > We've gotten around this by increasing the max buffer size to 10MB, but it > wasn't clear to me whether the ZNode allowed data size is impacted by the > number of sub nodes, or if this buffer size is just reused in various > places in the client code. > > ZK seems to operate just fine with these large numbers of sub nodes, it's > just the client tool that was complaining when trying to list sub nodes. > > /Ryan > > On 7/14/23, 3:01 PM, "Steph van Schalkwyk" <[email protected] > <mailto:[email protected]>> wrote: > > > To your last point - ZK was designed to distribute small packets, hence the > 1M buffer. > I've had a client who had a Solr connector that kept on creating new fields > from different sources, and the Solr schema quickly grew to 4M. That's > about the biggest I've seen ZK operate reliably. > > > On Fri, Jul 14, 2023 at 1:09 PM Aishwarya Soni <[email protected] > <mailto:[email protected]>> > wrote: > > > > Hi, > > > > I want to find what is the current size/memory of a znode, i.e. how much > > its utilizing including all its child znodes. I know > > *zk_approximate_data_size* is the approximate memory consumption for ALL > > znodes stored in the ZooKeeper ensemble. But I need to find the active > size > > of a specific znode out of multiple znodes. > > > > How can we get it? > > > > Also, what is the safe max value we can assign to jute.maxbuffer? I am > > seeing packet length of 1 GB coming from a couple of clients and it is > > getting errored out with IOException due to jute.maxbuffer set to the > > default value of 1MB. > > > > Regards, > > Aishwarya > > > > > >
