On 3/1/2018 7:59 PM, Jim Keeney wrote:
Read about the maxbuffer and am pretty sure that this might explain the behavior we are seeing since it occurs when there has been a significant reboot of all the servers. We have over 2 mb of config files for all of our indexes and if all the Solr nodes are sync ing their configs at once it seems like that might overflow the buffer.
You probably recognize me from the Solr side. Hello again. I do know enough to handle this part, so I'm answering. I didn't consider the maxbuffer setting, because I didn't see anything about large packets in the logs you shared on the Solr mailing list, and it's very rare for Solr users to need to increase it.
You only need to worry about the maxbuffer if any single part of the config in ZK (what is called a "znode") is over 1MB. Each file in the configs that you upload will go into its own znode. So if none of the individual files in your configs is really large, you probably won't need to set jute.maxbuffer.
As for the other things that Solr puts in ZK: Unless you have a REALLY huge cluster (tons of collections, shards, replicas, servers, etc) then that information should be quite small.
Newbie question, where would i set the -Djute.maxbuffer ? Should I update the zkServer.sh file so this is applied every time zookeeper is started or restarted.
If jute.maxbuffer is needed, it must be set on the startup options for every ZK server and every client that will access large znodes. Which means all your ZK servers, all your Solr servers, and any invocations of things like the scripts Solr includes for uploading configs.
Thanks, Shawn
