Shawn - Thanks for jumping in on the ZK side as well.
I will take a hard look at my config files but I checked and I do not have any one file over 1MB. The combined files (10 indexes) is 2.2MB. I am using micros for the nodes which are very limited in memory. I'm not currently using a java.env file so I guess I'm using the default values for the JVM which is typically xmx512M if I remember correctly. Could it be just a memory issue? JiM K. On Thu, Mar 1, 2018 at 11:13 PM, Shawn Heisey <[email protected]> wrote: > On 3/1/2018 7:59 PM, Jim Keeney wrote: > >> Read about the maxbuffer and am pretty sure that this might explain the >> behavior we are seeing since it occurs when there has been a significant >> reboot of all the servers. We have over 2 mb of config files for all of >> our >> indexes and if all the Solr nodes are sync ing their configs at once it >> seems like that might overflow the buffer. >> > > You probably recognize me from the Solr side. Hello again. I do know > enough to handle this part, so I'm answering. I didn't consider the > maxbuffer setting, because I didn't see anything about large packets in the > logs you shared on the Solr mailing list, and it's very rare for Solr users > to need to increase it. > > You only need to worry about the maxbuffer if any single part of the > config in ZK (what is called a "znode") is over 1MB. Each file in the > configs that you upload will go into its own znode. So if none of the > individual files in your configs is really large, you probably won't need > to set jute.maxbuffer. > > As for the other things that Solr puts in ZK: Unless you have a REALLY > huge cluster (tons of collections, shards, replicas, servers, etc) then > that information should be quite small. > > Newbie question, where would i set the -Djute.maxbuffer ? Should I update >> the zkServer.sh file so this is applied every time zookeeper is started or >> restarted. >> > > If jute.maxbuffer is needed, it must be set on the startup options for > every ZK server and every client that will access large znodes. Which > means all your ZK servers, all your Solr servers, and any invocations of > things like the scripts Solr includes for uploading configs. > > Thanks, > Shawn > > -- Jim Keeney President, FitterWeb E: [email protected] M: 703-568-5887 <(703)%20568-5887> *FitterWeb Consulting* *Are you lean and agile enough? *
