Zookeeper file size limits are probably the most common failure. I had to mess around a lot with our suggestion dictionary to get it to upload.
wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Feb 11, 2024, at 11:25 AM, Eric Pugh <ep...@opensourceconnections.com> > wrote: > > Ah.. yeah, I can’t speak to Solr 6.x! In 9x at least you could use the > configset API to deploy configs and avoid the direct ZK interaction. > > It would be interesting to explore if the process of deploying a configset is > risky, has a high chance of things failing, then how do we account for that > as part of the process? So you don’t have to do things like upload the > previous config ;-). > > And other common reasons to use ZK directly? > >> On Feb 11, 2024, at 12:14 PM, Walter Underwood <wun...@wunderwood.org> wrote: >> >> The was deploying configs with Jenkins on Solr 6.x. Maybe the APIs were >> there, but I didn't know about them. >> >> Rebuilding the suggester did need external help, since that needs to be done >> separately on each node. >> >> I think working directly with Zookeeper is less risky. If there is any issue >> with the upload, then don’t reload the collections. You can back out the >> changes by uploading the previous config to Zookeeper. >> >> wunder >> Walter Underwood >> wun...@wunderwood.org <mailto:wun...@wunderwood.org> >> http://observer.wunderwood.org/ (my blog) >> >>> On Feb 11, 2024, at 11:07 AM, Eric Pugh <ep...@opensourceconnections.com >>> <mailto:ep...@opensourceconnections.com>> wrote: >>> >>> Could you share more about “update Solr remotely” that you were doing? >>> Are we missing some APIs that would have made whatever you had to do >>> require ZK direct access? >>> >>> While it’s cool that we can impact Solr via hacking around in ZK, it also >>> seems like an approach fraught with risk! >>> >>>> On Feb 11, 2024, at 11:32 AM, Walter Underwood <wun...@wunderwood.org> >>>> wrote: >>>> >>>> I wanted something that didn’t require installing Solr locally in order to >>>> update Solr remotely, so I didn’t use the provided zk commands. I wrote >>>> some Python to dig the Zookeeper addresses out of clusterstatus (I think) >>>> then uploaded directly to Zookeeper with the Python kazoo package. >>>> >>>> The tool had a bunch of other things, like async reload checking for >>>> results, and rebuilding suggestion dictionaries on each node. >>>> >>>> wunder >>>> Walter Underwood >>>> wun...@wunderwood.org >>>> http://observer.wunderwood.org/ (my blog) >>>> >>>>> On Feb 11, 2024, at 9:04 AM, Gus Heck <gus.h...@gmail.com> wrote: >>>>> >>>>> I pretty much always use zk upconfig, which also works for overwriting >>>>> existing. I certainly tell my clients to use apis from the ref guide for >>>>> such operations, but zk upconfig certainly counts as one. Mostly I tell >>>>> them that they should only break out things like >>>>> https://github.com/rgs1/zk_shell as a last resort (which is what I think >>>>> of >>>>> as direct modification), and if they are unsure, call me *before* doing >>>>> anything in zk directly. >>>>> >>>>> By the way, I don't know if this has come up in a dev/build setting or >>>>> not, >>>>> but are you aware of https://plugins.gradle.org/search?term=solr ? It is >>>>> presently only really suitable for local dev, with a single config set, >>>>> but >>>>> could easily grow patches and suggestions welcome of course. >>>>> >>>>> On Sun, Feb 11, 2024, 9:10 AM Eric Pugh <ep...@opensourceconnections.com> >>>>> wrote: >>>>> >>>>>> Hi all.. I was playing around with a cluster and wanted to upload a >>>>>> configset into Solr…. >>>>>> >>>>>> I ran bin/solr and noticed a bin/solr config -h command, but it just lets >>>>>> me tweak a config. Then I ran bin/solr create -h and it appears to let >>>>>> me >>>>>> upload a configset, but I have to create the collection as well, and I’m >>>>>> not ready to do that. >>>>>> >>>>>> Then I poked around and discovered hidden under bin/solr zk a command >>>>>> upconfig…. So bin/solr zk upconfig will let me get my configset into >>>>>> Solr, >>>>>> but does require me to remember what my magic ZK string is ;-). >>>>>> >>>>>> I went and checked the ref guide, and yes, it states that there are two >>>>>> ways: >>>>>> >>>>>> A configset can be uploaded to ZooKeeper either via the Configsets API < >>>>>> https://solr.apache.org/guide/solr/latest/configuration-guide/configsets-api.html> >>>>>> or more directly via bin/solr zk upconfig < >>>>>> https://solr.apache.org/guide/solr/latest/deployment-guide/solr-control-script-reference.html#upload-a-configuration-set>. >>>>>> The Configsets API has some other operations as well, and likewise, so >>>>>> does >>>>>> the CLI. >>>>>> >>>>>> Are there use cases where interacting directly with ZooKeeper is >>>>>> preferred >>>>>> over making changes via the APIs? Of is the use of bin/solr zk upconfig >>>>>> more of a evolutionary byproduct of how we built SolrCloud? >>>>>> >>>>>> Eric >>>>>> >>>>>> _______________________ >>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | >>>>>> http://www.opensourceconnections.com < >>>>>> http://www.opensourceconnections.com/> | My Free/Busy < >>>>>> http://tinyurl.com/eric-cal> >>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < >>>>>> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> >>>>>> >>>>>> This e-mail and all contents, including attachments, is considered to be >>>>>> Company Confidential unless explicitly stated otherwise, regardless of >>>>>> whether attachments are marked as such. >>>>>> >>>>>> >>>> >>> >>> _______________________ >>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | >>> http://www.opensourceconnections.com >>> <http://www.opensourceconnections.com/> >>> <http://www.opensourceconnections.com/> | My Free/Busy >>> <http://tinyurl.com/eric-cal> >>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >>> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> >>> >>> This e-mail and all contents, including attachments, is considered to be >>> Company Confidential unless explicitly stated otherwise, regardless of >>> whether attachments are marked as such. > > _______________________ > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | > http://www.opensourceconnections.com <http://www.opensourceconnections.com/> > | My Free/Busy <http://tinyurl.com/eric-cal> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed > <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> > > This e-mail and all contents, including attachments, is considered to be > Company Confidential unless explicitly stated otherwise, regardless of > whether attachments are marked as such. >
