Assuming an API existed, would that then be a good way? Or are there times when an API wouldn’t do what direct ZK does?
> On Feb 12, 2024, at 9:13 AM, Jason Gerlowski <gerlowsk...@gmail.com> wrote: > >> What other use cases are there for us interacting directly with ZK? > > Reading cluster/replica/collection properties for sure. There are some > Solr APIs to modify these things, but afaik users that want to check the > value of a particular (e.g.) cluster-property don't have a way to do that > yet without reading it out of ZK directly. > > On Sun, Feb 11, 2024 at 1:58 PM Eric Pugh <ep...@opensourceconnections.com > <mailto:ep...@opensourceconnections.com>> > wrote: > >> I like your suggestion David on the CLI supporting configset directly. >> I’m sort of waiting till the V2 api for putting configsets is done. >> According to the Solr v2 API Proposed Changes spreadsheet, while there is a >> V2 api, it hasn’t been given the “restful/jax-rs/exposed in solrj” >> treatment yet…. >> >> I also agree that figuring out how to make ZK an internal detail is >> somewhat underlying my question…. >> >> What other use cases are there for us interacting directly with ZK? >> >>> On Feb 11, 2024, at 12:57 PM, David Smiley <dsmi...@apache.org> wrote: >>> >>> I suppose direct ZK has an advantage of not requiring that Solr is >>> running yet. But maybe that's moot. I do think it would make sense >>> to have CLI options for configset upload that use Solr's API and don't >>> refer to ZK. Broadly, this is the direction we're going in; ZK is >>> more of an internal detail, even though it's obviously a critical >>> thing. >>> >>> On Sun, Feb 11, 2024 at 12:26 PM Eric Pugh >>> <ep...@opensourceconnections.com <mailto:ep...@opensourceconnections.com> >>> <mailto:ep...@opensourceconnections.com>> >> wrote: >>>> >>>> Ah.. yeah, I can’t speak to Solr 6.x! In 9x at least you could use >> the configset API to deploy configs and avoid the direct ZK interaction. >>>> >>>> It would be interesting to explore if the process of deploying a >> configset is risky, has a high chance of things failing, then how do we >> account for that as part of the process? So you don’t have to do things >> like upload the previous config ;-). >>>> >>>> And other common reasons to use ZK directly? >>>> >>>>> On Feb 11, 2024, at 12:14 PM, Walter Underwood <wun...@wunderwood.org >>>>> <mailto:wun...@wunderwood.org>> >> wrote: >>>>> >>>>> The was deploying configs with Jenkins on Solr 6.x. Maybe the APIs >> were there, but I didn't know about them. >>>>> >>>>> Rebuilding the suggester did need external help, since that needs to >> be done separately on each node. >>>>> >>>>> I think working directly with Zookeeper is less risky. If there is any >> issue with the upload, then don’t reload the collections. You can back out >> the changes by uploading the previous config to Zookeeper. >>>>> >>>>> wunder >>>>> Walter Underwood >>>>> wun...@wunderwood.org <mailto:wun...@wunderwood.org> >>>>> <mailto:wun...@wunderwood.org> <mailto: >> wun...@wunderwood.org <mailto:wun...@wunderwood.org>> >>>>> http://observer.wunderwood.org/ (my blog) >>>>> >>>>>> On Feb 11, 2024, at 11:07 AM, Eric Pugh < >> ep...@opensourceconnections.com <mailto:ep...@opensourceconnections.com> >> <mailto:ep...@opensourceconnections.com> >> <mailto:ep...@opensourceconnections.com>> wrote: >>>>>> >>>>>> Could you share more about “update Solr remotely” that you were >> doing? Are we missing some APIs that would have made whatever you had to >> do require ZK direct access? >>>>>> >>>>>> While it’s cool that we can impact Solr via hacking around in ZK, it >> also seems like an approach fraught with risk! >>>>>> >>>>>>> On Feb 11, 2024, at 11:32 AM, Walter Underwood < >> wun...@wunderwood.org <mailto:wun...@wunderwood.org> >> <mailto:wun...@wunderwood.org>> wrote: >>>>>>> >>>>>>> I wanted something that didn’t require installing Solr locally in >> order to update Solr remotely, so I didn’t use the provided zk commands. I >> wrote some Python to dig the Zookeeper addresses out of clusterstatus (I >> think) then uploaded directly to Zookeeper with the Python kazoo package. >>>>>>> >>>>>>> The tool had a bunch of other things, like async reload checking for >> results, and rebuilding suggestion dictionaries on each node. >>>>>>> >>>>>>> wunder >>>>>>> Walter Underwood >>>>>>> wun...@wunderwood.org <mailto:wun...@wunderwood.org> >>>>>>> <mailto:wun...@wunderwood.org> >>>>>>> http://observer.wunderwood.org/ (my blog) >>>>>>> >>>>>>>> On Feb 11, 2024, at 9:04 AM, Gus Heck <gus.h...@gmail.com >>>>>>>> <mailto:gus.h...@gmail.com> <mailto: >> gus.h...@gmail.com <mailto:gus.h...@gmail.com>>> wrote: >>>>>>>> >>>>>>>> I pretty much always use zk upconfig, which also works for >> overwriting >>>>>>>> existing. I certainly tell my clients to use apis from the ref >> guide for >>>>>>>> such operations, but zk upconfig certainly counts as one. Mostly I >> tell >>>>>>>> them that they should only break out things like >>>>>>>> https://github.com/rgs1/zk_shell as a last resort (which is what I >> think of >>>>>>>> as direct modification), and if they are unsure, call me *before* >> doing >>>>>>>> anything in zk directly. >>>>>>>> >>>>>>>> By the way, I don't know if this has come up in a dev/build setting >> or not, >>>>>>>> but are you aware of https://plugins.gradle.org/search?term=solr ? >> It is >>>>>>>> presently only really suitable for local dev, with a single config >> set, but >>>>>>>> could easily grow patches and suggestions welcome of course. >>>>>>>> >>>>>>>> On Sun, Feb 11, 2024, 9:10 AM Eric Pugh < >> ep...@opensourceconnections.com <mailto:ep...@opensourceconnections.com> >> <mailto:ep...@opensourceconnections.com>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi all.. I was playing around with a cluster and wanted to >> upload a >>>>>>>>> configset into Solr…. >>>>>>>>> >>>>>>>>> I ran bin/solr and noticed a bin/solr config -h command, but it >> just lets >>>>>>>>> me tweak a config. Then I ran bin/solr create -h and it appears >> to let me >>>>>>>>> upload a configset, but I have to create the collection as well, >> and I’m >>>>>>>>> not ready to do that. >>>>>>>>> >>>>>>>>> Then I poked around and discovered hidden under bin/solr zk a >> command >>>>>>>>> upconfig…. So bin/solr zk upconfig will let me get my configset >> into Solr, >>>>>>>>> but does require me to remember what my magic ZK string is ;-). >>>>>>>>> >>>>>>>>> I went and checked the ref guide, and yes, it states that there >> are two >>>>>>>>> ways: >>>>>>>>> >>>>>>>>> A configset can be uploaded to ZooKeeper either via the Configsets >> API < >>>>>>>>> >> https://solr.apache.org/guide/solr/latest/configuration-guide/configsets-api.html >>> >>>>>>>>> or more directly via bin/solr zk upconfig < >>>>>>>>> >> https://solr.apache.org/guide/solr/latest/deployment-guide/solr-control-script-reference.html#upload-a-configuration-set >>> . >>>>>>>>> The Configsets API has some other operations as well, and >> likewise, so does >>>>>>>>> the CLI. >>>>>>>>> >>>>>>>>> Are there use cases where interacting directly with ZooKeeper is >> preferred >>>>>>>>> over making changes via the APIs? Of is the use of bin/solr zk >> upconfig >>>>>>>>> more of a evolutionary byproduct of how we built SolrCloud? >>>>>>>>> >>>>>>>>> Eric >>>>>>>>> >>>>>>>>> _______________________ >>>>>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | >> 434.466.1467 | >>>>>>>>> http://www.opensourceconnections.com >>>>>>>>> <http://www.opensourceconnections.com/> < >> http://www.opensourceconnections.com/> < >>>>>>>>> http://www.opensourceconnections.com/> | My Free/Busy < >>>>>>>>> http://tinyurl.com/eric-cal> >>>>>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < >>>>>>>>> >> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw >>> >>>>>>>>> >>>>>>>>> This e-mail and all contents, including attachments, is considered >> to be >>>>>>>>> Company Confidential unless explicitly stated otherwise, >> regardless of >>>>>>>>> whether attachments are marked as such. >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>>>> _______________________ >>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | >> 434.466.1467 | http://www.opensourceconnections.com >> <http://www.opensourceconnections.com/> < >> http://www.opensourceconnections.com/> < >> http://www.opensourceconnections.com/> < >> http://www.opensourceconnections.com/> | My Free/Busy < >> http://tinyurl.com/eric-cal> >>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < >> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw >>> >>>>>> This e-mail and all contents, including attachments, is considered to >> be Company Confidential unless explicitly stated otherwise, regardless of >> whether attachments are marked as such. >>>> >>>> _______________________ >>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 >> | http://www.opensourceconnections.com >> <http://www.opensourceconnections.com/> < >> http://www.opensourceconnections.com/> < >> http://www.opensourceconnections.com/> | My Free/Busy < >> http://tinyurl.com/eric-cal> >>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < >> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw >>> >>>> This e-mail and all contents, including attachments, is considered to >> be Company Confidential unless explicitly stated otherwise, regardless of >> whether attachments are marked as such. >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org >>> <mailto:dev-unsubscr...@solr.apache.org> <mailto: >> dev-unsubscr...@solr.apache.org <mailto:dev-unsubscr...@solr.apache.org>> >>> For additional commands, e-mail: dev-h...@solr.apache.org >>> <mailto:dev-h...@solr.apache.org> <mailto: >> dev-h...@solr.apache.org <mailto:dev-h...@solr.apache.org>> >> _______________________ >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | >> http://www.opensourceconnections.com <http://www.opensourceconnections.com/> >> < >> http://www.opensourceconnections.com/> | My Free/Busy < >> http://tinyurl.com/eric-cal> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < >> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> >> >> This e-mail and all contents, including attachments, is considered to be >> Company Confidential unless explicitly stated otherwise, regardless of >> whether attachments are marked as such. _______________________ Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
