I won't step in the way of a single file update. I haven't needed it so far though. I usually have the configsets in a Git repo (all the configset together) and I have a simple bash script that essentialy what's described in the docs[1]: Generate the zip on the fly and upload (optionally set the auth too). This can become a problem with big zips, but then again, ZooKeeper limits the size of the configs, so far it hasn't been an issue for me.
[1] https://lucene.apache.org/solr/guide/8_6/configsets-api.html#configsets-upload On Thu, Sep 24, 2020 at 7:13 AM Eric Pugh <ep...@opensourceconnections.com> wrote: > It would be great if we had a simple API for updating a file in a > configset that didn’t assume you were just uploading a zip file. > > As an example use case, if you use the Querqy library, you need to deploy > a “rules.txt” file, which in olden days just went on the filesystem and you > would click reload on a core. In the SolrCloud world, we do this awkward > “Let me stick the file in Zookeeper directly, avoiding Solr, and then do a > collection RELOAD” to push out the file everywhere. [1] It works! But > it’s just awkward. > > It’s great to know that I’ll be able to change out this awkward process > using these magic parameters to configSet. Even nicer would be to just > wrap the overwrite=true&cleanup=false and the Zip requirement into > something that sets those. > > > [1] > https://github.com/querqy/chorus/blob/master/smui/conf/smui2solrcloud.sh#L37 > > On Sep 23, 2020, at 11:57 PM, Tomás Fernández Löbbe <tomasflo...@gmail.com> > wrote: > > > Hmmm; seems the configSet API doesn't have an API to update a single > file! I wonder if uploading a configSet to the same name effectively > overwrites newly updated files but does not delete the existing files? > I've been working on this recently. As of 8.7, the UPLOAD command supports > overwriting (before, an UPLOAD on an existing configset name would fail > with BAD_REQUEST) and you can choose to cleanup or not the extra files with > the "cleanup" parameter. > You could upload a single file if you say overwrite=true&cleanup=false, > but it would still need to be in a zip file (and needs to be located in the > right path of the zip, for example, a synonyms file may be in > lang/synonyms_foo.txt or something) > > On Wed, Sep 23, 2020 at 8:10 PM David Smiley <dsmi...@apache.org> wrote: > >> +1 to deprecate managed resources in lieu of easier to maintain (and more >> flexible) file based GET/PUT into the configset. >> >> > I don't know if 9 is too soon from a deprecation stand point >> >> IMO it's never too soon as long as there is a deprecated release. Users >> take their time upgrading to major versions. >> >> > How much harder are the use-cases currently covered by managed >> resources, if that module was removed? >> >> I believe in practice, users synchronize one-way from their DB to Solr if >> they have dynamic resources like this. This is true where I work. >> Otherwise, they would probably be using Solr as the source of truth, which >> doesn't seem architecturally-sound for most apps IMO. Those users >> (hopefully few) would have to spend some time re-engineering the approach. >> Given one-way sync, the transition here is pretty easy: serialize the >> client-managed data to the right Solr format (stopwords vs synonyms vs ...) >> and then a file upload to Solr/ZK and then telling Solr which collections >> to "reload". >> >> Hmmm; seems the configSet API doesn't have an API to update a single >> file! I wonder if uploading a configSet to the same name effectively >> overwrites newly updated files but does not delete the existing files? >> >> ~ David Smiley >> Apache Lucene/Solr Search Developer >> http://www.linkedin.com/in/davidwsmiley >> >> >> On Wed, Sep 23, 2020 at 10:28 AM Timothy Potter <thelabd...@gmail.com> >> wrote: >> >>> I agree we should deprecate the managed resources feature, it was the >>> first thing I was asked to build by LW nearly 7 years ago, before I was a >>> committer. Restlet was already in place and I built on top of that, not >>> sure who introduced it originally (nor do I care). Clearly from the vantage >>> point of looking back, JAX-RS and Jersey won the day with REST in Java but >>> that simply wasn't the case back then. What's important is how we move >>> forward vs. bestowing judgement backed by wisdom of hindsight on decisions >>> made many years ago. >>> >>> In the short term, does Apache have an Artifactory (or similar) where we >>> can host the Restlet dependencies for Github to pull them from? If not, >>> then we can port the code that's using Restlet over to using JAX-RS / >>> Jersey. Personally I'd prefer we remove Managed Resources support from 9 >>> instead of porting the Restlet code but I don't know if 9 is too soon from >>> a deprecation stand point? >>> >>> Tim >>> >>> >>> On Mon, Sep 21, 2020 at 11:33 PM Noble Paul <noble.p...@gmail.com> >>> wrote: >>> >>>> We should deprecate that feature and remove restlet dependency >>>> altogether >>>> >>>> On Mon, Sep 21, 2020 at 10:20 PM Joel Bernstein <joels...@gmail.com> >>>> wrote: >>>> > >>>> > Restlet again!!!!!!! >>>> > >>>> > >>>> > >>>> > Joel Bernstein >>>> > http://joelsolr.blogspot.com/ >>>> > >>>> > >>>> > On Mon, Sep 21, 2020 at 7:18 AM Eric Pugh < >>>> ep...@opensourceconnections.com> wrote: >>>> >> >>>> >> Do we have a community blessed alternative to restlet already? >>>> >> >>>> >> On Sep 20, 2020, at 9:40 AM, Noble Paul <noble.p...@gmail.com> >>>> wrote: >>>> >> >>>> >> Haha. >>>> >> >>>> >> In fact schema APIs don't use restlet. Only the managed resources >>>> use it >>>> >> >>>> >> On Sat, Sep 19, 2020, 3:35 PM Ishan Chattopadhyaya < >>>> ichattopadhy...@gmail.com> wrote: >>>> >>> >>>> >>> If I were talend, I'd immediately start publishing to maven >>>> central. If I were the developer who built the schema APIs, I would never >>>> have used restlet to begin with. >>>> >>> >>>> >>> On Sat, 19 Sep, 2020, 1:13 am Uwe Schindler, <u...@thetaphi.de> >>>> wrote: >>>> >>>> >>>> >>>> I was thinking the same. Because GitHub does not cache the >>>> downloaded artifacts like our jenkins servers. >>>> >>>> >>>> >>>> It seems to run it in a new VM or container every time, so it >>>> downloads all artifacts. If I were Talend, I'd also block this. >>>> >>>> >>>> >>>> Uwe >>>> >>>> >>>> >>>> Am September 18, 2020 7:32:47 PM UTC schrieb Dawid Weiss < >>>> dawid.we...@gmail.com>: >>>> >>>>> >>>> >>>>> I don't think it's http/https - I believe restlet repository >>>> simply >>>> >>>>> bans github servers because of excessive traffic? These URLs work >>>> for >>>> >>>>> me locally... >>>> >>>>> >>>> >>>>> Dawid >>>> >>>>> >>>> >>>>> On Fri, Sep 18, 2020 at 6:35 PM Christine Poerschke (BLOOMBERG/ >>>> >>>>> LONDON) <cpoersc...@bloomberg.net> wrote: >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> This sounds vaguely familiar. "http works, https does not work" >>>> and https://issues.apache.org/jira/browse/SOLR-13756 possibly related? >>>> >>>>>> >>>> >>>>>> From: dev@lucene.apache.org At: 09/18/20 10:01:29 >>>> >>>>>> To: dev@lucene.apache.org >>>> >>>>>> Subject: Re: restlet dependencies >>>> >>>>>> >>>> >>>>>> I don't think it is, sadly. >>>> >>>>>> https://repo1.maven.org/maven2/org/restlet >>>> >>>>>> >>>> >>>>>> The link you provided (mvnrepository) aggregates from several >>>> maven >>>> >>>>>> repositories. >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> D. >>>> >>>>>> >>>> >>>>>> On Fri, Sep 18, 2020 at 10:46 AM Ishan Chattopadhyaya >>>> >>>>>> <ichattopadhy...@gmail.com> wrote: >>>> >>>>>>> >>>> >>>>>>> >>>> >>>>>>> Sorry, afk, but I heard (*hearsay*) that restlet is also on >>>> maven central >>>> >>>>>> >>>> >>>>>> these days. Can we confirm and switch to that? Sorry, if that's >>>> not the case. >>>> >>>>>>> >>>> >>>>>>> >>>> >>>>>>> On Fri, 18 Sep, 2020, 1:15 pm Dawid Weiss, < >>>> dawid.we...@gmail.com> wrote: >>>> >>>>>>>> >>>> >>>>>>>> >>>> >>>>>>>> Just FYI: can't get PR builds on github to work recently >>>> because of this: >>>> >>>>>>>> >>>> >>>>>>>>> Could not resolve all files for configuration >>>> >>>>>> >>>> >>>>>> ':solr:core:compileClasspath'. >>>> >>>>>>>> >>>> >>>>>>>> 350 > Could not download org.restlet.ext.servlet-2.4.3.jar >>>> >>>>>>>> (org.restlet.jee:org.restlet.ext.servlet:2.4.3) >>>> >>>>>>>> 351 > Could not get resource >>>> >>>>>>>> >>>> >>>>>> ' >>>> https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >>>> >>>>>> tlet.ext.servlet-2.4.3.jar'. >>>> >>>>>>>> >>>> >>>>>>>> 352 > Could not GET >>>> >>>>>>>> >>>> >>>>>> ' >>>> https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >>>> >>>>>> tlet.ext.servlet-2.4.3.jar'. >>>> >>>>>>>> >>>> >>>>>>>> 353 > Connection reset >>>> >>>>>>>> 354 > Could not download org.restlet-2.4.3.jar >>>> >>>>>>>> (org.restlet.jee:org.restlet:2.4.3) >>>> >>>>>>>> 355 > Could not get resource >>>> >>>>>>>> >>>> >>>>>> ' >>>> https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.j >>>> >>>>>> ar'. >>>> >>>>>>>> >>>> >>>>>>>> 356 > Could not GET >>>> >>>>>>>> >>>> >>>>>> ' >>>> https://maven.restlet.talend.com/org/restlet/jee/org.restlet/2.4.3/org.restlet- >>>> >>>>>> 2.4.3.jar'. >>>> >>>>>>>> >>>> >>>>>>>> 357 > Connection reset >>>> >>>>>>>> >>>> >>>>>>>> D. >>>> >>>>>>>> ________________________________ >>>> >>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>> >>>>>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>> >>>>>>>> >>>> >>>>>> ________________________________ >>>> >>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>> >>>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>> >>>>>> >>>> >>>>>> >>>> >>>>> ________________________________ >>>> >>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>> >>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Uwe Schindler >>>> >>>> Achterdiek 19, 28357 Bremen >>>> >>>> https://www.thetaphi.de >>>> >> >>>> >> >>>> >> _______________________ >>>> >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | >>>> 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy >>>> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >>>> >> This e-mail and all contents, including attachments, is considered >>>> to be Company Confidential unless explicitly stated otherwise, regardless >>>> of whether attachments are marked as such. >>>> >> >>>> >>>> >>>> -- >>>> ----------------------------------------------------- >>>> Noble Paul >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>> >>>> > _______________________ > *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467 > | http://www.opensourceconnections.com | My Free/Busy > <http://tinyurl.com/eric-cal> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed > <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> > This e-mail and all contents, including attachments, is considered to be > Company Confidential unless explicitly stated otherwise, regardless > of whether attachments are marked as such. > >
