It would be great if we had a simple API for updating a file in a configset that didn’t assume you were just uploading a zip file.
As an example use case, if you use the Querqy library, you need to deploy a “rules.txt” file, which in olden days just went on the filesystem and you would click reload on a core. In the SolrCloud world, we do this awkward “Let me stick the file in Zookeeper directly, avoiding Solr, and then do a collection RELOAD” to push out the file everywhere. [1] It works! But it’s just awkward. It’s great to know that I’ll be able to change out this awkward process using these magic parameters to configSet. Even nicer would be to just wrap the overwrite=true&cleanup=false and the Zip requirement into something that sets those. [1] https://github.com/querqy/chorus/blob/master/smui/conf/smui2solrcloud.sh#L37 > On Sep 23, 2020, at 11:57 PM, Tomás Fernández Löbbe <[email protected]> > wrote: > > > Hmmm; seems the configSet API doesn't have an API to update a single file! > > I wonder if uploading a configSet to the same name effectively overwrites > > newly updated files but does not delete the existing files? > I've been working on this recently. As of 8.7, the UPLOAD command supports > overwriting (before, an UPLOAD on an existing configset name would fail with > BAD_REQUEST) and you can choose to cleanup or not the extra files with the > "cleanup" parameter. > You could upload a single file if you say overwrite=true&cleanup=false, but > it would still need to be in a zip file (and needs to be located in the right > path of the zip, for example, a synonyms file may be in lang/synonyms_foo.txt > or something) > > On Wed, Sep 23, 2020 at 8:10 PM David Smiley <[email protected] > <mailto:[email protected]>> wrote: > +1 to deprecate managed resources in lieu of easier to maintain (and more > flexible) file based GET/PUT into the configset. > > > I don't know if 9 is too soon from a deprecation stand point > > IMO it's never too soon as long as there is a deprecated release. Users take > their time upgrading to major versions. > > > How much harder are the use-cases currently covered by managed resources, > > if that module was removed? > > I believe in practice, users synchronize one-way from their DB to Solr if > they have dynamic resources like this. This is true where I work. > Otherwise, they would probably be using Solr as the source of truth, which > doesn't seem architecturally-sound for most apps IMO. Those users (hopefully > few) would have to spend some time re-engineering the approach. Given > one-way sync, the transition here is pretty easy: serialize the > client-managed data to the right Solr format (stopwords vs synonyms vs ...) > and then a file upload to Solr/ZK and then telling Solr which collections to > "reload". > > Hmmm; seems the configSet API doesn't have an API to update a single file! I > wonder if uploading a configSet to the same name effectively overwrites newly > updated files but does not delete the existing files? > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > <http://www.linkedin.com/in/davidwsmiley> > > On Wed, Sep 23, 2020 at 10:28 AM Timothy Potter <[email protected] > <mailto:[email protected]>> wrote: > I agree we should deprecate the managed resources feature, it was the first > thing I was asked to build by LW nearly 7 years ago, before I was a > committer. Restlet was already in place and I built on top of that, not sure > who introduced it originally (nor do I care). Clearly from the vantage point > of looking back, JAX-RS and Jersey won the day with REST in Java but that > simply wasn't the case back then. What's important is how we move forward vs. > bestowing judgement backed by wisdom of hindsight on decisions made many > years ago. > > In the short term, does Apache have an Artifactory (or similar) where we can > host the Restlet dependencies for Github to pull them from? If not, then we > can port the code that's using Restlet over to using JAX-RS / Jersey. > Personally I'd prefer we remove Managed Resources support from 9 instead of > porting the Restlet code but I don't know if 9 is too soon from a deprecation > stand point? > > Tim > > > On Mon, Sep 21, 2020 at 11:33 PM Noble Paul <[email protected] > <mailto:[email protected]>> wrote: > We should deprecate that feature and remove restlet dependency altogether > > On Mon, Sep 21, 2020 at 10:20 PM Joel Bernstein <[email protected] > <mailto:[email protected]>> wrote: > > > > Restlet again!!!!!!! > > > > > > > > Joel Bernstein > > http://joelsolr.blogspot.com/ <http://joelsolr.blogspot.com/> > > > > > > On Mon, Sep 21, 2020 at 7:18 AM Eric Pugh <[email protected] > > <mailto:[email protected]>> wrote: > >> > >> Do we have a community blessed alternative to restlet already? > >> > >> On Sep 20, 2020, at 9:40 AM, Noble Paul <[email protected] > >> <mailto:[email protected]>> wrote: > >> > >> Haha. > >> > >> In fact schema APIs don't use restlet. Only the managed resources use it > >> > >> On Sat, Sep 19, 2020, 3:35 PM Ishan Chattopadhyaya > >> <[email protected] <mailto:[email protected]>> wrote: > >>> > >>> If I were talend, I'd immediately start publishing to maven central. If I > >>> were the developer who built the schema APIs, I would never have used > >>> restlet to begin with. > >>> > >>> On Sat, 19 Sep, 2020, 1:13 am Uwe Schindler, <[email protected] > >>> <mailto:[email protected]>> wrote: > >>>> > >>>> I was thinking the same. Because GitHub does not cache the downloaded > >>>> artifacts like our jenkins servers. > >>>> > >>>> It seems to run it in a new VM or container every time, so it downloads > >>>> all artifacts. If I were Talend, I'd also block this. > >>>> > >>>> Uwe > >>>> > >>>> Am September 18, 2020 7:32:47 PM UTC schrieb Dawid Weiss > >>>> <[email protected] <mailto:[email protected]>>: > >>>>> > >>>>> I don't think it's http/https - I believe restlet repository simply > >>>>> bans github servers because of excessive traffic? These URLs work for > >>>>> me locally... > >>>>> > >>>>> Dawid > >>>>> > >>>>> On Fri, Sep 18, 2020 at 6:35 PM Christine Poerschke (BLOOMBERG/ > >>>>> LONDON) <[email protected] <mailto:[email protected]>> > >>>>> wrote: > >>>>>> > >>>>>> > >>>>>> This sounds vaguely familiar. "http works, https does not work" and > >>>>>> https://issues.apache.org/jira/browse/SOLR-13756 > >>>>>> <https://issues.apache.org/jira/browse/SOLR-13756> possibly related? > >>>>>> > >>>>>> From: [email protected] <mailto:[email protected]> At: > >>>>>> 09/18/20 10:01:29 > >>>>>> To: [email protected] <mailto:[email protected]> > >>>>>> Subject: Re: restlet dependencies > >>>>>> > >>>>>> I don't think it is, sadly. > >>>>>> https://repo1.maven.org/maven2/org/restlet > >>>>>> <https://repo1.maven.org/maven2/org/restlet> > >>>>>> > >>>>>> The link you provided (mvnrepository) aggregates from several maven > >>>>>> repositories. > >>>>>> > >>>>>> > >>>>>> D. > >>>>>> > >>>>>> On Fri, Sep 18, 2020 at 10:46 AM Ishan Chattopadhyaya > >>>>>> <[email protected] <mailto:[email protected]>> wrote: > >>>>>>> > >>>>>>> > >>>>>>> Sorry, afk, but I heard (*hearsay*) that restlet is also on maven > >>>>>>> central > >>>>>> > >>>>>> these days. Can we confirm and switch to that? Sorry, if that's not > >>>>>> the case. > >>>>>>> > >>>>>>> > >>>>>>> On Fri, 18 Sep, 2020, 1:15 pm Dawid Weiss, <[email protected] > >>>>>>> <mailto:[email protected]>> wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> Just FYI: can't get PR builds on github to work recently because of > >>>>>>>> this: > >>>>>>>> > >>>>>>>>> Could not resolve all files for configuration > >>>>>> > >>>>>> ':solr:core:compileClasspath'. > >>>>>>>> > >>>>>>>> 350 > Could not download org.restlet.ext.servlet-2.4.3.jar > >>>>>>>> (org.restlet.jee:org.restlet.ext.servlet:2.4.3) > >>>>>>>> 351 > Could not get resource > >>>>>>>> > >>>>>> 'https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res > >>>>>> > >>>>>> <https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res> > >>>>>> tlet.ext.servlet-2.4.3.jar'. > >>>>>>>> > >>>>>>>> 352 > Could not GET > >>>>>>>> > >>>>>> 'https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res > >>>>>> > >>>>>> <https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res> > >>>>>> tlet.ext.servlet-2.4.3.jar'. > >>>>>>>> > >>>>>>>> 353 > Connection reset > >>>>>>>> 354 > Could not download org.restlet-2.4.3.jar > >>>>>>>> (org.restlet.jee:org.restlet:2.4.3) > >>>>>>>> 355 > Could not get resource > >>>>>>>> > >>>>>> 'https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.j > >>>>>> > >>>>>> <https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.j> > >>>>>> ar'. > >>>>>>>> > >>>>>>>> 356 > Could not GET > >>>>>>>> > >>>>>> 'https://maven.restlet.talend.com/org/restlet/jee/org.restlet/2.4.3/org.restlet- > >>>>>> > >>>>>> <https://maven.restlet.talend.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-> > >>>>>> 2.4.3.jar'. > >>>>>>>> > >>>>>>>> 357 > Connection reset > >>>>>>>> > >>>>>>>> D. > >>>>>>>> ________________________________ > >>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>> <mailto:[email protected]> > >>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>> <mailto:[email protected]> > >>>>>>>> > >>>>>> ________________________________ > >>>>>> To unsubscribe, e-mail: [email protected] > >>>>>> <mailto:[email protected]> > >>>>>> For additional commands, e-mail: [email protected] > >>>>>> <mailto:[email protected]> > >>>>>> > >>>>>> > >>>>> ________________________________ > >>>>> To unsubscribe, e-mail: [email protected] > >>>>> <mailto:[email protected]> > >>>>> For additional commands, e-mail: [email protected] > >>>>> <mailto:[email protected]> > >>>>> > >>>> > >>>> -- > >>>> Uwe Schindler > >>>> Achterdiek 19, 28357 Bremen > >>>> https://www.thetaphi.de <https://www.thetaphi.de/> > >> > >> > >> _______________________ > >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | > >> http://www.opensourceconnections.com > >> <http://www.opensourceconnections.com/> | My Free/Busy > >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed > >> This e-mail and all contents, including attachments, is considered to be > >> Company Confidential unless explicitly stated otherwise, regardless of > >> whether attachments are marked as such. > >> > > > -- > ----------------------------------------------------- > Noble Paul > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > <mailto:[email protected]> > For additional commands, e-mail: [email protected] > <mailto:[email protected]> > _______________________ Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
