Yea, managed resources is one of the features I’ll not miss. It seemed smart back then, but for some reason I don’t think people use it that much, and I agree it makes more sense for the user to be aware of the configset and update the configset.
I’d appreciate a upload one file API. Ishan, you could say that you can upload a single file to a config set only if auth is enabled OR if the configset is already untrusted. Jan > 25. sep. 2020 kl. 06:02 skrev Ishan Chattopadhyaya > <ichattopadhy...@gmail.com>: > > SOLR-5287 has some discussion. The changes were backed out due to security > reasons. Anyway, Eric, Tomas & David, let us not hijack this thread on > restlet and discuss something totally unrelated. > > On Fri, 25 Sep, 2020, 9:20 am Ishan Chattopadhyaya, > <ichattopadhy...@gmail.com <mailto:ichattopadhy...@gmail.com>> wrote: > An entire configset, when uploaded via api without security enabled, is > marked untrusted. When creating a collection from untrusted configset, > certain vulnerable components can't be initialized. > > On Fri, 25 Sep, 2020, 9:04 am David Smiley, <dsmi...@apache.org > <mailto:dsmi...@apache.org>> wrote: > Ishan: can you be more specific please? How is it less secure or harder to > secure than, say, a configSet upload (internally multiple files)? > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > <http://www.linkedin.com/in/davidwsmiley> > > On Thu, Sep 24, 2020 at 1:53 PM Ishan Chattopadhyaya > <ichattopadhy...@gmail.com <mailto:ichattopadhy...@gmail.com>> wrote: > Single file update capability is a security nightmare. Even if it can be > done, it should be supported only once authc/authz have been enabled. > > On Thu, 24 Sep, 2020, 10:16 pm Tomás Fernández Löbbe, <tomasflo...@gmail.com > <mailto:tomasflo...@gmail.com>> wrote: > I won't step in the way of a single file update. I haven't needed it so far > though. I usually have the configsets in a Git repo (all the configset > together) and I have a simple bash script that essentialy what's described in > the docs[1]: Generate the zip on the fly and upload (optionally set the auth > too). This can become a problem with big zips, but then again, ZooKeeper > limits the size of the configs, so far it hasn't been an issue for me. > > > [1] > https://lucene.apache.org/solr/guide/8_6/configsets-api.html#configsets-upload > > <https://lucene.apache.org/solr/guide/8_6/configsets-api.html#configsets-upload> > On Thu, Sep 24, 2020 at 7:13 AM Eric Pugh <ep...@opensourceconnections.com > <mailto:ep...@opensourceconnections.com>> wrote: > It would be great if we had a simple API for updating a file in a configset > that didn’t assume you were just uploading a zip file. > > As an example use case, if you use the Querqy library, you need to deploy a > “rules.txt” file, which in olden days just went on the filesystem and you > would click reload on a core. In the SolrCloud world, we do this awkward > “Let me stick the file in Zookeeper directly, avoiding Solr, and then do a > collection RELOAD” to push out the file everywhere. [1] It works! But it’s > just awkward. > > It’s great to know that I’ll be able to change out this awkward process using > these magic parameters to configSet. Even nicer would be to just wrap the > overwrite=true&cleanup=false and the Zip requirement into something that sets > those. > > > [1] > https://github.com/querqy/chorus/blob/master/smui/conf/smui2solrcloud.sh#L37 > <https://github.com/querqy/chorus/blob/master/smui/conf/smui2solrcloud.sh#L37> > >> On Sep 23, 2020, at 11:57 PM, Tomás Fernández Löbbe <tomasflo...@gmail.com >> <mailto:tomasflo...@gmail.com>> wrote: >> >> > Hmmm; seems the configSet API doesn't have an API to update a single file! >> > I wonder if uploading a configSet to the same name effectively overwrites >> > newly updated files but does not delete the existing files? >> I've been working on this recently. As of 8.7, the UPLOAD command supports >> overwriting (before, an UPLOAD on an existing configset name would fail with >> BAD_REQUEST) and you can choose to cleanup or not the extra files with the >> "cleanup" parameter. >> You could upload a single file if you say overwrite=true&cleanup=false, but >> it would still need to be in a zip file (and needs to be located in the >> right path of the zip, for example, a synonyms file may be in >> lang/synonyms_foo.txt or something) >> >> On Wed, Sep 23, 2020 at 8:10 PM David Smiley <dsmi...@apache.org >> <mailto:dsmi...@apache.org>> wrote: >> +1 to deprecate managed resources in lieu of easier to maintain (and more >> flexible) file based GET/PUT into the configset. >> >> > I don't know if 9 is too soon from a deprecation stand point >> >> IMO it's never too soon as long as there is a deprecated release. Users >> take their time upgrading to major versions. >> >> > How much harder are the use-cases currently covered by managed resources, >> > if that module was removed? >> >> I believe in practice, users synchronize one-way from their DB to Solr if >> they have dynamic resources like this. This is true where I work. >> Otherwise, they would probably be using Solr as the source of truth, which >> doesn't seem architecturally-sound for most apps IMO. Those users >> (hopefully few) would have to spend some time re-engineering the approach. >> Given one-way sync, the transition here is pretty easy: serialize the >> client-managed data to the right Solr format (stopwords vs synonyms vs ...) >> and then a file upload to Solr/ZK and then telling Solr which collections to >> "reload". >> >> Hmmm; seems the configSet API doesn't have an API to update a single file! >> I wonder if uploading a configSet to the same name effectively overwrites >> newly updated files but does not delete the existing files? >> >> ~ David Smiley >> Apache Lucene/Solr Search Developer >> http://www.linkedin.com/in/davidwsmiley >> <http://www.linkedin.com/in/davidwsmiley> >> >> On Wed, Sep 23, 2020 at 10:28 AM Timothy Potter <thelabd...@gmail.com >> <mailto:thelabd...@gmail.com>> wrote: >> I agree we should deprecate the managed resources feature, it was the first >> thing I was asked to build by LW nearly 7 years ago, before I was a >> committer. Restlet was already in place and I built on top of that, not sure >> who introduced it originally (nor do I care). Clearly from the vantage point >> of looking back, JAX-RS and Jersey won the day with REST in Java but that >> simply wasn't the case back then. What's important is how we move forward >> vs. bestowing judgement backed by wisdom of hindsight on decisions made many >> years ago. >> >> In the short term, does Apache have an Artifactory (or similar) where we can >> host the Restlet dependencies for Github to pull them from? If not, then we >> can port the code that's using Restlet over to using JAX-RS / Jersey. >> Personally I'd prefer we remove Managed Resources support from 9 instead of >> porting the Restlet code but I don't know if 9 is too soon from a >> deprecation stand point? >> >> Tim >> >> >> On Mon, Sep 21, 2020 at 11:33 PM Noble Paul <noble.p...@gmail.com >> <mailto:noble.p...@gmail.com>> wrote: >> We should deprecate that feature and remove restlet dependency altogether >> >> On Mon, Sep 21, 2020 at 10:20 PM Joel Bernstein <joels...@gmail.com >> <mailto:joels...@gmail.com>> wrote: >> > >> > Restlet again!!!!!!! >> > >> > >> > >> > Joel Bernstein >> > http://joelsolr.blogspot.com/ <http://joelsolr.blogspot.com/> >> > >> > >> > On Mon, Sep 21, 2020 at 7:18 AM Eric Pugh <ep...@opensourceconnections.com >> > <mailto:ep...@opensourceconnections.com>> wrote: >> >> >> >> Do we have a community blessed alternative to restlet already? >> >> >> >> On Sep 20, 2020, at 9:40 AM, Noble Paul <noble.p...@gmail.com >> >> <mailto:noble.p...@gmail.com>> wrote: >> >> >> >> Haha. >> >> >> >> In fact schema APIs don't use restlet. Only the managed resources use it >> >> >> >> On Sat, Sep 19, 2020, 3:35 PM Ishan Chattopadhyaya >> >> <ichattopadhy...@gmail.com <mailto:ichattopadhy...@gmail.com>> wrote: >> >>> >> >>> If I were talend, I'd immediately start publishing to maven central. If >> >>> I were the developer who built the schema APIs, I would never have used >> >>> restlet to begin with. >> >>> >> >>> On Sat, 19 Sep, 2020, 1:13 am Uwe Schindler, <u...@thetaphi.de >> >>> <mailto:u...@thetaphi.de>> wrote: >> >>>> >> >>>> I was thinking the same. Because GitHub does not cache the downloaded >> >>>> artifacts like our jenkins servers. >> >>>> >> >>>> It seems to run it in a new VM or container every time, so it downloads >> >>>> all artifacts. If I were Talend, I'd also block this. >> >>>> >> >>>> Uwe >> >>>> >> >>>> Am September 18, 2020 7:32:47 PM UTC schrieb Dawid Weiss >> >>>> <dawid.we...@gmail.com <mailto:dawid.we...@gmail.com>>: >> >>>>> >> >>>>> I don't think it's http/https - I believe restlet repository simply >> >>>>> bans github servers because of excessive traffic? These URLs work for >> >>>>> me locally... >> >>>>> >> >>>>> Dawid >> >>>>> >> >>>>> On Fri, Sep 18, 2020 at 6:35 PM Christine Poerschke (BLOOMBERG/ >> >>>>> LONDON) <cpoersc...@bloomberg.net <mailto:cpoersc...@bloomberg.net>> >> >>>>> wrote: >> >>>>>> >> >>>>>> >> >>>>>> This sounds vaguely familiar. "http works, https does not work" and >> >>>>>> https://issues.apache.org/jira/browse/SOLR-13756 >> >>>>>> <https://issues.apache.org/jira/browse/SOLR-13756> possibly related? >> >>>>>> >> >>>>>> From: dev@lucene.apache.org <mailto:dev@lucene.apache.org> At: >> >>>>>> 09/18/20 10:01:29 >> >>>>>> To: dev@lucene.apache.org <mailto:dev@lucene.apache.org> >> >>>>>> Subject: Re: restlet dependencies >> >>>>>> >> >>>>>> I don't think it is, sadly. >> >>>>>> https://repo1.maven.org/maven2/org/restlet >> >>>>>> <https://repo1.maven.org/maven2/org/restlet> >> >>>>>> >> >>>>>> The link you provided (mvnrepository) aggregates from several maven >> >>>>>> repositories. >> >>>>>> >> >>>>>> >> >>>>>> D. >> >>>>>> >> >>>>>> On Fri, Sep 18, 2020 at 10:46 AM Ishan Chattopadhyaya >> >>>>>> <ichattopadhy...@gmail.com <mailto:ichattopadhy...@gmail.com>> wrote: >> >>>>>>> >> >>>>>>> >> >>>>>>> Sorry, afk, but I heard (*hearsay*) that restlet is also on maven >> >>>>>>> central >> >>>>>> >> >>>>>> these days. Can we confirm and switch to that? Sorry, if that's not >> >>>>>> the case. >> >>>>>>> >> >>>>>>> >> >>>>>>> On Fri, 18 Sep, 2020, 1:15 pm Dawid Weiss, <dawid.we...@gmail.com >> >>>>>>> <mailto:dawid.we...@gmail.com>> wrote: >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> Just FYI: can't get PR builds on github to work recently because >> >>>>>>>> of this: >> >>>>>>>> >> >>>>>>>>> Could not resolve all files for configuration >> >>>>>> >> >>>>>> ':solr:core:compileClasspath'. >> >>>>>>>> >> >>>>>>>> 350 > Could not download org.restlet.ext.servlet-2.4.3.jar >> >>>>>>>> (org.restlet.jee:org.restlet.ext.servlet:2.4.3) >> >>>>>>>> 351 > Could not get resource >> >>>>>>>> >> >>>>>> 'https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >> >>>>>> >> >>>>>> <https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res> >> >>>>>> tlet.ext.servlet-2.4.3.jar'. >> >>>>>>>> >> >>>>>>>> 352 > Could not GET >> >>>>>>>> >> >>>>>> 'https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >> >>>>>> >> >>>>>> <https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res> >> >>>>>> tlet.ext.servlet-2.4.3.jar'. >> >>>>>>>> >> >>>>>>>> 353 > Connection reset >> >>>>>>>> 354 > Could not download org.restlet-2.4.3.jar >> >>>>>>>> (org.restlet.jee:org.restlet:2.4.3) >> >>>>>>>> 355 > Could not get resource >> >>>>>>>> >> >>>>>> 'https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.j >> >>>>>> >> >>>>>> <https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.j> >> >>>>>> ar'. >> >>>>>>>> >> >>>>>>>> 356 > Could not GET >> >>>>>>>> >> >>>>>> 'https://maven.restlet.talend.com/org/restlet/jee/org.restlet/2.4.3/org.restlet- >> >>>>>> >> >>>>>> <https://maven.restlet.talend.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-> >> >>>>>> 2.4.3.jar'. >> >>>>>>>> >> >>>>>>>> 357 > Connection reset >> >>>>>>>> >> >>>>>>>> D. >> >>>>>>>> ________________________________ >> >>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> >>>>>>>> <mailto:dev-unsubscr...@lucene.apache.org> >> >>>>>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >> >>>>>>>> <mailto:dev-h...@lucene.apache.org> >> >>>>>>>> >> >>>>>> ________________________________ >> >>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> >>>>>> <mailto:dev-unsubscr...@lucene.apache.org> >> >>>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >> >>>>>> <mailto:dev-h...@lucene.apache.org> >> >>>>>> >> >>>>>> >> >>>>> ________________________________ >> >>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> >>>>> <mailto:dev-unsubscr...@lucene.apache.org> >> >>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >> >>>>> <mailto:dev-h...@lucene.apache.org> >> >>>>> >> >>>> >> >>>> -- >> >>>> Uwe Schindler >> >>>> Achterdiek 19, 28357 Bremen >> >>>> https://www.thetaphi.de <https://www.thetaphi.de/> >> >> >> >> >> >> _______________________ >> >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | >> >> http://www.opensourceconnections.com >> >> <http://www.opensourceconnections.com/> | My Free/Busy >> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >> >> This e-mail and all contents, including attachments, is considered to be >> >> Company Confidential unless explicitly stated otherwise, regardless of >> >> whether attachments are marked as such. >> >> >> >> >> -- >> ----------------------------------------------------- >> Noble Paul >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> <mailto:dev-unsubscr...@lucene.apache.org> >> For additional commands, e-mail: dev-h...@lucene.apache.org >> <mailto:dev-h...@lucene.apache.org> >> > > _______________________ > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | > http://www.opensourceconnections.com <http://www.opensourceconnections.com/> > | My Free/Busy <http://tinyurl.com/eric-cal> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed > <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> > > This e-mail and all contents, including attachments, is considered to be > Company Confidential unless explicitly stated otherwise, regardless of > whether attachments are marked as such. >
