What if we suggest that you just change the files directly using the same configset discussion we had, with caveats etc. And maybe someday we get a nicer api etc for doing this type of thing...
> On Sep 25, 2020, at 9:49 AM, Timothy Potter <[email protected]> wrote: > > Ok, so we're removing from 9 (I can do that) but what should we do about the > upcoming 8.7 release? I'm fine with deprecating, but usually an alternative > approach is suggested with the deprecation notice. For cloud mode, we can > suggest our hypothetical users of this MR API to use the config set API but > what about standalone? I can do the work in 8.7 too but not really sure how > to proceed on the standalone question ... > > On Fri, Sep 25, 2020 at 12:20 AM Jan Høydahl <[email protected] > <mailto:[email protected]>> wrote: > Yea, managed resources is one of the features I’ll not miss. It seemed smart > back then, but for some reason I don’t think people use it that much, and I > agree it makes more sense for the user to be aware of the configset and > update the configset. > > I’d appreciate a upload one file API. Ishan, you could say that you can > upload a single file to a config set only if auth is enabled OR if the > configset is already untrusted. > > Jan > >> 25. sep. 2020 kl. 06:02 skrev Ishan Chattopadhyaya >> <[email protected] <mailto:[email protected]>>: >> >> SOLR-5287 has some discussion. The changes were backed out due to security >> reasons. Anyway, Eric, Tomas & David, let us not hijack this thread on >> restlet and discuss something totally unrelated. >> >> On Fri, 25 Sep, 2020, 9:20 am Ishan Chattopadhyaya, >> <[email protected] <mailto:[email protected]>> wrote: >> An entire configset, when uploaded via api without security enabled, is >> marked untrusted. When creating a collection from untrusted configset, >> certain vulnerable components can't be initialized. >> >> On Fri, 25 Sep, 2020, 9:04 am David Smiley, <[email protected] >> <mailto:[email protected]>> wrote: >> Ishan: can you be more specific please? How is it less secure or harder to >> secure than, say, a configSet upload (internally multiple files)? >> >> ~ David Smiley >> Apache Lucene/Solr Search Developer >> http://www.linkedin.com/in/davidwsmiley >> <http://www.linkedin.com/in/davidwsmiley> >> >> On Thu, Sep 24, 2020 at 1:53 PM Ishan Chattopadhyaya >> <[email protected] <mailto:[email protected]>> wrote: >> Single file update capability is a security nightmare. Even if it can be >> done, it should be supported only once authc/authz have been enabled. >> >> On Thu, 24 Sep, 2020, 10:16 pm Tomás Fernández Löbbe, <[email protected] >> <mailto:[email protected]>> wrote: >> I won't step in the way of a single file update. I haven't needed it so far >> though. I usually have the configsets in a Git repo (all the configset >> together) and I have a simple bash script that essentialy what's described >> in the docs[1]: Generate the zip on the fly and upload (optionally set the >> auth too). This can become a problem with big zips, but then again, >> ZooKeeper limits the size of the configs, so far it hasn't been an issue for >> me. >> >> >> [1] >> https://lucene.apache.org/solr/guide/8_6/configsets-api.html#configsets-upload >> >> <https://lucene.apache.org/solr/guide/8_6/configsets-api.html#configsets-upload> >> On Thu, Sep 24, 2020 at 7:13 AM Eric Pugh <[email protected] >> <mailto:[email protected]>> wrote: >> It would be great if we had a simple API for updating a file in a configset >> that didn’t assume you were just uploading a zip file. >> >> As an example use case, if you use the Querqy library, you need to deploy a >> “rules.txt” file, which in olden days just went on the filesystem and you >> would click reload on a core. In the SolrCloud world, we do this awkward >> “Let me stick the file in Zookeeper directly, avoiding Solr, and then do a >> collection RELOAD” to push out the file everywhere. [1] It works! But it’s >> just awkward. >> >> It’s great to know that I’ll be able to change out this awkward process >> using these magic parameters to configSet. Even nicer would be to just wrap >> the overwrite=true&cleanup=false and the Zip requirement into something that >> sets those. >> >> >> [1] >> https://github.com/querqy/chorus/blob/master/smui/conf/smui2solrcloud.sh#L37 >> <https://github.com/querqy/chorus/blob/master/smui/conf/smui2solrcloud.sh#L37> >> >>> On Sep 23, 2020, at 11:57 PM, Tomás Fernández Löbbe <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> > Hmmm; seems the configSet API doesn't have an API to update a single >>> > file! I wonder if uploading a configSet to the same name effectively >>> > overwrites newly updated files but does not delete the existing files? >>> I've been working on this recently. As of 8.7, the UPLOAD command supports >>> overwriting (before, an UPLOAD on an existing configset name would fail >>> with BAD_REQUEST) and you can choose to cleanup or not the extra files with >>> the "cleanup" parameter. >>> You could upload a single file if you say overwrite=true&cleanup=false, but >>> it would still need to be in a zip file (and needs to be located in the >>> right path of the zip, for example, a synonyms file may be in >>> lang/synonyms_foo.txt or something) >>> >>> On Wed, Sep 23, 2020 at 8:10 PM David Smiley <[email protected] >>> <mailto:[email protected]>> wrote: >>> +1 to deprecate managed resources in lieu of easier to maintain (and more >>> flexible) file based GET/PUT into the configset. >>> >>> > I don't know if 9 is too soon from a deprecation stand point >>> >>> IMO it's never too soon as long as there is a deprecated release. Users >>> take their time upgrading to major versions. >>> >>> > How much harder are the use-cases currently covered by managed resources, >>> > if that module was removed? >>> >>> I believe in practice, users synchronize one-way from their DB to Solr if >>> they have dynamic resources like this. This is true where I work. >>> Otherwise, they would probably be using Solr as the source of truth, which >>> doesn't seem architecturally-sound for most apps IMO. Those users >>> (hopefully few) would have to spend some time re-engineering the approach. >>> Given one-way sync, the transition here is pretty easy: serialize the >>> client-managed data to the right Solr format (stopwords vs synonyms vs ...) >>> and then a file upload to Solr/ZK and then telling Solr which collections >>> to "reload". >>> >>> Hmmm; seems the configSet API doesn't have an API to update a single file! >>> I wonder if uploading a configSet to the same name effectively overwrites >>> newly updated files but does not delete the existing files? >>> >>> ~ David Smiley >>> Apache Lucene/Solr Search Developer >>> http://www.linkedin.com/in/davidwsmiley >>> <http://www.linkedin.com/in/davidwsmiley> >>> >>> On Wed, Sep 23, 2020 at 10:28 AM Timothy Potter <[email protected] >>> <mailto:[email protected]>> wrote: >>> I agree we should deprecate the managed resources feature, it was the first >>> thing I was asked to build by LW nearly 7 years ago, before I was a >>> committer. Restlet was already in place and I built on top of that, not >>> sure who introduced it originally (nor do I care). Clearly from the vantage >>> point of looking back, JAX-RS and Jersey won the day with REST in Java but >>> that simply wasn't the case back then. What's important is how we move >>> forward vs. bestowing judgement backed by wisdom of hindsight on decisions >>> made many years ago. >>> >>> In the short term, does Apache have an Artifactory (or similar) where we >>> can host the Restlet dependencies for Github to pull them from? If not, >>> then we can port the code that's using Restlet over to using JAX-RS / >>> Jersey. Personally I'd prefer we remove Managed Resources support from 9 >>> instead of porting the Restlet code but I don't know if 9 is too soon from >>> a deprecation stand point? >>> >>> Tim >>> >>> >>> On Mon, Sep 21, 2020 at 11:33 PM Noble Paul <[email protected] >>> <mailto:[email protected]>> wrote: >>> We should deprecate that feature and remove restlet dependency altogether >>> >>> On Mon, Sep 21, 2020 at 10:20 PM Joel Bernstein <[email protected] >>> <mailto:[email protected]>> wrote: >>> > >>> > Restlet again!!!!!!! >>> > >>> > >>> > >>> > Joel Bernstein >>> > http://joelsolr.blogspot.com/ <http://joelsolr.blogspot.com/> >>> > >>> > >>> > On Mon, Sep 21, 2020 at 7:18 AM Eric Pugh >>> > <[email protected] >>> > <mailto:[email protected]>> wrote: >>> >> >>> >> Do we have a community blessed alternative to restlet already? >>> >> >>> >> On Sep 20, 2020, at 9:40 AM, Noble Paul <[email protected] >>> >> <mailto:[email protected]>> wrote: >>> >> >>> >> Haha. >>> >> >>> >> In fact schema APIs don't use restlet. Only the managed resources use it >>> >> >>> >> On Sat, Sep 19, 2020, 3:35 PM Ishan Chattopadhyaya >>> >> <[email protected] <mailto:[email protected]>> wrote: >>> >>> >>> >>> If I were talend, I'd immediately start publishing to maven central. If >>> >>> I were the developer who built the schema APIs, I would never have used >>> >>> restlet to begin with. >>> >>> >>> >>> On Sat, 19 Sep, 2020, 1:13 am Uwe Schindler, <[email protected] >>> >>> <mailto:[email protected]>> wrote: >>> >>>> >>> >>>> I was thinking the same. Because GitHub does not cache the downloaded >>> >>>> artifacts like our jenkins servers. >>> >>>> >>> >>>> It seems to run it in a new VM or container every time, so it >>> >>>> downloads all artifacts. If I were Talend, I'd also block this. >>> >>>> >>> >>>> Uwe >>> >>>> >>> >>>> Am September 18, 2020 7:32:47 PM UTC schrieb Dawid Weiss >>> >>>> <[email protected] <mailto:[email protected]>>: >>> >>>>> >>> >>>>> I don't think it's http/https - I believe restlet repository simply >>> >>>>> bans github servers because of excessive traffic? These URLs work for >>> >>>>> me locally... >>> >>>>> >>> >>>>> Dawid >>> >>>>> >>> >>>>> On Fri, Sep 18, 2020 at 6:35 PM Christine Poerschke (BLOOMBERG/ >>> >>>>> LONDON) <[email protected] <mailto:[email protected]>> >>> >>>>> wrote: >>> >>>>>> >>> >>>>>> >>> >>>>>> This sounds vaguely familiar. "http works, https does not work" and >>> >>>>>> https://issues.apache.org/jira/browse/SOLR-13756 >>> >>>>>> <https://issues.apache.org/jira/browse/SOLR-13756> possibly related? >>> >>>>>> >>> >>>>>> From: [email protected] <mailto:[email protected]> At: >>> >>>>>> 09/18/20 10:01:29 >>> >>>>>> To: [email protected] <mailto:[email protected]> >>> >>>>>> Subject: Re: restlet dependencies >>> >>>>>> >>> >>>>>> I don't think it is, sadly. >>> >>>>>> https://repo1.maven.org/maven2/org/restlet >>> >>>>>> <https://repo1.maven.org/maven2/org/restlet> >>> >>>>>> >>> >>>>>> The link you provided (mvnrepository) aggregates from several maven >>> >>>>>> repositories. >>> >>>>>> >>> >>>>>> >>> >>>>>> D. >>> >>>>>> >>> >>>>>> On Fri, Sep 18, 2020 at 10:46 AM Ishan Chattopadhyaya >>> >>>>>> <[email protected] <mailto:[email protected]>> >>> >>>>>> wrote: >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> Sorry, afk, but I heard (*hearsay*) that restlet is also on maven >>> >>>>>>> central >>> >>>>>> >>> >>>>>> these days. Can we confirm and switch to that? Sorry, if that's not >>> >>>>>> the case. >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> On Fri, 18 Sep, 2020, 1:15 pm Dawid Weiss, <[email protected] >>> >>>>>>> <mailto:[email protected]>> wrote: >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> Just FYI: can't get PR builds on github to work recently because >>> >>>>>>>> of this: >>> >>>>>>>> >>> >>>>>>>>> Could not resolve all files for configuration >>> >>>>>> >>> >>>>>> ':solr:core:compileClasspath'. >>> >>>>>>>> >>> >>>>>>>> 350 > Could not download org.restlet.ext.servlet-2.4.3.jar >>> >>>>>>>> (org.restlet.jee:org.restlet.ext.servlet:2.4.3) >>> >>>>>>>> 351 > Could not get resource >>> >>>>>>>> >>> >>>>>> 'https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >>> >>>>>> >>> >>>>>> <https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res> >>> >>>>>> tlet.ext.servlet-2.4.3.jar'. >>> >>>>>>>> >>> >>>>>>>> 352 > Could not GET >>> >>>>>>>> >>> >>>>>> 'https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >>> >>>>>> >>> >>>>>> <https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res> >>> >>>>>> tlet.ext.servlet-2.4.3.jar'. >>> >>>>>>>> >>> >>>>>>>> 353 > Connection reset >>> >>>>>>>> 354 > Could not download org.restlet-2.4.3.jar >>> >>>>>>>> (org.restlet.jee:org.restlet:2.4.3) >>> >>>>>>>> 355 > Could not get resource >>> >>>>>>>> >>> >>>>>> 'https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.j >>> >>>>>> >>> >>>>>> <https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.j> >>> >>>>>> ar'. >>> >>>>>>>> >>> >>>>>>>> 356 > Could not GET >>> >>>>>>>> >>> >>>>>> 'https://maven.restlet.talend.com/org/restlet/jee/org.restlet/2.4.3/org.restlet- >>> >>>>>> >>> >>>>>> <https://maven.restlet.talend.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-> >>> >>>>>> 2.4.3.jar'. >>> >>>>>>>> >>> >>>>>>>> 357 > Connection reset >>> >>>>>>>> >>> >>>>>>>> D. >>> >>>>>>>> ________________________________ >>> >>>>>>>> To unsubscribe, e-mail: [email protected] >>> >>>>>>>> <mailto:[email protected]> >>> >>>>>>>> For additional commands, e-mail: [email protected] >>> >>>>>>>> <mailto:[email protected]> >>> >>>>>>>> >>> >>>>>> ________________________________ >>> >>>>>> To unsubscribe, e-mail: [email protected] >>> >>>>>> <mailto:[email protected]> >>> >>>>>> For additional commands, e-mail: [email protected] >>> >>>>>> <mailto:[email protected]> >>> >>>>>> >>> >>>>>> >>> >>>>> ________________________________ >>> >>>>> To unsubscribe, e-mail: [email protected] >>> >>>>> <mailto:[email protected]> >>> >>>>> For additional commands, e-mail: [email protected] >>> >>>>> <mailto:[email protected]> >>> >>>>> >>> >>>> >>> >>>> -- >>> >>>> Uwe Schindler >>> >>>> Achterdiek 19, 28357 Bremen >>> >>>> https://www.thetaphi.de <https://www.thetaphi.de/> >>> >> >>> >> >>> >> _______________________ >>> >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | >>> >> http://www.opensourceconnections.com >>> >> <http://www.opensourceconnections.com/> | My Free/Busy >>> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >>> >> This e-mail and all contents, including attachments, is considered to be >>> >> Company Confidential unless explicitly stated otherwise, regardless of >>> >> whether attachments are marked as such. >>> >> >>> >>> >>> -- >>> ----------------------------------------------------- >>> Noble Paul >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> <mailto:[email protected]> >>> For additional commands, e-mail: [email protected] >>> <mailto:[email protected]> >>> >> >> _______________________ >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | >> http://www.opensourceconnections.com <http://www.opensourceconnections.com/> >> | My Free/Busy <http://tinyurl.com/eric-cal> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> >> >> This e-mail and all contents, including attachments, is considered to be >> Company Confidential unless explicitly stated otherwise, regardless of >> whether attachments are marked as such. >> > _______________________ Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
