SOLR-5287 has some discussion. The changes were backed out due to security reasons. Anyway, Eric, Tomas & David, let us not hijack this thread on restlet and discuss something totally unrelated.
On Fri, 25 Sep, 2020, 9:20 am Ishan Chattopadhyaya, < [email protected]> wrote: > An entire configset, when uploaded via api without security enabled, is > marked untrusted. When creating a collection from untrusted configset, > certain vulnerable components can't be initialized. > > On Fri, 25 Sep, 2020, 9:04 am David Smiley, <[email protected]> wrote: > >> Ishan: can you be more specific please? How is it less secure or harder >> to secure than, say, a configSet upload (internally multiple files)? >> >> ~ David Smiley >> Apache Lucene/Solr Search Developer >> http://www.linkedin.com/in/davidwsmiley >> >> >> On Thu, Sep 24, 2020 at 1:53 PM Ishan Chattopadhyaya < >> [email protected]> wrote: >> >>> Single file update capability is a security nightmare. Even if it can be >>> done, it should be supported only once authc/authz have been enabled. >>> >>> On Thu, 24 Sep, 2020, 10:16 pm Tomás Fernández Löbbe, < >>> [email protected]> wrote: >>> >>>> I won't step in the way of a single file update. I haven't needed it so >>>> far though. I usually have the configsets in a Git repo (all the configset >>>> together) and I have a simple bash script that essentialy what's described >>>> in the docs[1]: Generate the zip on the fly and upload (optionally set the >>>> auth too). This can become a problem with big zips, but then again, >>>> ZooKeeper limits the size of the configs, so far it hasn't been an issue >>>> for me. >>>> >>>> >>>> [1] >>>> https://lucene.apache.org/solr/guide/8_6/configsets-api.html#configsets-upload >>>> >>>> On Thu, Sep 24, 2020 at 7:13 AM Eric Pugh < >>>> [email protected]> wrote: >>>> >>>>> It would be great if we had a simple API for updating a file in a >>>>> configset that didn’t assume you were just uploading a zip file. >>>>> >>>>> As an example use case, if you use the Querqy library, you need to >>>>> deploy a “rules.txt” file, which in olden days just went on the filesystem >>>>> and you would click reload on a core. In the SolrCloud world, we do this >>>>> awkward “Let me stick the file in Zookeeper directly, avoiding Solr, and >>>>> then do a collection RELOAD” to push out the file everywhere. [1] It >>>>> works! But it’s just awkward. >>>>> >>>>> It’s great to know that I’ll be able to change out this awkward >>>>> process using these magic parameters to configSet. Even nicer would be to >>>>> just wrap the overwrite=true&cleanup=false and the Zip requirement into >>>>> something that sets those. >>>>> >>>>> >>>>> [1] >>>>> https://github.com/querqy/chorus/blob/master/smui/conf/smui2solrcloud.sh#L37 >>>>> >>>>> On Sep 23, 2020, at 11:57 PM, Tomás Fernández Löbbe < >>>>> [email protected]> wrote: >>>>> >>>>> > Hmmm; seems the configSet API doesn't have an API to update a single >>>>> file! I wonder if uploading a configSet to the same name effectively >>>>> overwrites newly updated files but does not delete the existing files? >>>>> I've been working on this recently. As of 8.7, the UPLOAD command >>>>> supports overwriting (before, an UPLOAD on an existing configset name >>>>> would >>>>> fail with BAD_REQUEST) and you can choose to cleanup or not the extra >>>>> files >>>>> with the "cleanup" parameter. >>>>> You could upload a single file if you say >>>>> overwrite=true&cleanup=false, but it would still need to be in a zip file >>>>> (and needs to be located in the right path of the zip, for example, a >>>>> synonyms file may be in lang/synonyms_foo.txt or something) >>>>> >>>>> On Wed, Sep 23, 2020 at 8:10 PM David Smiley <[email protected]> >>>>> wrote: >>>>> >>>>>> +1 to deprecate managed resources in lieu of easier to maintain (and >>>>>> more flexible) file based GET/PUT into the configset. >>>>>> >>>>>> > I don't know if 9 is too soon from a deprecation stand point >>>>>> >>>>>> IMO it's never too soon as long as there is a deprecated release. >>>>>> Users take their time upgrading to major versions. >>>>>> >>>>>> > How much harder are the use-cases currently covered by managed >>>>>> resources, if that module was removed? >>>>>> >>>>>> I believe in practice, users synchronize one-way from their DB to >>>>>> Solr if they have dynamic resources like this. This is true where I >>>>>> work. >>>>>> Otherwise, they would probably be using Solr as the source of truth, >>>>>> which >>>>>> doesn't seem architecturally-sound for most apps IMO. Those users >>>>>> (hopefully few) would have to spend some time re-engineering the >>>>>> approach. >>>>>> Given one-way sync, the transition here is pretty easy: serialize the >>>>>> client-managed data to the right Solr format (stopwords vs synonyms vs >>>>>> ...) >>>>>> and then a file upload to Solr/ZK and then telling Solr which collections >>>>>> to "reload". >>>>>> >>>>>> Hmmm; seems the configSet API doesn't have an API to update a single >>>>>> file! I wonder if uploading a configSet to the same name effectively >>>>>> overwrites newly updated files but does not delete the existing files? >>>>>> >>>>>> ~ David Smiley >>>>>> Apache Lucene/Solr Search Developer >>>>>> http://www.linkedin.com/in/davidwsmiley >>>>>> >>>>>> >>>>>> On Wed, Sep 23, 2020 at 10:28 AM Timothy Potter <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> I agree we should deprecate the managed resources feature, it was >>>>>>> the first thing I was asked to build by LW nearly 7 years ago, before I >>>>>>> was >>>>>>> a committer. Restlet was already in place and I built on top of that, >>>>>>> not >>>>>>> sure who introduced it originally (nor do I care). Clearly from the >>>>>>> vantage >>>>>>> point of looking back, JAX-RS and Jersey won the day with REST in Java >>>>>>> but >>>>>>> that simply wasn't the case back then. What's important is how we move >>>>>>> forward vs. bestowing judgement backed by wisdom of hindsight on >>>>>>> decisions >>>>>>> made many years ago. >>>>>>> >>>>>>> In the short term, does Apache have an Artifactory (or similar) >>>>>>> where we can host the Restlet dependencies for Github to pull them >>>>>>> from? If >>>>>>> not, then we can port the code that's using Restlet over to using >>>>>>> JAX-RS / >>>>>>> Jersey. Personally I'd prefer we remove Managed Resources support from 9 >>>>>>> instead of porting the Restlet code but I don't know if 9 is too soon >>>>>>> from >>>>>>> a deprecation stand point? >>>>>>> >>>>>>> Tim >>>>>>> >>>>>>> >>>>>>> On Mon, Sep 21, 2020 at 11:33 PM Noble Paul <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> We should deprecate that feature and remove restlet dependency >>>>>>>> altogether >>>>>>>> >>>>>>>> On Mon, Sep 21, 2020 at 10:20 PM Joel Bernstein <[email protected]> >>>>>>>> wrote: >>>>>>>> > >>>>>>>> > Restlet again!!!!!!! >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > Joel Bernstein >>>>>>>> > http://joelsolr.blogspot.com/ >>>>>>>> > >>>>>>>> > >>>>>>>> > On Mon, Sep 21, 2020 at 7:18 AM Eric Pugh < >>>>>>>> [email protected]> wrote: >>>>>>>> >> >>>>>>>> >> Do we have a community blessed alternative to restlet already? >>>>>>>> >> >>>>>>>> >> On Sep 20, 2020, at 9:40 AM, Noble Paul <[email protected]> >>>>>>>> wrote: >>>>>>>> >> >>>>>>>> >> Haha. >>>>>>>> >> >>>>>>>> >> In fact schema APIs don't use restlet. Only the managed >>>>>>>> resources use it >>>>>>>> >> >>>>>>>> >> On Sat, Sep 19, 2020, 3:35 PM Ishan Chattopadhyaya < >>>>>>>> [email protected]> wrote: >>>>>>>> >>> >>>>>>>> >>> If I were talend, I'd immediately start publishing to maven >>>>>>>> central. If I were the developer who built the schema APIs, I would >>>>>>>> never >>>>>>>> have used restlet to begin with. >>>>>>>> >>> >>>>>>>> >>> On Sat, 19 Sep, 2020, 1:13 am Uwe Schindler, <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>> >>>>>>>> >>>> I was thinking the same. Because GitHub does not cache the >>>>>>>> downloaded artifacts like our jenkins servers. >>>>>>>> >>>> >>>>>>>> >>>> It seems to run it in a new VM or container every time, so it >>>>>>>> downloads all artifacts. If I were Talend, I'd also block this. >>>>>>>> >>>> >>>>>>>> >>>> Uwe >>>>>>>> >>>> >>>>>>>> >>>> Am September 18, 2020 7:32:47 PM UTC schrieb Dawid Weiss < >>>>>>>> [email protected]>: >>>>>>>> >>>>> >>>>>>>> >>>>> I don't think it's http/https - I believe restlet repository >>>>>>>> simply >>>>>>>> >>>>> bans github servers because of excessive traffic? These URLs >>>>>>>> work for >>>>>>>> >>>>> me locally... >>>>>>>> >>>>> >>>>>>>> >>>>> Dawid >>>>>>>> >>>>> >>>>>>>> >>>>> On Fri, Sep 18, 2020 at 6:35 PM Christine Poerschke >>>>>>>> (BLOOMBERG/ >>>>>>>> >>>>> LONDON) <[email protected]> wrote: >>>>>>>> >>>>>> >>>>>>>> >>>>>> >>>>>>>> >>>>>> This sounds vaguely familiar. "http works, https does not >>>>>>>> work" and https://issues.apache.org/jira/browse/SOLR-13756 >>>>>>>> possibly related? >>>>>>>> >>>>>> >>>>>>>> >>>>>> From: [email protected] At: 09/18/20 10:01:29 >>>>>>>> >>>>>> To: [email protected] >>>>>>>> >>>>>> Subject: Re: restlet dependencies >>>>>>>> >>>>>> >>>>>>>> >>>>>> I don't think it is, sadly. >>>>>>>> >>>>>> https://repo1.maven.org/maven2/org/restlet >>>>>>>> >>>>>> >>>>>>>> >>>>>> The link you provided (mvnrepository) aggregates from >>>>>>>> several maven >>>>>>>> >>>>>> repositories. >>>>>>>> >>>>>> >>>>>>>> >>>>>> >>>>>>>> >>>>>> D. >>>>>>>> >>>>>> >>>>>>>> >>>>>> On Fri, Sep 18, 2020 at 10:46 AM Ishan Chattopadhyaya >>>>>>>> >>>>>> <[email protected]> wrote: >>>>>>>> >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> >>>>>>> Sorry, afk, but I heard (*hearsay*) that restlet is also >>>>>>>> on maven central >>>>>>>> >>>>>> >>>>>>>> >>>>>> these days. Can we confirm and switch to that? Sorry, if >>>>>>>> that's not the case. >>>>>>>> >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> >>>>>>> On Fri, 18 Sep, 2020, 1:15 pm Dawid Weiss, < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Just FYI: can't get PR builds on github to work recently >>>>>>>> because of this: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Could not resolve all files for configuration >>>>>>>> >>>>>> >>>>>>>> >>>>>> ':solr:core:compileClasspath'. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 350 > Could not download org.restlet.ext.servlet-2.4.3.jar >>>>>>>> >>>>>>>> (org.restlet.jee:org.restlet.ext.servlet:2.4.3) >>>>>>>> >>>>>>>> 351 > Could not get resource >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> ' >>>>>>>> https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >>>>>>>> >>>>>> tlet.ext.servlet-2.4.3.jar'. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 352 > Could not GET >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> ' >>>>>>>> https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >>>>>>>> >>>>>> tlet.ext.servlet-2.4.3.jar'. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 353 > Connection reset >>>>>>>> >>>>>>>> 354 > Could not download org.restlet-2.4.3.jar >>>>>>>> >>>>>>>> (org.restlet.jee:org.restlet:2.4.3) >>>>>>>> >>>>>>>> 355 > Could not get resource >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> ' >>>>>>>> https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.j >>>>>>>> >>>>>> ar'. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 356 > Could not GET >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> ' >>>>>>>> https://maven.restlet.talend.com/org/restlet/jee/org.restlet/2.4.3/org.restlet- >>>>>>>> >>>>>> 2.4.3.jar'. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 357 > Connection reset >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> D. >>>>>>>> >>>>>>>> ________________________________ >>>>>>>> >>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>> >>>>>>>> For additional commands, e-mail: >>>>>>>> [email protected] >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> ________________________________ >>>>>>>> >>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>> >>>>>> For additional commands, e-mail: [email protected] >>>>>>>> >>>>>> >>>>>>>> >>>>>> >>>>>>>> >>>>> ________________________________ >>>>>>>> >>>>> To unsubscribe, e-mail: [email protected] >>>>>>>> >>>>> For additional commands, e-mail: [email protected] >>>>>>>> >>>>> >>>>>>>> >>>> >>>>>>>> >>>> -- >>>>>>>> >>>> Uwe Schindler >>>>>>>> >>>> Achterdiek 19, 28357 Bremen >>>>>>>> >>>> https://www.thetaphi.de >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> _______________________ >>>>>>>> >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | >>>>>>>> 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy >>>>>>>> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >>>>>>>> >> This e-mail and all contents, including attachments, is >>>>>>>> considered to be Company Confidential unless explicitly stated >>>>>>>> otherwise, >>>>>>>> regardless of whether attachments are marked as such. >>>>>>>> >> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> ----------------------------------------------------- >>>>>>>> Noble Paul >>>>>>>> >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>> >>>>>>>> >>>>> _______________________ >>>>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467 >>>>> | http://www.opensourceconnections.com | My Free/Busy >>>>> <http://tinyurl.com/eric-cal> >>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >>>>> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> >>>>> This e-mail and all contents, including attachments, is considered to >>>>> be Company Confidential unless explicitly stated otherwise, regardless >>>>> of whether attachments are marked as such. >>>>> >>>>>
