An entire configset, when uploaded via api without security enabled, is marked untrusted. When creating a collection from untrusted configset, certain vulnerable components can't be initialized.
On Fri, 25 Sep, 2020, 9:04 am David Smiley, <dsmi...@apache.org> wrote: > Ishan: can you be more specific please? How is it less secure or harder > to secure than, say, a configSet upload (internally multiple files)? > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Thu, Sep 24, 2020 at 1:53 PM Ishan Chattopadhyaya < > ichattopadhy...@gmail.com> wrote: > >> Single file update capability is a security nightmare. Even if it can be >> done, it should be supported only once authc/authz have been enabled. >> >> On Thu, 24 Sep, 2020, 10:16 pm Tomás Fernández Löbbe, < >> tomasflo...@gmail.com> wrote: >> >>> I won't step in the way of a single file update. I haven't needed it so >>> far though. I usually have the configsets in a Git repo (all the configset >>> together) and I have a simple bash script that essentialy what's described >>> in the docs[1]: Generate the zip on the fly and upload (optionally set the >>> auth too). This can become a problem with big zips, but then again, >>> ZooKeeper limits the size of the configs, so far it hasn't been an issue >>> for me. >>> >>> >>> [1] >>> https://lucene.apache.org/solr/guide/8_6/configsets-api.html#configsets-upload >>> >>> On Thu, Sep 24, 2020 at 7:13 AM Eric Pugh < >>> ep...@opensourceconnections.com> wrote: >>> >>>> It would be great if we had a simple API for updating a file in a >>>> configset that didn’t assume you were just uploading a zip file. >>>> >>>> As an example use case, if you use the Querqy library, you need to >>>> deploy a “rules.txt” file, which in olden days just went on the filesystem >>>> and you would click reload on a core. In the SolrCloud world, we do this >>>> awkward “Let me stick the file in Zookeeper directly, avoiding Solr, and >>>> then do a collection RELOAD” to push out the file everywhere. [1] It >>>> works! But it’s just awkward. >>>> >>>> It’s great to know that I’ll be able to change out this awkward process >>>> using these magic parameters to configSet. Even nicer would be to just >>>> wrap the overwrite=true&cleanup=false and the Zip requirement into >>>> something that sets those. >>>> >>>> >>>> [1] >>>> https://github.com/querqy/chorus/blob/master/smui/conf/smui2solrcloud.sh#L37 >>>> >>>> On Sep 23, 2020, at 11:57 PM, Tomás Fernández Löbbe < >>>> tomasflo...@gmail.com> wrote: >>>> >>>> > Hmmm; seems the configSet API doesn't have an API to update a single >>>> file! I wonder if uploading a configSet to the same name effectively >>>> overwrites newly updated files but does not delete the existing files? >>>> I've been working on this recently. As of 8.7, the UPLOAD command >>>> supports overwriting (before, an UPLOAD on an existing configset name would >>>> fail with BAD_REQUEST) and you can choose to cleanup or not the extra files >>>> with the "cleanup" parameter. >>>> You could upload a single file if you say overwrite=true&cleanup=false, >>>> but it would still need to be in a zip file (and needs to be located in the >>>> right path of the zip, for example, a synonyms file may be in >>>> lang/synonyms_foo.txt or something) >>>> >>>> On Wed, Sep 23, 2020 at 8:10 PM David Smiley <dsmi...@apache.org> >>>> wrote: >>>> >>>>> +1 to deprecate managed resources in lieu of easier to maintain (and >>>>> more flexible) file based GET/PUT into the configset. >>>>> >>>>> > I don't know if 9 is too soon from a deprecation stand point >>>>> >>>>> IMO it's never too soon as long as there is a deprecated release. >>>>> Users take their time upgrading to major versions. >>>>> >>>>> > How much harder are the use-cases currently covered by managed >>>>> resources, if that module was removed? >>>>> >>>>> I believe in practice, users synchronize one-way from their DB to Solr >>>>> if they have dynamic resources like this. This is true where I work. >>>>> Otherwise, they would probably be using Solr as the source of truth, which >>>>> doesn't seem architecturally-sound for most apps IMO. Those users >>>>> (hopefully few) would have to spend some time re-engineering the approach. >>>>> Given one-way sync, the transition here is pretty easy: serialize the >>>>> client-managed data to the right Solr format (stopwords vs synonyms vs >>>>> ...) >>>>> and then a file upload to Solr/ZK and then telling Solr which collections >>>>> to "reload". >>>>> >>>>> Hmmm; seems the configSet API doesn't have an API to update a single >>>>> file! I wonder if uploading a configSet to the same name effectively >>>>> overwrites newly updated files but does not delete the existing files? >>>>> >>>>> ~ David Smiley >>>>> Apache Lucene/Solr Search Developer >>>>> http://www.linkedin.com/in/davidwsmiley >>>>> >>>>> >>>>> On Wed, Sep 23, 2020 at 10:28 AM Timothy Potter <thelabd...@gmail.com> >>>>> wrote: >>>>> >>>>>> I agree we should deprecate the managed resources feature, it was the >>>>>> first thing I was asked to build by LW nearly 7 years ago, before I was a >>>>>> committer. Restlet was already in place and I built on top of that, not >>>>>> sure who introduced it originally (nor do I care). Clearly from the >>>>>> vantage >>>>>> point of looking back, JAX-RS and Jersey won the day with REST in Java >>>>>> but >>>>>> that simply wasn't the case back then. What's important is how we move >>>>>> forward vs. bestowing judgement backed by wisdom of hindsight on >>>>>> decisions >>>>>> made many years ago. >>>>>> >>>>>> In the short term, does Apache have an Artifactory (or similar) where >>>>>> we can host the Restlet dependencies for Github to pull them from? If >>>>>> not, >>>>>> then we can port the code that's using Restlet over to using JAX-RS / >>>>>> Jersey. Personally I'd prefer we remove Managed Resources support from 9 >>>>>> instead of porting the Restlet code but I don't know if 9 is too soon >>>>>> from >>>>>> a deprecation stand point? >>>>>> >>>>>> Tim >>>>>> >>>>>> >>>>>> On Mon, Sep 21, 2020 at 11:33 PM Noble Paul <noble.p...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> We should deprecate that feature and remove restlet dependency >>>>>>> altogether >>>>>>> >>>>>>> On Mon, Sep 21, 2020 at 10:20 PM Joel Bernstein <joels...@gmail.com> >>>>>>> wrote: >>>>>>> > >>>>>>> > Restlet again!!!!!!! >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > Joel Bernstein >>>>>>> > http://joelsolr.blogspot.com/ >>>>>>> > >>>>>>> > >>>>>>> > On Mon, Sep 21, 2020 at 7:18 AM Eric Pugh < >>>>>>> ep...@opensourceconnections.com> wrote: >>>>>>> >> >>>>>>> >> Do we have a community blessed alternative to restlet already? >>>>>>> >> >>>>>>> >> On Sep 20, 2020, at 9:40 AM, Noble Paul <noble.p...@gmail.com> >>>>>>> wrote: >>>>>>> >> >>>>>>> >> Haha. >>>>>>> >> >>>>>>> >> In fact schema APIs don't use restlet. Only the managed resources >>>>>>> use it >>>>>>> >> >>>>>>> >> On Sat, Sep 19, 2020, 3:35 PM Ishan Chattopadhyaya < >>>>>>> ichattopadhy...@gmail.com> wrote: >>>>>>> >>> >>>>>>> >>> If I were talend, I'd immediately start publishing to maven >>>>>>> central. If I were the developer who built the schema APIs, I would >>>>>>> never >>>>>>> have used restlet to begin with. >>>>>>> >>> >>>>>>> >>> On Sat, 19 Sep, 2020, 1:13 am Uwe Schindler, <u...@thetaphi.de> >>>>>>> wrote: >>>>>>> >>>> >>>>>>> >>>> I was thinking the same. Because GitHub does not cache the >>>>>>> downloaded artifacts like our jenkins servers. >>>>>>> >>>> >>>>>>> >>>> It seems to run it in a new VM or container every time, so it >>>>>>> downloads all artifacts. If I were Talend, I'd also block this. >>>>>>> >>>> >>>>>>> >>>> Uwe >>>>>>> >>>> >>>>>>> >>>> Am September 18, 2020 7:32:47 PM UTC schrieb Dawid Weiss < >>>>>>> dawid.we...@gmail.com>: >>>>>>> >>>>> >>>>>>> >>>>> I don't think it's http/https - I believe restlet repository >>>>>>> simply >>>>>>> >>>>> bans github servers because of excessive traffic? These URLs >>>>>>> work for >>>>>>> >>>>> me locally... >>>>>>> >>>>> >>>>>>> >>>>> Dawid >>>>>>> >>>>> >>>>>>> >>>>> On Fri, Sep 18, 2020 at 6:35 PM Christine Poerschke (BLOOMBERG/ >>>>>>> >>>>> LONDON) <cpoersc...@bloomberg.net> wrote: >>>>>>> >>>>>> >>>>>>> >>>>>> >>>>>>> >>>>>> This sounds vaguely familiar. "http works, https does not >>>>>>> work" and https://issues.apache.org/jira/browse/SOLR-13756 possibly >>>>>>> related? >>>>>>> >>>>>> >>>>>>> >>>>>> From: dev@lucene.apache.org At: 09/18/20 10:01:29 >>>>>>> >>>>>> To: dev@lucene.apache.org >>>>>>> >>>>>> Subject: Re: restlet dependencies >>>>>>> >>>>>> >>>>>>> >>>>>> I don't think it is, sadly. >>>>>>> >>>>>> https://repo1.maven.org/maven2/org/restlet >>>>>>> >>>>>> >>>>>>> >>>>>> The link you provided (mvnrepository) aggregates from >>>>>>> several maven >>>>>>> >>>>>> repositories. >>>>>>> >>>>>> >>>>>>> >>>>>> >>>>>>> >>>>>> D. >>>>>>> >>>>>> >>>>>>> >>>>>> On Fri, Sep 18, 2020 at 10:46 AM Ishan Chattopadhyaya >>>>>>> >>>>>> <ichattopadhy...@gmail.com> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Sorry, afk, but I heard (*hearsay*) that restlet is also on >>>>>>> maven central >>>>>>> >>>>>> >>>>>>> >>>>>> these days. Can we confirm and switch to that? Sorry, if >>>>>>> that's not the case. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, 18 Sep, 2020, 1:15 pm Dawid Weiss, < >>>>>>> dawid.we...@gmail.com> wrote: >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> Just FYI: can't get PR builds on github to work recently >>>>>>> because of this: >>>>>>> >>>>>>>> >>>>>>> >>>>>>>>> Could not resolve all files for configuration >>>>>>> >>>>>> >>>>>>> >>>>>> ':solr:core:compileClasspath'. >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> 350 > Could not download org.restlet.ext.servlet-2.4.3.jar >>>>>>> >>>>>>>> (org.restlet.jee:org.restlet.ext.servlet:2.4.3) >>>>>>> >>>>>>>> 351 > Could not get resource >>>>>>> >>>>>>>> >>>>>>> >>>>>> ' >>>>>>> https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >>>>>>> >>>>>> tlet.ext.servlet-2.4.3.jar'. >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> 352 > Could not GET >>>>>>> >>>>>>>> >>>>>>> >>>>>> ' >>>>>>> https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >>>>>>> >>>>>> tlet.ext.servlet-2.4.3.jar'. >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> 353 > Connection reset >>>>>>> >>>>>>>> 354 > Could not download org.restlet-2.4.3.jar >>>>>>> >>>>>>>> (org.restlet.jee:org.restlet:2.4.3) >>>>>>> >>>>>>>> 355 > Could not get resource >>>>>>> >>>>>>>> >>>>>>> >>>>>> ' >>>>>>> https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.j >>>>>>> >>>>>> ar'. >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> 356 > Could not GET >>>>>>> >>>>>>>> >>>>>>> >>>>>> ' >>>>>>> https://maven.restlet.talend.com/org/restlet/jee/org.restlet/2.4.3/org.restlet- >>>>>>> >>>>>> 2.4.3.jar'. >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> 357 > Connection reset >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> D. >>>>>>> >>>>>>>> ________________________________ >>>>>>> >>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>>>> >>>>>>>> For additional commands, e-mail: >>>>>>> dev-h...@lucene.apache.org >>>>>>> >>>>>>>> >>>>>>> >>>>>> ________________________________ >>>>>>> >>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>>>> >>>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>>>> >>>>>> >>>>>>> >>>>>> >>>>>>> >>>>> ________________________________ >>>>>>> >>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>>>> >>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>>>> >>>>> >>>>>>> >>>> >>>>>>> >>>> -- >>>>>>> >>>> Uwe Schindler >>>>>>> >>>> Achterdiek 19, 28357 Bremen >>>>>>> >>>> https://www.thetaphi.de >>>>>>> >> >>>>>>> >> >>>>>>> >> _______________________ >>>>>>> >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | >>>>>>> 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy >>>>>>> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >>>>>>> >> This e-mail and all contents, including attachments, is >>>>>>> considered to be Company Confidential unless explicitly stated >>>>>>> otherwise, >>>>>>> regardless of whether attachments are marked as such. >>>>>>> >> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> ----------------------------------------------------- >>>>>>> Noble Paul >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>>>> >>>>>>> >>>> _______________________ >>>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467 >>>> | http://www.opensourceconnections.com | My Free/Busy >>>> <http://tinyurl.com/eric-cal> >>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >>>> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> >>>> This e-mail and all contents, including attachments, is considered to >>>> be Company Confidential unless explicitly stated otherwise, regardless >>>> of whether attachments are marked as such. >>>> >>>>
