Ok, so we're removing from 9 (I can do that) but what should we do about the upcoming 8.7 release? I'm fine with deprecating, but usually an alternative approach is suggested with the deprecation notice. For cloud mode, we can suggest our hypothetical users of this MR API to use the config set API but what about standalone? I can do the work in 8.7 too but not really sure how to proceed on the standalone question ...
On Fri, Sep 25, 2020 at 12:20 AM Jan Høydahl <jan....@cominvent.com> wrote: > Yea, managed resources is one of the features I’ll not miss. It seemed > smart back then, but for some reason I don’t think people use it that much, > and I agree it makes more sense for the user to be aware of the configset > and update the configset. > > I’d appreciate a upload one file API. Ishan, you could say that you can > upload a single file to a config set only if auth is enabled OR if the > configset is already untrusted. > > Jan > > 25. sep. 2020 kl. 06:02 skrev Ishan Chattopadhyaya < > ichattopadhy...@gmail.com>: > > SOLR-5287 has some discussion. The changes were backed out due to security > reasons. Anyway, Eric, Tomas & David, let us not hijack this thread on > restlet and discuss something totally unrelated. > > On Fri, 25 Sep, 2020, 9:20 am Ishan Chattopadhyaya, < > ichattopadhy...@gmail.com> wrote: > >> An entire configset, when uploaded via api without security enabled, is >> marked untrusted. When creating a collection from untrusted configset, >> certain vulnerable components can't be initialized. >> >> On Fri, 25 Sep, 2020, 9:04 am David Smiley, <dsmi...@apache.org> wrote: >> >>> Ishan: can you be more specific please? How is it less secure or harder >>> to secure than, say, a configSet upload (internally multiple files)? >>> >>> ~ David Smiley >>> Apache Lucene/Solr Search Developer >>> http://www.linkedin.com/in/davidwsmiley >>> >>> >>> On Thu, Sep 24, 2020 at 1:53 PM Ishan Chattopadhyaya < >>> ichattopadhy...@gmail.com> wrote: >>> >>>> Single file update capability is a security nightmare. Even if it can >>>> be done, it should be supported only once authc/authz have been enabled. >>>> >>>> On Thu, 24 Sep, 2020, 10:16 pm Tomás Fernández Löbbe, < >>>> tomasflo...@gmail.com> wrote: >>>> >>>>> I won't step in the way of a single file update. I haven't needed it >>>>> so far though. I usually have the configsets in a Git repo (all the >>>>> configset together) and I have a simple bash script that essentialy what's >>>>> described in the docs[1]: Generate the zip on the fly and upload >>>>> (optionally set the auth too). This can become a problem with big zips, >>>>> but >>>>> then again, ZooKeeper limits the size of the configs, so far it hasn't >>>>> been >>>>> an issue for me. >>>>> >>>>> >>>>> [1] >>>>> https://lucene.apache.org/solr/guide/8_6/configsets-api.html#configsets-upload >>>>> >>>>> On Thu, Sep 24, 2020 at 7:13 AM Eric Pugh < >>>>> ep...@opensourceconnections.com> wrote: >>>>> >>>>>> It would be great if we had a simple API for updating a file in a >>>>>> configset that didn’t assume you were just uploading a zip file. >>>>>> >>>>>> As an example use case, if you use the Querqy library, you need to >>>>>> deploy a “rules.txt” file, which in olden days just went on the >>>>>> filesystem >>>>>> and you would click reload on a core. In the SolrCloud world, we do >>>>>> this >>>>>> awkward “Let me stick the file in Zookeeper directly, avoiding Solr, and >>>>>> then do a collection RELOAD” to push out the file everywhere. [1] It >>>>>> works! But it’s just awkward. >>>>>> >>>>>> It’s great to know that I’ll be able to change out this awkward >>>>>> process using these magic parameters to configSet. Even nicer would be >>>>>> to >>>>>> just wrap the overwrite=true&cleanup=false and the Zip requirement into >>>>>> something that sets those. >>>>>> >>>>>> >>>>>> [1] >>>>>> https://github.com/querqy/chorus/blob/master/smui/conf/smui2solrcloud.sh#L37 >>>>>> >>>>>> On Sep 23, 2020, at 11:57 PM, Tomás Fernández Löbbe < >>>>>> tomasflo...@gmail.com> wrote: >>>>>> >>>>>> > Hmmm; seems the configSet API doesn't have an API to update a >>>>>> single file! I wonder if uploading a configSet to the same name >>>>>> effectively overwrites newly updated files but does not delete the >>>>>> existing >>>>>> files? >>>>>> I've been working on this recently. As of 8.7, the UPLOAD command >>>>>> supports overwriting (before, an UPLOAD on an existing configset name >>>>>> would >>>>>> fail with BAD_REQUEST) and you can choose to cleanup or not the extra >>>>>> files >>>>>> with the "cleanup" parameter. >>>>>> You could upload a single file if you say >>>>>> overwrite=true&cleanup=false, but it would still need to be in a zip file >>>>>> (and needs to be located in the right path of the zip, for example, a >>>>>> synonyms file may be in lang/synonyms_foo.txt or something) >>>>>> >>>>>> On Wed, Sep 23, 2020 at 8:10 PM David Smiley <dsmi...@apache.org> >>>>>> wrote: >>>>>> >>>>>>> +1 to deprecate managed resources in lieu of easier to maintain (and >>>>>>> more flexible) file based GET/PUT into the configset. >>>>>>> >>>>>>> > I don't know if 9 is too soon from a deprecation stand point >>>>>>> >>>>>>> IMO it's never too soon as long as there is a deprecated release. >>>>>>> Users take their time upgrading to major versions. >>>>>>> >>>>>>> > How much harder are the use-cases currently covered by managed >>>>>>> resources, if that module was removed? >>>>>>> >>>>>>> I believe in practice, users synchronize one-way from their DB to >>>>>>> Solr if they have dynamic resources like this. This is true where I >>>>>>> work. >>>>>>> Otherwise, they would probably be using Solr as the source of truth, >>>>>>> which >>>>>>> doesn't seem architecturally-sound for most apps IMO. Those users >>>>>>> (hopefully few) would have to spend some time re-engineering the >>>>>>> approach. >>>>>>> Given one-way sync, the transition here is pretty easy: serialize the >>>>>>> client-managed data to the right Solr format (stopwords vs synonyms vs >>>>>>> ...) >>>>>>> and then a file upload to Solr/ZK and then telling Solr which >>>>>>> collections >>>>>>> to "reload". >>>>>>> >>>>>>> Hmmm; seems the configSet API doesn't have an API to update a single >>>>>>> file! I wonder if uploading a configSet to the same name effectively >>>>>>> overwrites newly updated files but does not delete the existing files? >>>>>>> >>>>>>> ~ David Smiley >>>>>>> Apache Lucene/Solr Search Developer >>>>>>> http://www.linkedin.com/in/davidwsmiley >>>>>>> >>>>>>> >>>>>>> On Wed, Sep 23, 2020 at 10:28 AM Timothy Potter < >>>>>>> thelabd...@gmail.com> wrote: >>>>>>> >>>>>>>> I agree we should deprecate the managed resources feature, it was >>>>>>>> the first thing I was asked to build by LW nearly 7 years ago, before >>>>>>>> I was >>>>>>>> a committer. Restlet was already in place and I built on top of that, >>>>>>>> not >>>>>>>> sure who introduced it originally (nor do I care). Clearly from the >>>>>>>> vantage >>>>>>>> point of looking back, JAX-RS and Jersey won the day with REST in Java >>>>>>>> but >>>>>>>> that simply wasn't the case back then. What's important is how we move >>>>>>>> forward vs. bestowing judgement backed by wisdom of hindsight on >>>>>>>> decisions >>>>>>>> made many years ago. >>>>>>>> >>>>>>>> In the short term, does Apache have an Artifactory (or similar) >>>>>>>> where we can host the Restlet dependencies for Github to pull them >>>>>>>> from? If >>>>>>>> not, then we can port the code that's using Restlet over to using >>>>>>>> JAX-RS / >>>>>>>> Jersey. Personally I'd prefer we remove Managed Resources support from >>>>>>>> 9 >>>>>>>> instead of porting the Restlet code but I don't know if 9 is too soon >>>>>>>> from >>>>>>>> a deprecation stand point? >>>>>>>> >>>>>>>> Tim >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Sep 21, 2020 at 11:33 PM Noble Paul <noble.p...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> We should deprecate that feature and remove restlet dependency >>>>>>>>> altogether >>>>>>>>> >>>>>>>>> On Mon, Sep 21, 2020 at 10:20 PM Joel Bernstein < >>>>>>>>> joels...@gmail.com> wrote: >>>>>>>>> > >>>>>>>>> > Restlet again!!!!!!! >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > Joel Bernstein >>>>>>>>> > http://joelsolr.blogspot.com/ >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > On Mon, Sep 21, 2020 at 7:18 AM Eric Pugh < >>>>>>>>> ep...@opensourceconnections.com> wrote: >>>>>>>>> >> >>>>>>>>> >> Do we have a community blessed alternative to restlet already? >>>>>>>>> >> >>>>>>>>> >> On Sep 20, 2020, at 9:40 AM, Noble Paul <noble.p...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >> >>>>>>>>> >> Haha. >>>>>>>>> >> >>>>>>>>> >> In fact schema APIs don't use restlet. Only the managed >>>>>>>>> resources use it >>>>>>>>> >> >>>>>>>>> >> On Sat, Sep 19, 2020, 3:35 PM Ishan Chattopadhyaya < >>>>>>>>> ichattopadhy...@gmail.com> wrote: >>>>>>>>> >>> >>>>>>>>> >>> If I were talend, I'd immediately start publishing to maven >>>>>>>>> central. If I were the developer who built the schema APIs, I would >>>>>>>>> never >>>>>>>>> have used restlet to begin with. >>>>>>>>> >>> >>>>>>>>> >>> On Sat, 19 Sep, 2020, 1:13 am Uwe Schindler, <u...@thetaphi.de> >>>>>>>>> wrote: >>>>>>>>> >>>> >>>>>>>>> >>>> I was thinking the same. Because GitHub does not cache the >>>>>>>>> downloaded artifacts like our jenkins servers. >>>>>>>>> >>>> >>>>>>>>> >>>> It seems to run it in a new VM or container every time, so it >>>>>>>>> downloads all artifacts. If I were Talend, I'd also block this. >>>>>>>>> >>>> >>>>>>>>> >>>> Uwe >>>>>>>>> >>>> >>>>>>>>> >>>> Am September 18, 2020 7:32:47 PM UTC schrieb Dawid Weiss < >>>>>>>>> dawid.we...@gmail.com>: >>>>>>>>> >>>>> >>>>>>>>> >>>>> I don't think it's http/https - I believe restlet repository >>>>>>>>> simply >>>>>>>>> >>>>> bans github servers because of excessive traffic? These URLs >>>>>>>>> work for >>>>>>>>> >>>>> me locally... >>>>>>>>> >>>>> >>>>>>>>> >>>>> Dawid >>>>>>>>> >>>>> >>>>>>>>> >>>>> On Fri, Sep 18, 2020 at 6:35 PM Christine Poerschke >>>>>>>>> (BLOOMBERG/ >>>>>>>>> >>>>> LONDON) <cpoersc...@bloomberg.net> wrote: >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> This sounds vaguely familiar. "http works, https does not >>>>>>>>> work" and https://issues.apache.org/jira/browse/SOLR-13756 >>>>>>>>> possibly related? >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> From: dev@lucene.apache.org At: 09/18/20 10:01:29 >>>>>>>>> >>>>>> To: dev@lucene.apache.org >>>>>>>>> >>>>>> Subject: Re: restlet dependencies >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> I don't think it is, sadly. >>>>>>>>> >>>>>> https://repo1.maven.org/maven2/org/restlet >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> The link you provided (mvnrepository) aggregates from >>>>>>>>> several maven >>>>>>>>> >>>>>> repositories. >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> D. >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> On Fri, Sep 18, 2020 at 10:46 AM Ishan Chattopadhyaya >>>>>>>>> >>>>>> <ichattopadhy...@gmail.com> wrote: >>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>> Sorry, afk, but I heard (*hearsay*) that restlet is also >>>>>>>>> on maven central >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> these days. Can we confirm and switch to that? Sorry, if >>>>>>>>> that's not the case. >>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>> On Fri, 18 Sep, 2020, 1:15 pm Dawid Weiss, < >>>>>>>>> dawid.we...@gmail.com> wrote: >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> Just FYI: can't get PR builds on github to work recently >>>>>>>>> because of this: >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Could not resolve all files for configuration >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> ':solr:core:compileClasspath'. >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> 350 > Could not download >>>>>>>>> org.restlet.ext.servlet-2.4.3.jar >>>>>>>>> >>>>>>>> (org.restlet.jee:org.restlet.ext.servlet:2.4.3) >>>>>>>>> >>>>>>>> 351 > Could not get resource >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>> ' >>>>>>>>> https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >>>>>>>>> >>>>>> tlet.ext.servlet-2.4.3.jar'. >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> 352 > Could not GET >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>> ' >>>>>>>>> https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >>>>>>>>> >>>>>> tlet.ext.servlet-2.4.3.jar'. >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> 353 > Connection reset >>>>>>>>> >>>>>>>> 354 > Could not download org.restlet-2.4.3.jar >>>>>>>>> >>>>>>>> (org.restlet.jee:org.restlet:2.4.3) >>>>>>>>> >>>>>>>> 355 > Could not get resource >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>> ' >>>>>>>>> https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.j >>>>>>>>> >>>>>> ar'. >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> 356 > Could not GET >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>> ' >>>>>>>>> https://maven.restlet.talend.com/org/restlet/jee/org.restlet/2.4.3/org.restlet- >>>>>>>>> >>>>>> 2.4.3.jar'. >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> 357 > Connection reset >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> D. >>>>>>>>> >>>>>>>> ________________________________ >>>>>>>>> >>>>>>>> To unsubscribe, e-mail: >>>>>>>>> dev-unsubscr...@lucene.apache.org >>>>>>>>> >>>>>>>> For additional commands, e-mail: >>>>>>>>> dev-h...@lucene.apache.org >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>> ________________________________ >>>>>>>>> >>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>>>>>> >>>>>> For additional commands, e-mail: >>>>>>>>> dev-h...@lucene.apache.org >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> >>>>>>>>> >>>>> ________________________________ >>>>>>>>> >>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>>>>>> >>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>>>>>> >>>>> >>>>>>>>> >>>> >>>>>>>>> >>>> -- >>>>>>>>> >>>> Uwe Schindler >>>>>>>>> >>>> Achterdiek 19, 28357 Bremen >>>>>>>>> >>>> https://www.thetaphi.de >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> _______________________ >>>>>>>>> >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | >>>>>>>>> 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy >>>>>>>>> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >>>>>>>>> >> This e-mail and all contents, including attachments, is >>>>>>>>> considered to be Company Confidential unless explicitly stated >>>>>>>>> otherwise, >>>>>>>>> regardless of whether attachments are marked as such. >>>>>>>>> >> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> ----------------------------------------------------- >>>>>>>>> Noble Paul >>>>>>>>> >>>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>>>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>>>>>> >>>>>>>>> >>>>>> _______________________ >>>>>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | >>>>>> 434.466.1467 >>>>>> | http://www.opensourceconnections.com | My Free/Busy >>>>>> <http://tinyurl.com/eric-cal> >>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >>>>>> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> >>>>>> This e-mail and all contents, including attachments, is considered to >>>>>> be Company Confidential unless explicitly stated otherwise, regardless >>>>>> of whether attachments are marked as such. >>>>>> >>>>>> >
