Ok, good enough for me, thanks for feedback David and Noble, proceeding with the cherry-picks to 8x
Tim On Tue, Oct 6, 2020 at 8:25 PM David Smiley <dsmi...@apache.org> wrote: > No strong opinion from me. I think the back-compat concern is minor. > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Tue, Oct 6, 2020 at 5:42 PM Noble Paul <noble.p...@gmail.com> wrote: > >> I think we should call that out in the changes.txt and make the changes >> right away. >> >> On Wed, Oct 7, 2020, 8:20 AM Timothy Potter <thelabd...@gmail.com> wrote: >> >>> Just want to close the loop on this restlet issue. I've removed the >>> restlet dependency in master (e879a52291ef7dcd0514e7419d811b6ff800bcce) but >>> have not backported that to 8.x yet. >>> >>> I'm hesitant to backport because I had to change public function >>> signatures on ManagedResource, e.g. >>> https://github.com/apache/lucene-solr/commit/e879a52291ef7dcd0514e7419d811b6ff800bcce#diff-faaf5cd2bfe038f81ca4c0cb1986b42dR355 >>> ... >>> >>> So technically, this could break upgrades for environments with >>> extensions to those classes; practically speaking, I think the risk of that >>> is low, but not sure it is worth it? From what I understand, the restlet >>> maven issue was mostly affecting master / gradle builds and the ant/ivy >>> builds in 8.x weren't affected as much? >>> >>> It's an easy back-port from master to 8.x if that's what we want to do, >>> but wanted to raise awareness that it will break public function signatures >>> on classes that may have extensions in the wild ;-) >>> >>> ~ Tim >>> >>> On Thu, Oct 1, 2020 at 8:36 AM Timothy Potter <thelabd...@gmail.com> >>> wrote: >>> >>>> Awesome guys, thanks for the pointers ... am cooking up a PR (for >>>> master) for this today >>>> >>>> On Thu, Oct 1, 2020 at 2:22 AM Noble Paul <noble.p...@gmail.com> wrote: >>>> >>>>> The annotation ( >>>>> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/api/EndPoint.java >>>>> ) >>>>> supports wild cards and templated paths >>>>> >>>>> On Thu, Oct 1, 2020 at 5:28 PM Ishan Chattopadhyaya >>>>> <ichattopadhy...@gmail.com> wrote: >>>>> > >>>>> > > But when I suggested porting the code that uses restlet to JAX-RS >>>>> / Jersey, Ishan said >>>>> > > that wasn't necessary and is already supported with some >>>>> Annotations ... I have no idea >>>>> > > what that means and need more info about what is already in place. >>>>> > >>>>> > I was mainly referring to the @Endpoint annotations. It is available >>>>> for V2 APIs today (and I think it should be fine to use for anything we >>>>> build now onwards, including managed resources V2). >>>>> > It is possible to make it work with V1, but that will require some >>>>> work. >>>>> > >>>>> > On Thu, Oct 1, 2020 at 12:50 PM Ishan Chattopadhyaya < >>>>> ichattopadhy...@gmail.com> wrote: >>>>> >> >>>>> >> @Tim >>>>> >> >>>>> >> Please check ClusterAPI or ZookeeperReadAPI etc. >>>>> >> Recently used it in Yasa: >>>>> https://github.com/yasa-org/yasa/blob/master/yasa-solr-plugin/src/main/java/io/github/kezhenxu94/YasaHandler.java >>>>> >> >>>>> >> On Thu, Oct 1, 2020 at 10:46 AM Noble Paul <noble.p...@gmail.com> >>>>> wrote: >>>>> >>> >>>>> >>> @Tim Potter >>>>> >>> >>>>> >>> I tried several times to get rid of the restlet dependency & keep >>>>> the >>>>> >>> functionality as is. I failed miserably. I'm not saying this to >>>>> >>> discourage anyone who wants to give a try. Just letting you know >>>>> that >>>>> >>> it is not as easy as it may sound >>>>> >>> >>>>> >>> On Thu, Oct 1, 2020 at 2:42 AM Houston Putman < >>>>> houstonput...@gmail.com> wrote: >>>>> >>> > >>>>> >>> > +1 to Tomas' proposal. Created SOLR-14907 to track the effort. >>>>> >>> > >>>>> >>> > - Houston >>>>> >>> > >>>>> >>> > On Wed, Sep 30, 2020 at 12:26 PM Tomás Fernández Löbbe < >>>>> tomasflo...@gmail.com> wrote: >>>>> >>> >> >>>>> >>> >> > Let's support the single file upload feature >>>>> >>> >> +1, but let this behave exactly as a zip file with a single >>>>> file in it (regarding trusted/untrusted). We just need to change the >>>>> configset handler to be able to handle non-zip files, and have a way to >>>>> "locate" that file inside the configset (in case it needs to go somewhere >>>>> other than the root). >>>>> >>> >> >>>>> >>> >> On Wed, Sep 30, 2020 at 8:45 AM Eric Pugh < >>>>> ep...@opensourceconnections.com> wrote: >>>>> >>> >>> >>>>> >>> >>> I think that me in “violent agreement” with you. Let’s >>>>> understand the Annotations approach that we have, or pick something that >>>>> is >>>>> commonly used like JAX-RS / Jersey. >>>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>> On Sep 30, 2020, at 11:41 AM, Timothy Potter < >>>>> thelabd...@gmail.com> wrote: >>>>> >>> >>> >>>>> >>> >>> I'm sorry, I don't understand what you mean by "make it a >>>>> single pattern (the annotations?)" Eric? >>>>> >>> >>> >>>>> >>> >>> To me, the pattern is well established in the Java world: >>>>> JAX-RS (with Jersey as the underlying impl. which has nice integration >>>>> with >>>>> Jetty). But when I suggested porting the code that uses restlet to JAX-RS >>>>> / >>>>> Jersey, Ishan said that wasn't necessary and is already supported with >>>>> some >>>>> Annotations ... I have no idea what that means and need more info about >>>>> what is already in place. Short of that, replacing restlet with JAX-RS / >>>>> Jersey looks like a trivial amount of work to me (and I'm happy to take it >>>>> on). >>>>> >>> >>> >>>>> >>> >>> Tim >>>>> >>> >>> >>>>> >>> >>> On Wed, Sep 30, 2020 at 9:36 AM Eric Pugh < >>>>> ep...@opensourceconnections.com> wrote: >>>>> >>> >>>> >>>>> >>> >>>> The use case of “I want to update something via a API” is I >>>>> think pretty common, and it would be nice to make it a single pattern (the >>>>> annotations?) with lots of examples/developer docs for the next person. >>>>> >>> >>>> >>>>> >>> >>>> >>>>> >>> >>>> >>>>> >>> >>>> On Sep 30, 2020, at 11:04 AM, Timothy Potter < >>>>> thelabd...@gmail.com> wrote: >>>>> >>> >>>> >>>>> >>> >>>> I started looking into removing Managed Resources in master >>>>> and wanted to mention that the LTR contrib also relies on this framework >>>>> (ManagedModelStore and ManagedFeatureStore, see: >>>>> https://lucene.apache.org/solr/guide/8_6/learning-to-rank.html#uploading-a-model). >>>>> I only mention this b/c it's been said several times in this thread that >>>>> nobody uses this feature and it's only for editing config/schema like >>>>> synonyms. Afaik, LTR is a broadly used feature of Solr so now I'm not so >>>>> bullish on removing the ability to manage dynamic resources using a REST >>>>> like API. I agree that changing resources like the synonym set could be >>>>> replaced with configSet updates but I don't see how to replace the RESTful >>>>> model / feature store API w/o something like Managed Resources? >>>>> >>> >>>> >>>>> >>> >>>> From where I sit, I think we should just remove the use of >>>>> restlet in the implementation but keep the API for Solr 9 (master). >>>>> >>> >>>> >>>>> >>> >>>> @Ishan ~ you mentioned there is a way to get REST API like >>>>> behavior w/o using JAX-RS / Jersey ... something about annotations? Can >>>>> you >>>>> point me to some example code of how that is done please? >>>>> >>> >>>> >>>>> >>> >>>> Cheers, >>>>> >>> >>>> Tim >>>>> >>> >>>> >>>>> >>> >>>> On Wed, Sep 30, 2020 at 8:29 AM David Smiley < >>>>> dsmi...@apache.org> wrote: >>>>> >>> >>>>> >>>>> >>> >>>>> These resources are fundamentally a part of the configSet >>>>> and can (in general) affect query results and thus flushing caches (via a >>>>> reload) is appropriate. >>>>> >>> >>>>> >>>>> >>> >>>>> ~ David Smiley >>>>> >>> >>>>> Apache Lucene/Solr Search Developer >>>>> >>> >>>>> http://www.linkedin.com/in/davidwsmiley >>>>> >>> >>>>> >>>>> >>> >>>>> >>>>> >>> >>>>> On Wed, Sep 30, 2020 at 9:06 AM Noble Paul < >>>>> noble.p...@gmail.com> wrote: >>>>> >>> >>>>>> >>>>> >>> >>>>>> Well, I believe we should have a mechanism to upload a >>>>> single file to >>>>> >>> >>>>>> a configset. >>>>> >>> >>>>>> >>>>> >>> >>>>>> > A single file configset upload would require the user to >>>>> reload the collection, so it isn't better than managed resources. >>>>> >>> >>>>>> >>>>> >>> >>>>>> This is not true >>>>> >>> >>>>>> >>>>> >>> >>>>>> Only config/schema file changes result in core reload. >>>>> >>> >>>>>> >>>>> >>> >>>>>> On Wed, Sep 30, 2020 at 10:23 PM David Smiley < >>>>> dsmi...@apache.org> wrote: >>>>> >>> >>>>>> > >>>>> >>> >>>>>> > Definitely don't remove in 8.x! >>>>> >>> >>>>>> > >>>>> >>> >>>>>> > > A single file configset upload would require the user >>>>> to reload the collection, so it isn't better than managed resources. >>>>> >>> >>>>>> > >>>>> >>> >>>>>> > Do you view that as a substantial point in favor of >>>>> managed-resources? I view that as a trivial matter, and one I prefer to >>>>> automagic and potentially premature reload if there are additional edits >>>>> to >>>>> be done (e.g. query-elevation or other word lists). >>>>> >>> >>>>>> > >>>>> >>> >>>>>> > ~ David Smiley >>>>> >>> >>>>>> > Apache Lucene/Solr Search Developer >>>>> >>> >>>>>> > http://www.linkedin.com/in/davidwsmiley >>>>> >>> >>>>>> > >>>>> >>> >>>>>> > >>>>> >>> >>>>>> > On Wed, Sep 30, 2020 at 5:46 AM Ishan Chattopadhyaya < >>>>> ichattopadhy...@gmail.com> wrote: >>>>> >>> >>>>>> >> >>>>> >>> >>>>>> >> > * Nobody knows how it works. It's unsupported >>>>> >>> >>>>>> >> It is supported and documented: >>>>> https://lucene.apache.org/solr/guide/8_6/managed-resources.html >>>>> >>> >>>>>> >> >>>>> >>> >>>>>> >> > * RESTlet dependency >>>>> >>> >>>>>> >> > * Cannot be secured using standard permissions >>>>> >>> >>>>>> >> > * It's extremely complex for the functionality it >>>>> offers. >>>>> >>> >>>>>> >> >>>>> >>> >>>>>> >> I agree. Whatever alternative we build should address >>>>> these, before we consider removing managed resources. >>>>> >>> >>>>>> >> >>>>> >>> >>>>>> >> On Wed, Sep 30, 2020 at 2:52 PM Ishan Chattopadhyaya < >>>>> ichattopadhy...@gmail.com> wrote: >>>>> >>> >>>>>> >>> >>>>> >>> >>>>>> >>> The managed resources is the only reasonable way to >>>>> upload synonyms on the fly for users today. A single file configset upload >>>>> would require the user to reload the collection, so it isn't better than >>>>> managed resources. I would not recommend we remove the functionality >>>>> without first building a suitable alternative. I agree that the feature >>>>> isn't built using proper framework or proper APIs, but it is a feature >>>>> that >>>>> works well. >>>>> >>> >>>>>> >>> >>>>> >>> >>>>>> >>> Usually, I support throwing features out even without >>>>> existence of an alternative, but I do that for non essential features. In >>>>> my mind, ability to manage synonyms elegantly is an essential feature for >>>>> a >>>>> search engine. >>>>> >>> >>>>>> >>> >>>>> >>> >>>>>> >>> On Wed, 30 Sep, 2020, 2:44 pm Uwe Schindler, < >>>>> u...@thetaphi.de> wrote: >>>>> >>> >>>>>> >>>> >>>>> >>> >>>>>> >>>> Please don't do this. >>>>> >>> >>>>>> >>>> >>>>> >>> >>>>>> >>>> In short: remove restlet stuff from master. Pull >>>>> requests on master are executed with Gradle on GitHub hardware. >>>>> >>> >>>>>> >>>> >>>>> >>> >>>>>> >>>> Ivy stuff in 8.x is built in more or less persistent >>>>> servers and there is no issue. >>>>> >>> >>>>>> >>>> >>>>> >>> >>>>>> >>>> What's the problem? >>>>> >>> >>>>>> >>>> >>>>> >>> >>>>>> >>>> Uwe >>>>> >>> >>>>>> >>>> >>>>> >>> >>>>>> >>>> Am September 30, 2020 8:59:06 AM UTC schrieb Ishan >>>>> Chattopadhyaya <ichattopadhy...@gmail.com>: >>>>> >>> >>>>>> >>>>> >>>>> >>> >>>>>> >>>>> Can we discuss this with ASF and get an exception for >>>>> this? >>>>> >>> >>>>>> >>>>> >>>>> >>> >>>>>> >>>>> On Wed, 30 Sep, 2020, 11:57 am Dawid Weiss, < >>>>> dawid.we...@gmail.com> wrote: >>>>> >>> >>>>>> >>>>>> >>>>> >>> >>>>>> >>>>>> We can't have or redistribute binaries in ASL >>>>> sources - that's my understanding. >>>>> >>> >>>>>> >>>>>> >>>>> >>> >>>>>> >>>>>> Dawid >>>>> >>> >>>>>> >>>>>> >>>>> >>> >>>>>> >>>>>> On Tue, Sep 29, 2020 at 10:02 PM Ishan Chattopadhyaya >>>>> >>> >>>>>> >>>>>> <ichattopadhy...@gmail.com> wrote: >>>>> >>> >>>>>> >>>>>> > >>>>> >>> >>>>>> >>>>>> > Can we pull in the jar inside our codebase? >>>>> >>> >>>>>> >>>>>> > >>>>> >>> >>>>>> >>>>>> > On Wed, 30 Sep, 2020, 1:19 am Dawid Weiss, < >>>>> dawid.we...@gmail.com> wrote: >>>>> >>> >>>>>> >>>>>> >> >>>>> >>> >>>>>> >>>>>> >> >>>>> >>> >>>>>> >>>>>> >> We can upgrade if it doesn't break anything... >>>>> which I can't guarantee. ;) >>>>> >>> >>>>>> >>>>>> >> >>>>> >>> >>>>>> >>>>>> >> Dawid >>>>> >>> >>>>>> >>>>>> >>>>> >>> >>>>>> >>>>>> >>>>> --------------------------------------------------------------------- >>>>> >>> >>>>>> >>>>>> To unsubscribe, e-mail: >>>>> dev-unsubscr...@lucene.apache.org >>>>> >>> >>>>>> >>>>>> For additional commands, e-mail: >>>>> dev-h...@lucene.apache.org >>>>> >>> >>>>>> >>>>>> >>>>> >>> >>>>>> >>>> >>>>> >>> >>>>>> >>>> -- >>>>> >>> >>>>>> >>>> Uwe Schindler >>>>> >>> >>>>>> >>>> Achterdiek 19, 28357 Bremen >>>>> >>> >>>>>> >>>> https://www.thetaphi.de >>>>> >>> >>>>>> >>>>> >>> >>>>>> >>>>> >>> >>>>>> >>>>> >>> >>>>>> -- >>>>> >>> >>>>>> ----------------------------------------------------- >>>>> >>> >>>>>> Noble Paul >>>>> >>> >>>>>> >>>>> >>> >>>>>> >>>>> --------------------------------------------------------------------- >>>>> >>> >>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>> >>> >>>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>> >>> >>>>>> >>>>> >>> >>>> >>>>> >>> >>>> _______________________ >>>>> >>> >>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | >>>>> 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy >>>>> >>> >>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >>>>> >>> >>>> This e-mail and all contents, including attachments, is >>>>> considered to be Company Confidential unless explicitly stated otherwise, >>>>> regardless of whether attachments are marked as such. >>>>> >>> >>>> >>>>> >>> >>> >>>>> >>> >>> _______________________ >>>>> >>> >>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | >>>>> 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy >>>>> >>> >>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >>>>> >>> >>> This e-mail and all contents, including attachments, is >>>>> considered to be Company Confidential unless explicitly stated otherwise, >>>>> regardless of whether attachments are marked as such. >>>>> >>> >>> >>>>> >>> >>>>> >>> >>>>> >>> -- >>>>> >>> ----------------------------------------------------- >>>>> >>> Noble Paul >>>>> >>> >>>>> >>> >>>>> --------------------------------------------------------------------- >>>>> >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>> >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>> >>> >>>>> >>>>> >>>>> -- >>>>> ----------------------------------------------------- >>>>> Noble Paul >>>>> >>>>
