Hi, Sure! I will play a little bit around. If needed, I can add some property to the lucene build to locate a "local repo" (I think we have that already). I just have to figure out, if some local repo, copied to some Webserver works well as a full M2 repository, especially how to handle multiple intermediary releases in it.
Uwe Am May 7, 2021 1:47:45 PM UTC schrieb David Smiley <[email protected]>: >Sounds great Uwe! Can you do it please? > >~ David Smiley >Apache Lucene/Solr Search Developer >http://www.linkedin.com/in/davidwsmiley > > >On Fri, May 7, 2021 at 1:55 AM Uwe Schindler <[email protected]> wrote: > >> Hi Cassandra. Sure, We can create a job on ASF Jenkins that is only >> manually triggered. >> It builds the maven artifacts into a local file system repo. As post >build >> job we publish the output folder on nighties repo like the ref guide. >> >> The version number is passed as -DreleaseVersion system property >(e.g. >> populated by Jenkins from the build number), like we do on artifacts >builds >> on jenkins anyways. It's no snapshot anymore, but like an half >official >> build with some custom version internal to our process. >> >> If you want to upgrade Solr's Lucene, just press "build" on jenkins, >wait >> for it to finish, and finally copy-paste the version number from >Jenkins >> build description into gradle dependency version file. >> >> Uwe >> >> Uwe >> >> Am May 6, 2021 8:13:00 PM UTC schrieb Cassandra Targett < >> [email protected]>: >>> >>> It could be a manual process if that’s what you think is easiest and >it >>> could still be on nightlies. It’s just a curl command to push >something up: >>> https://nightlies.apache.org/authoring.html. If it is a manual >process, >>> I think it would be better for it to be hosted in a place any of us >could >>> access, in case you’re busy/something happens to you and we need to >pin a >>> different snapshot. >>> >>> I’m not sure I see a reason why Solr couldn’t have our own Jenkins >job >>> that built Lucene once a week (or on demand, or whatever cadence >that >>> works) and pushes the snapshot to nightlies - we don’t have to use >one of >>> Lucene’s jobs, do we? >>> >>> I feel like we could also cascade a new Lucene snapshot build into a >>> “Lucene validation” Solr build that verifies everything is OK with a >new >>> snapshot before updating the hosted snapshot on nightlies (although, >I >>> don’t actually know how to do that, it just seems like something >that could >>> be done). >>> >>> Cassandra >>> On May 6, 2021, 10:13 AM -0500, David Smiley <[email protected]>, >wrote: >>> >>> I asked on the Lucene dev list about possible use of the Nightlies >>> server. We don't want to pollute nightlies on a regular basis (I >think); >>> this would be an on-demand thing. As such, I'm not sure a CI server >is the >>> best way to approach this vs a manual script publishing to an ASF >personal >>> Home space. >>> >>> ~ David Smiley >>> Apache Lucene/Solr Search Developer >>> http://www.linkedin.com/in/davidwsmiley >>> >>> >>> On Mon, May 3, 2021 at 10:55 AM Cassandra Targett ><[email protected]> >>> wrote: >>> >>>> I’m sort of surprised no one has mentioned the >>>> https://nightlies.apache.org/ server, which could be used for this >>>> purpose. Jenkins can push to it, and nothing gets deleted until we >decide >>>> to delete it (or overwrite it). Solr Operator builds go there as do >drafts >>>> of the Ref Guide pre-publication (in different directories now, but >we’ll >>>> fix that eventually). >>>> >>>> Cassandra >>>> On May 3, 2021, 9:15 AM -0500, David Smiley <[email protected]>, >wrote: >>>> >>>> RE "an external system like Maven" -- we're merely talking about >adding >>>> another JAR repo to the list of repos we already have. Heck, for >this >>>> limited purpose, we could even use http://home.apache.org/~dsmiley/ >>>> Note that the Lucene project already uses home dirs of some users >for >>>> benchmark data. If we were talking about adding a Maven *build* >then I >>>> would totally appreciate your concern. >>>> >>>> I volunteer to use my space for this purpose. >>>> >>>> ~ David Smiley >>>> Apache Lucene/Solr Search Developer >>>> http://www.linkedin.com/in/davidwsmiley >>>> >>>> >>>> On Sun, May 2, 2021 at 12:17 AM Ishan Chattopadhyaya < >>>> [email protected]> wrote: >>>> >>>>> > Let’s not complicate things. >>>>> By bringing in external systems like Maven, I think we're >complicating >>>>> things even though a straight forward way (git submodules) exists. >>>>> >>>>> On Sat, May 1, 2021 at 5:00 PM Jan Høydahl ><[email protected]> >>>>> wrote: >>>>> >>>>>> Sub modules is for organizing internal repos, not for pulling in >>>>>> external deps. Let’s not complicate things. And once we switch >main to 10.x >>>>>> we’d need to use pure jar dependencies in branch_9x to depend >upon actually >>>>>> released and voted Lucene binaries. >>>>>> >>>>>> We need some tooling to smoothly work with bleeding edge Lucene >>>>>> including local snapshot builds with not yet pushed changes to >Lucene. I’d >>>>>> rather put in some work to establish such tooling or procedures. >>>>>> >>>>>> Jan Høydahl >>>>>> >>>>>> 1. mai 2021 kl. 08:41 skrev Ishan Chattopadhyaya < >>>>>> [email protected]>: >>>>>> >>>>>> >>>>>> > What submodules don't solve is releases - if you're >>>>>> > on a >>>>>> particular unreleased Lucene version then releasing >>>>>> > Solr would still mean you need some kind of >>>>>> > "public" pinned Lucene release for the >>>>>> > Mavenworld. >>>>>> >>>>>> >>>>>> Can we then update the submodule to point to the release tag or >sha >>>>>> that Lucene got released from? >>>>>> >>>>>> On Sat, 1 May, 2021, 11:45 am Dawid Weiss, ><[email protected]> >>>>>> wrote: >>>>>> >>>>>>> > Other than literally adding the git submodule, would we do >anything >>>>>>> else to modify the gradle build so that or do we (and >Jenkinsfile) have to >>>>>>> manually do a step to install Lucene before proceeding? >>>>>>> >>>>>>> Technically you add a submodule and then make a composite build >from >>>>>>> Solr side. I can provide a PR that does it if you wish... This >is >>>>>>> simple. What submodules don't solve is releases - if you're on a >>>>>>> particular unreleased Lucene version then releasing Solr would >still >>>>>>> mean you need some kind of "public" pinned Lucene release for >the >>>>>>> Maven world. If you distribute the entire thing as binaries you >can >>>>>>> just publish a snapshot of Lucene code, of course. >>>>>>> >>>>>>> > I think adding git submodule means that we have to add back in >all >>>>>>> the build code for it. >>>>>>> >>>>>>> No. The submodule is just a reference to a particular repository >(and >>>>>>> commit) - the build code and remains nearly identical as it was >with >>>>>>> Lucene. Composite builds in gradle are quite nice in that >they're >>>>>>> (almost) transparent compared to dependency references. >>>>>>> >>>>>>> > At which point, I'd rather just copy and commit the >>>>>>> > code they have so we don't have to learn another git system. >>>>>>> >>>>>>> It's not really a different git system - it's a mechanism of >>>>>>> interacting with multiple repositories at once. Yes, it does add >some >>>>>>> additional complexity but I think sooner or later you'll have to >>>>>>> interact with it anyway - many people favor monorepos these days >and >>>>>>> this is a common way to assemble them from fragmented >>>>>>> sub-repositories. >>>>>>> >>>>>>> > I've >>>>>>> > heard submodules don't play nice with Jenkins, but don't have >any >>>>>>> > direct experience with them. >>>>>>> >>>>>>> Some tools may require an additional flag to initialize and >clone >>>>>>> sub-modules on the initial pull. Or manually. >>>>>>> >>>>>>> git submodule init >>>>>>> git submodule update --recursive. >>>>>>> >>>>>>> To summarize - I am not pushing for submodules, I do think >they're a >>>>>>> viable alternative but pinned Maven references are simpler. The >>>>>>> problem with maven references is that we'd need a long(er) term >maven >>>>>>> repository where such pinned references would live (without >being >>>>>>> wiped out). Perhaps infra can help here and it'd solve the >problem. >>>>>>> >>>>>>> Dawid >>>>>>> >>>>>>> >--------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>> For additional commands, e-mail: [email protected] >>>>>>> >>>>>>> >> -- >> Uwe Schindler >> Achterdiek 19, 28357 Bremen >> https://www.thetaphi.de >> -- Uwe Schindler Achterdiek 19, 28357 Bremen https://www.thetaphi.de
