One of the other problems I have with maven, besides where will the snapshots go, is the local development workflow. If I were to work on a feature that needs changes to both Lucene and Solr, I would need to make Lucene changes in a separate git repo, push them to local Maven, have solr build against that and so on. This is considerably more complicated, to my mind, than working on a single repo (with the other repo being a submodule) and single project inside the IDE, which git submodule can allow us to do.
On Mon, 3 May, 2021, 8:27 pm Houston Putman, <[email protected]> wrote: > I agree, Cassandra. The nightlies are a very good option that is very easy > to get started with. > > On Mon, May 3, 2021 at 9:55 AM Cassandra Targett <[email protected]> > wrote: > >> I’m sort of surprised no one has mentioned the >> https://nightlies.apache.org/ server, which could be used for this >> purpose. Jenkins can push to it, and nothing gets deleted until we decide >> to delete it (or overwrite it). Solr Operator builds go there as do drafts >> of the Ref Guide pre-publication (in different directories now, but we’ll >> fix that eventually). >> >> Cassandra >> On May 3, 2021, 9:15 AM -0500, David Smiley <[email protected]>, wrote: >> >> RE "an external system like Maven" -- we're merely talking about adding >> another JAR repo to the list of repos we already have. Heck, for this >> limited purpose, we could even use http://home.apache.org/~dsmiley/ >> Note that the Lucene project already uses home dirs of some users for >> benchmark data. If we were talking about adding a Maven *build* then I >> would totally appreciate your concern. >> >> I volunteer to use my space for this purpose. >> >> ~ David Smiley >> Apache Lucene/Solr Search Developer >> http://www.linkedin.com/in/davidwsmiley >> >> >> On Sun, May 2, 2021 at 12:17 AM Ishan Chattopadhyaya < >> [email protected]> wrote: >> >>> > Let’s not complicate things. >>> By bringing in external systems like Maven, I think we're complicating >>> things even though a straight forward way (git submodules) exists. >>> >>> On Sat, May 1, 2021 at 5:00 PM Jan Høydahl <[email protected]> >>> wrote: >>> >>>> Sub modules is for organizing internal repos, not for pulling in >>>> external deps. Let’s not complicate things. And once we switch main to 10.x >>>> we’d need to use pure jar dependencies in branch_9x to depend upon actually >>>> released and voted Lucene binaries. >>>> >>>> We need some tooling to smoothly work with bleeding edge Lucene >>>> including local snapshot builds with not yet pushed changes to Lucene. I’d >>>> rather put in some work to establish such tooling or procedures. >>>> >>>> Jan Høydahl >>>> >>>> 1. mai 2021 kl. 08:41 skrev Ishan Chattopadhyaya < >>>> [email protected]>: >>>> >>>> >>>> > What submodules don't solve is releases - if you're >>>> > on a >>>> particular unreleased Lucene version then releasing >>>> > Solr would still mean you need some kind of >>>> > "public" pinned Lucene release for the >>>> > Mavenworld. >>>> >>>> >>>> Can we then update the submodule to point to the release tag or sha >>>> that Lucene got released from? >>>> >>>> On Sat, 1 May, 2021, 11:45 am Dawid Weiss, <[email protected]> >>>> wrote: >>>> >>>>> > Other than literally adding the git submodule, would we do anything >>>>> else to modify the gradle build so that or do we (and Jenkinsfile) have to >>>>> manually do a step to install Lucene before proceeding? >>>>> >>>>> Technically you add a submodule and then make a composite build from >>>>> Solr side. I can provide a PR that does it if you wish... This is >>>>> simple. What submodules don't solve is releases - if you're on a >>>>> particular unreleased Lucene version then releasing Solr would still >>>>> mean you need some kind of "public" pinned Lucene release for the >>>>> Maven world. If you distribute the entire thing as binaries you can >>>>> just publish a snapshot of Lucene code, of course. >>>>> >>>>> > I think adding git submodule means that we have to add back in all >>>>> the build code for it. >>>>> >>>>> No. The submodule is just a reference to a particular repository (and >>>>> commit) - the build code and remains nearly identical as it was with >>>>> Lucene. Composite builds in gradle are quite nice in that they're >>>>> (almost) transparent compared to dependency references. >>>>> >>>>> > At which point, I'd rather just copy and commit the >>>>> > code they have so we don't have to learn another git system. >>>>> >>>>> It's not really a different git system - it's a mechanism of >>>>> interacting with multiple repositories at once. Yes, it does add some >>>>> additional complexity but I think sooner or later you'll have to >>>>> interact with it anyway - many people favor monorepos these days and >>>>> this is a common way to assemble them from fragmented >>>>> sub-repositories. >>>>> >>>>> > I've >>>>> > heard submodules don't play nice with Jenkins, but don't have any >>>>> > direct experience with them. >>>>> >>>>> Some tools may require an additional flag to initialize and clone >>>>> sub-modules on the initial pull. Or manually. >>>>> >>>>> git submodule init >>>>> git submodule update --recursive. >>>>> >>>>> To summarize - I am not pushing for submodules, I do think they're a >>>>> viable alternative but pinned Maven references are simpler. The >>>>> problem with maven references is that we'd need a long(er) term maven >>>>> repository where such pinned references would live (without being >>>>> wiped out). Perhaps infra can help here and it'd solve the problem. >>>>> >>>>> Dawid >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>>> >>>>>
