Re: Separate git repo(s) for Solr modules

[Jan Høydahl]

Folks

1. This is the dev@lucene list, please move discussions over to dev@solr
2. The thread about simultaneous lucene/solr dev is in dev@solr with subject 
"Proposal to pin the Lucene snapshot version on main"
3. If you want to discuss whether Solr should be a monorepo or multiple repos, 
start a new thread at dev@solr

Jan

> 4. mai 2021 kl. 20:15 skrev Dawid Weiss :
> 
> A submodule is nothing more than a checkout of a different repository
> (at a given commit). It's not a fork, unless you clone that repository
> and place it somewhere else.
> 
> In any case, I agree Solr should stick to official Lucene releases. If
> there is a need to have a "sticky" intermediate tagged release of
> Lucene for internal (development branch) purposes, Cassandra showed
> what seems like an adequate place to host it until an official Lucene
> release is out.
> 
> If you're working on both projects at once and wish to modify code in both,
> IDEs in many cases help you out by substituting code/classpaths from multiple
> sources. I know Eclipse can do it, I'm pretty sure you could try to do
> the same with
> IntelliJ.
> 
> Dawid
> 
> On Tue, May 4, 2021 at 4:40 PM Atri Sharma  wrote:
>> 
>> I am a bit confused -- if Lucene was to become a sub module of Solr, are we 
>> essentially forking Lucene?
>> 
>> I am in agreement with Ilan and Houston -- Solr should be depending on 
>> Lucene only as a dependency.
>> 
>> 
>> On Tue, 4 May 2021, 19:52 Noble Paul,  wrote:
>>> 
>>> @Ilan Ginzburg
>>> 
>>> I don't think the project split is counter productive because we have 
>>> lucene as a sub module. Solr does not use lucene like a simple library. 
>>> It's an integral part of Solr
>>> 
>>> 
>>> On Tue, May 4, 2021 at 10:02 PM Ilan Ginzburg  wrote:
 
 As with any dependency on any project, you update the dependency project 
 first then consume the updated dependency in Solr.
 
 If the idea is to be able to modify Lucene and Solr in parallel, then the 
 project split is counterproductive.
 
 From the Solr perspective, Lucene and Zookerper are really two “similar”
 dependencies and IMO we should think about them in that way.
 
 Ilan
 
 On Tue 4 May 2021 at 09:45, Noble Paul  wrote:
> 
> @Houston
> 
> So, Are you suggesting we should not do that ?
> 
> On Tue, May 4, 2021 at 2:35 PM Houston Putman  
> wrote:
>> 
>> In the future we wont be able to “work on both at the same time”, once 
>> Lucene 9 is cut. Why not pull that bandaid now?
>> 
>> On Mon, May 3, 2021 at 11:32 PM Noble Paul  wrote:
>>> 
>>> I'm still struggling to understand the workflow when I'm working on a 
>>> feature that spans lucene and solr.
>>> 
>>> I'm yet to see an argument against sub-modules
>>> 
>>> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski  
>>> wrote:
 
> Shoving such a component into lucene-solr repo makes no sense, given 
> its branching strategy is based on master / branch_8x
 
 I get how this could cause issues - I def hadn't thought much about
 multi-version support and branching.  But I don't think moving plugins
 to a separate repo solves that problem for us.  If our first class
 plugins are set up to have different release "lines" that don't line
 up with major Solr versions, it's only a matter of time before two of
 those plugins have branch requirements that conflict.  Unless I'm
 missing something here?
 
> I'd prefer that a module only leave our "contribs" area when the 
> concerns/limitations become real.  Doing it prematurely could lead to 
> atrophy of the module
 
 +1 to David's comment.   I def hadn't considered the branching-scheme
 issues that Ishan brought up in his last reply, and they seem like
 valid concerns to me.  But the risk and the downsides of "atrophy" are
 serious enough that I'd vote to not risk them until this starts to
 cause us issues in practice.  Even if, for now, that means we won't be
 able to build a single plugin jar that supports (e.g.) 3 major Solr
 versions.  IMO that's a much smaller loss.
 
 On Tue, Feb 16, 2021 at 9:40 AM David Smiley  
 wrote:
> 
> On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh 
>  wrote:
>> 
>> Testing across multiple versions is always very difficult ;-).  I 
>> recently saw this very interesting approach to using our Dockerized 
>> Solr’s to test a component against a number of previous versions of 
>> Solr.  https://github.com/querqy/querqy/pull/147. I’m hopeful it 
>> could be a model for other packages to adopt.
> 
> 
> Thanks for the link to that Querqy PR.  That is *very* similar to 
> what I do at work (minus multi-version testing), and also 

Re: Separate git repo(s) for Solr modules

[Dawid Weiss]

A submodule is nothing more than a checkout of a different repository
(at a given commit). It's not a fork, unless you clone that repository
and place it somewhere else.

In any case, I agree Solr should stick to official Lucene releases. If
there is a need to have a "sticky" intermediate tagged release of
Lucene for internal (development branch) purposes, Cassandra showed
what seems like an adequate place to host it until an official Lucene
release is out.

If you're working on both projects at once and wish to modify code in both,
IDEs in many cases help you out by substituting code/classpaths from multiple
sources. I know Eclipse can do it, I'm pretty sure you could try to do
the same with
IntelliJ.

Dawid

On Tue, May 4, 2021 at 4:40 PM Atri Sharma  wrote:
>
> I am a bit confused -- if Lucene was to become a sub module of Solr, are we 
> essentially forking Lucene?
>
> I am in agreement with Ilan and Houston -- Solr should be depending on Lucene 
> only as a dependency.
>
>
> On Tue, 4 May 2021, 19:52 Noble Paul,  wrote:
>>
>> @Ilan Ginzburg
>>
>> I don't think the project split is counter productive because we have lucene 
>> as a sub module. Solr does not use lucene like a simple library. It's an 
>> integral part of Solr
>>
>>
>> On Tue, May 4, 2021 at 10:02 PM Ilan Ginzburg  wrote:
>>>
>>> As with any dependency on any project, you update the dependency project 
>>> first then consume the updated dependency in Solr.
>>>
>>> If the idea is to be able to modify Lucene and Solr in parallel, then the 
>>> project split is counterproductive.
>>>
>>> From the Solr perspective, Lucene and Zookerper are really two “similar”
>>> dependencies and IMO we should think about them in that way.
>>>
>>> Ilan
>>>
>>> On Tue 4 May 2021 at 09:45, Noble Paul  wrote:

 @Houston

 So, Are you suggesting we should not do that ?

 On Tue, May 4, 2021 at 2:35 PM Houston Putman  
 wrote:
>
> In the future we wont be able to “work on both at the same time”, once 
> Lucene 9 is cut. Why not pull that bandaid now?
>
> On Mon, May 3, 2021 at 11:32 PM Noble Paul  wrote:
>>
>> I'm still struggling to understand the workflow when I'm working on a 
>> feature that spans lucene and solr.
>>
>> I'm yet to see an argument against sub-modules
>>
>> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski  
>> wrote:
>>>
>>> > Shoving such a component into lucene-solr repo makes no sense, given 
>>> > its branching strategy is based on master / branch_8x
>>>
>>> I get how this could cause issues - I def hadn't thought much about
>>> multi-version support and branching.  But I don't think moving plugins
>>> to a separate repo solves that problem for us.  If our first class
>>> plugins are set up to have different release "lines" that don't line
>>> up with major Solr versions, it's only a matter of time before two of
>>> those plugins have branch requirements that conflict.  Unless I'm
>>> missing something here?
>>>
>>> > I'd prefer that a module only leave our "contribs" area when the 
>>> > concerns/limitations become real.  Doing it prematurely could lead to 
>>> > atrophy of the module
>>>
>>> +1 to David's comment.   I def hadn't considered the branching-scheme
>>> issues that Ishan brought up in his last reply, and they seem like
>>> valid concerns to me.  But the risk and the downsides of "atrophy" are
>>> serious enough that I'd vote to not risk them until this starts to
>>> cause us issues in practice.  Even if, for now, that means we won't be
>>> able to build a single plugin jar that supports (e.g.) 3 major Solr
>>> versions.  IMO that's a much smaller loss.
>>>
>>> On Tue, Feb 16, 2021 at 9:40 AM David Smiley  wrote:
>>> >
>>> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh 
>>> >  wrote:
>>> >>
>>> >> Testing across multiple versions is always very difficult ;-).  I 
>>> >> recently saw this very interesting approach to using our Dockerized 
>>> >> Solr’s to test a component against a number of previous versions of 
>>> >> Solr.  https://github.com/querqy/querqy/pull/147. I’m hopeful it 
>>> >> could be a model for other packages to adopt.
>>> >
>>> >
>>> > Thanks for the link to that Querqy PR.  That is *very* similar to 
>>> > what I do at work (minus multi-version testing), and also similar to 
>>> > how I test multiple versions in one of my pet projects by using a CI 
>>> > build matrix of a configurable dependency.  I didn't know 
>>> > Testcontainer.org has its own Solr module -- 
>>> > https://www.testcontainers.org/modules/solr/ but we implemented that 
>>> > ourselves; not hard.
>>> >
>>> >>
>>> >> Trying to maintain across multiple versions is kind of a Sisyphean 
>>> >> task, and I don’t think plays to open source communities strengths.  
>>> >>  It’s 

Re: Separate git repo(s) for Solr modules

[Atri Sharma]

I am a bit confused -- if Lucene was to become a sub module of Solr, are we
essentially forking Lucene?

I am in agreement with Ilan and Houston -- Solr should be depending on
Lucene only as a dependency.


On Tue, 4 May 2021, 19:52 Noble Paul,  wrote:

> @Ilan Ginzburg 
>
> I don't think the project split is counter productive because we have
> lucene as a sub module. Solr does not use lucene like a simple library.
> It's an integral part of Solr
>
>
> On Tue, May 4, 2021 at 10:02 PM Ilan Ginzburg  wrote:
>
>> As with any dependency on any project, you update the dependency project
>> first then consume the updated dependency in Solr.
>>
>> If the idea is to be able to modify Lucene and Solr in parallel, then the
>> project split is counterproductive.
>>
>> From the Solr perspective, Lucene and Zookerper are really two “similar”
>> dependencies and IMO we should think about them in that way.
>>
>> Ilan
>>
>> On Tue 4 May 2021 at 09:45, Noble Paul  wrote:
>>
>>> @Houston
>>>
>>> So, Are you suggesting we should not do that ?
>>>
>>> On Tue, May 4, 2021 at 2:35 PM Houston Putman 
>>> wrote:
>>>
 In the future we wont be able to “work on both at the same time”, once
 Lucene 9 is cut. Why not pull that bandaid now?

 On Mon, May 3, 2021 at 11:32 PM Noble Paul 
 wrote:

> I'm still struggling to understand the workflow when I'm working on a
> feature that spans lucene and solr.
>
> I'm yet to see an argument against sub-modules
>
> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski 
> wrote:
>
>> > Shoving such a component into lucene-solr repo makes no sense,
>> given its branching strategy is based on master / branch_8x
>>
>> I get how this could cause issues - I def hadn't thought much about
>> multi-version support and branching.  But I don't think moving plugins
>> to a separate repo solves that problem for us.  If our first class
>> plugins are set up to have different release "lines" that don't line
>> up with major Solr versions, it's only a matter of time before two of
>> those plugins have branch requirements that conflict.  Unless I'm
>> missing something here?
>>
>> > I'd prefer that a module only leave our "contribs" area when the
>> concerns/limitations become real.  Doing it prematurely could lead to
>> atrophy of the module
>>
>> +1 to David's comment.   I def hadn't considered the branching-scheme
>> issues that Ishan brought up in his last reply, and they seem like
>> valid concerns to me.  But the risk and the downsides of "atrophy" are
>> serious enough that I'd vote to not risk them until this starts to
>> cause us issues in practice.  Even if, for now, that means we won't be
>> able to build a single plugin jar that supports (e.g.) 3 major Solr
>> versions.  IMO that's a much smaller loss.
>>
>> On Tue, Feb 16, 2021 at 9:40 AM David Smiley 
>> wrote:
>> >
>> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
>> ep...@opensourceconnections.com> wrote:
>> >>
>> >> Testing across multiple versions is always very difficult ;-).  I
>> recently saw this very interesting approach to using our Dockerized 
>> Solr’s
>> to test a component against a number of previous versions of Solr.
>> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
>> model for other packages to adopt.
>> >
>> >
>> > Thanks for the link to that Querqy PR.  That is *very* similar to
>> what I do at work (minus multi-version testing), and also similar to how 
>> I
>> test multiple versions in one of my pet projects by using a CI build 
>> matrix
>> of a configurable dependency.  I didn't know Testcontainer.org has its 
>> own
>> Solr module -- https://www.testcontainers.org/modules/solr/ but we
>> implemented that ourselves; not hard.
>> >
>> >>
>> >> Trying to maintain across multiple versions is kind of a Sisyphean
>> task, and I don’t think plays to open source communities strengths.   
>> It’s
>> hard enough to keep all components of Solr up to date with the latest
>> Lucene and the latest Solr….  (Try out wt=xlsx Response Writer, it’s
>> currently broken on master) .  Reminds me of the Apache Gump project ;-)
>> >>
>> >> If something is really going to be backcompatible across certain
>> versions, then maybe having it’s own repo makes sense,
>> >
>> >
>> > I'd prefer that a module only leave our "contribs" area when the
>> concerns/limitations become real.  Doing it prematurely could lead to
>> atrophy of the module
>> >
>> >>
>> >> but I suspect it means those components may go stale….   Look at
>> DIH and Velocity components that are moved over to their own repos, both
>> are getting stale, and I’d argue it’s because they don’t live in the main
>> project where all of us have 

Re: Separate git repo(s) for Solr modules

[Noble Paul]

@Ilan Ginzburg 

I don't think the project split is counter productive because we have
lucene as a sub module. Solr does not use lucene like a simple library.
It's an integral part of Solr


On Tue, May 4, 2021 at 10:02 PM Ilan Ginzburg  wrote:

> As with any dependency on any project, you update the dependency project
> first then consume the updated dependency in Solr.
>
> If the idea is to be able to modify Lucene and Solr in parallel, then the
> project split is counterproductive.
>
> From the Solr perspective, Lucene and Zookerper are really two “similar”
> dependencies and IMO we should think about them in that way.
>
> Ilan
>
> On Tue 4 May 2021 at 09:45, Noble Paul  wrote:
>
>> @Houston
>>
>> So, Are you suggesting we should not do that ?
>>
>> On Tue, May 4, 2021 at 2:35 PM Houston Putman 
>> wrote:
>>
>>> In the future we wont be able to “work on both at the same time”, once
>>> Lucene 9 is cut. Why not pull that bandaid now?
>>>
>>> On Mon, May 3, 2021 at 11:32 PM Noble Paul  wrote:
>>>
 I'm still struggling to understand the workflow when I'm working on a
 feature that spans lucene and solr.

 I'm yet to see an argument against sub-modules

 On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski 
 wrote:

> > Shoving such a component into lucene-solr repo makes no sense, given
> its branching strategy is based on master / branch_8x
>
> I get how this could cause issues - I def hadn't thought much about
> multi-version support and branching.  But I don't think moving plugins
> to a separate repo solves that problem for us.  If our first class
> plugins are set up to have different release "lines" that don't line
> up with major Solr versions, it's only a matter of time before two of
> those plugins have branch requirements that conflict.  Unless I'm
> missing something here?
>
> > I'd prefer that a module only leave our "contribs" area when the
> concerns/limitations become real.  Doing it prematurely could lead to
> atrophy of the module
>
> +1 to David's comment.   I def hadn't considered the branching-scheme
> issues that Ishan brought up in his last reply, and they seem like
> valid concerns to me.  But the risk and the downsides of "atrophy" are
> serious enough that I'd vote to not risk them until this starts to
> cause us issues in practice.  Even if, for now, that means we won't be
> able to build a single plugin jar that supports (e.g.) 3 major Solr
> versions.  IMO that's a much smaller loss.
>
> On Tue, Feb 16, 2021 at 9:40 AM David Smiley 
> wrote:
> >
> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
> ep...@opensourceconnections.com> wrote:
> >>
> >> Testing across multiple versions is always very difficult ;-).  I
> recently saw this very interesting approach to using our Dockerized Solr’s
> to test a component against a number of previous versions of Solr.
> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
> model for other packages to adopt.
> >
> >
> > Thanks for the link to that Querqy PR.  That is *very* similar to
> what I do at work (minus multi-version testing), and also similar to how I
> test multiple versions in one of my pet projects by using a CI build 
> matrix
> of a configurable dependency.  I didn't know Testcontainer.org has its own
> Solr module -- https://www.testcontainers.org/modules/solr/ but we
> implemented that ourselves; not hard.
> >
> >>
> >> Trying to maintain across multiple versions is kind of a Sisyphean
> task, and I don’t think plays to open source communities strengths.   It’s
> hard enough to keep all components of Solr up to date with the latest
> Lucene and the latest Solr….  (Try out wt=xlsx Response Writer, it’s
> currently broken on master) .  Reminds me of the Apache Gump project ;-)
> >>
> >> If something is really going to be backcompatible across certain
> versions, then maybe having it’s own repo makes sense,
> >
> >
> > I'd prefer that a module only leave our "contribs" area when the
> concerns/limitations become real.  Doing it prematurely could lead to
> atrophy of the module
> >
> >>
> >> but I suspect it means those components may go stale….   Look at
> DIH and Velocity components that are moved over to their own repos, both
> are getting stale, and I’d argue it’s because they don’t live in the main
> project where all of us have oversight and easy access.
> >
> >
> > Agreed :-(
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

 --
 -
 Noble Paul

>>>
>>
>> --
>> 

Re: Separate git repo(s) for Solr modules

[Houston Putman]

I completely agree with Ilan.

Keeping the projects in the same repo was the only long term viable way of
having Solr depend on a snapshot version of Lucene.

And I'm sorry if I misspoke earlier. You can always "work on both at
the same time" even when we rely on a release version of Lucene,
but you won't be able to commit the Solr code until a Lucene release has
been made that includes those changes.

Also I think discussion has taken over the wrong thread. The thread was
originally about modules for various Solr plugins and projects.

- Houston

On Tue, May 4, 2021 at 7:02 AM Ilan Ginzburg  wrote:

> As with any dependency on any project, you update the dependency project
> first then consume the updated dependency in Solr.
>
> If the idea is to be able to modify Lucene and Solr in parallel, then the
> project split is counterproductive.
>
> From the Solr perspective, Lucene and Zookerper are really two “similar”
> dependencies and IMO we should think about them in that way.
>
> Ilan
>
> On Tue 4 May 2021 at 09:45, Noble Paul  wrote:
>
>> @Houston
>>
>> So, Are you suggesting we should not do that ?
>>
>> On Tue, May 4, 2021 at 2:35 PM Houston Putman 
>> wrote:
>>
>>> In the future we wont be able to “work on both at the same time”, once
>>> Lucene 9 is cut. Why not pull that bandaid now?
>>>
>>> On Mon, May 3, 2021 at 11:32 PM Noble Paul  wrote:
>>>
 I'm still struggling to understand the workflow when I'm working on a
 feature that spans lucene and solr.

 I'm yet to see an argument against sub-modules

 On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski 
 wrote:

> > Shoving such a component into lucene-solr repo makes no sense, given
> its branching strategy is based on master / branch_8x
>
> I get how this could cause issues - I def hadn't thought much about
> multi-version support and branching.  But I don't think moving plugins
> to a separate repo solves that problem for us.  If our first class
> plugins are set up to have different release "lines" that don't line
> up with major Solr versions, it's only a matter of time before two of
> those plugins have branch requirements that conflict.  Unless I'm
> missing something here?
>
> > I'd prefer that a module only leave our "contribs" area when the
> concerns/limitations become real.  Doing it prematurely could lead to
> atrophy of the module
>
> +1 to David's comment.   I def hadn't considered the branching-scheme
> issues that Ishan brought up in his last reply, and they seem like
> valid concerns to me.  But the risk and the downsides of "atrophy" are
> serious enough that I'd vote to not risk them until this starts to
> cause us issues in practice.  Even if, for now, that means we won't be
> able to build a single plugin jar that supports (e.g.) 3 major Solr
> versions.  IMO that's a much smaller loss.
>
> On Tue, Feb 16, 2021 at 9:40 AM David Smiley 
> wrote:
> >
> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
> ep...@opensourceconnections.com> wrote:
> >>
> >> Testing across multiple versions is always very difficult ;-).  I
> recently saw this very interesting approach to using our Dockerized Solr’s
> to test a component against a number of previous versions of Solr.
> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
> model for other packages to adopt.
> >
> >
> > Thanks for the link to that Querqy PR.  That is *very* similar to
> what I do at work (minus multi-version testing), and also similar to how I
> test multiple versions in one of my pet projects by using a CI build 
> matrix
> of a configurable dependency.  I didn't know Testcontainer.org has its own
> Solr module -- https://www.testcontainers.org/modules/solr/ but we
> implemented that ourselves; not hard.
> >
> >>
> >> Trying to maintain across multiple versions is kind of a Sisyphean
> task, and I don’t think plays to open source communities strengths.   It’s
> hard enough to keep all components of Solr up to date with the latest
> Lucene and the latest Solr….  (Try out wt=xlsx Response Writer, it’s
> currently broken on master) .  Reminds me of the Apache Gump project ;-)
> >>
> >> If something is really going to be backcompatible across certain
> versions, then maybe having it’s own repo makes sense,
> >
> >
> > I'd prefer that a module only leave our "contribs" area when the
> concerns/limitations become real.  Doing it prematurely could lead to
> atrophy of the module
> >
> >>
> >> but I suspect it means those components may go stale….   Look at
> DIH and Velocity components that are moved over to their own repos, both
> are getting stale, and I’d argue it’s because they don’t live in the main
> project where all of us have oversight and easy access.
> >
> >

Re: Separate git repo(s) for Solr modules

[Ilan Ginzburg]

As with any dependency on any project, you update the dependency project
first then consume the updated dependency in Solr.

If the idea is to be able to modify Lucene and Solr in parallel, then the
project split is counterproductive.

>From the Solr perspective, Lucene and Zookerper are really two “similar”
dependencies and IMO we should think about them in that way.

Ilan

On Tue 4 May 2021 at 09:45, Noble Paul  wrote:

> @Houston
>
> So, Are you suggesting we should not do that ?
>
> On Tue, May 4, 2021 at 2:35 PM Houston Putman 
> wrote:
>
>> In the future we wont be able to “work on both at the same time”, once
>> Lucene 9 is cut. Why not pull that bandaid now?
>>
>> On Mon, May 3, 2021 at 11:32 PM Noble Paul  wrote:
>>
>>> I'm still struggling to understand the workflow when I'm working on a
>>> feature that spans lucene and solr.
>>>
>>> I'm yet to see an argument against sub-modules
>>>
>>> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski 
>>> wrote:
>>>
 > Shoving such a component into lucene-solr repo makes no sense, given
 its branching strategy is based on master / branch_8x

 I get how this could cause issues - I def hadn't thought much about
 multi-version support and branching.  But I don't think moving plugins
 to a separate repo solves that problem for us.  If our first class
 plugins are set up to have different release "lines" that don't line
 up with major Solr versions, it's only a matter of time before two of
 those plugins have branch requirements that conflict.  Unless I'm
 missing something here?

 > I'd prefer that a module only leave our "contribs" area when the
 concerns/limitations become real.  Doing it prematurely could lead to
 atrophy of the module

 +1 to David's comment.   I def hadn't considered the branching-scheme
 issues that Ishan brought up in his last reply, and they seem like
 valid concerns to me.  But the risk and the downsides of "atrophy" are
 serious enough that I'd vote to not risk them until this starts to
 cause us issues in practice.  Even if, for now, that means we won't be
 able to build a single plugin jar that supports (e.g.) 3 major Solr
 versions.  IMO that's a much smaller loss.

 On Tue, Feb 16, 2021 at 9:40 AM David Smiley 
 wrote:
 >
 > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
 ep...@opensourceconnections.com> wrote:
 >>
 >> Testing across multiple versions is always very difficult ;-).  I
 recently saw this very interesting approach to using our Dockerized Solr’s
 to test a component against a number of previous versions of Solr.
 https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
 model for other packages to adopt.
 >
 >
 > Thanks for the link to that Querqy PR.  That is *very* similar to
 what I do at work (minus multi-version testing), and also similar to how I
 test multiple versions in one of my pet projects by using a CI build matrix
 of a configurable dependency.  I didn't know Testcontainer.org has its own
 Solr module -- https://www.testcontainers.org/modules/solr/ but we
 implemented that ourselves; not hard.
 >
 >>
 >> Trying to maintain across multiple versions is kind of a Sisyphean
 task, and I don’t think plays to open source communities strengths.   It’s
 hard enough to keep all components of Solr up to date with the latest
 Lucene and the latest Solr….  (Try out wt=xlsx Response Writer, it’s
 currently broken on master) .  Reminds me of the Apache Gump project ;-)
 >>
 >> If something is really going to be backcompatible across certain
 versions, then maybe having it’s own repo makes sense,
 >
 >
 > I'd prefer that a module only leave our "contribs" area when the
 concerns/limitations become real.  Doing it prematurely could lead to
 atrophy of the module
 >
 >>
 >> but I suspect it means those components may go stale….   Look at DIH
 and Velocity components that are moved over to their own repos, both are
 getting stale, and I’d argue it’s because they don’t live in the main
 project where all of us have oversight and easy access.
 >
 >
 > Agreed :-(
 >

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


>>>
>>> --
>>> -
>>> Noble Paul
>>>
>>
>
> --
> -
> Noble Paul
>

Re: Separate git repo(s) for Solr modules

[Noble Paul]

@Houston

So, Are you suggesting we should not do that ?

On Tue, May 4, 2021 at 2:35 PM Houston Putman 
wrote:

> In the future we wont be able to “work on both at the same time”, once
> Lucene 9 is cut. Why not pull that bandaid now?
>
> On Mon, May 3, 2021 at 11:32 PM Noble Paul  wrote:
>
>> I'm still struggling to understand the workflow when I'm working on a
>> feature that spans lucene and solr.
>>
>> I'm yet to see an argument against sub-modules
>>
>> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski 
>> wrote:
>>
>>> > Shoving such a component into lucene-solr repo makes no sense, given
>>> its branching strategy is based on master / branch_8x
>>>
>>> I get how this could cause issues - I def hadn't thought much about
>>> multi-version support and branching.  But I don't think moving plugins
>>> to a separate repo solves that problem for us.  If our first class
>>> plugins are set up to have different release "lines" that don't line
>>> up with major Solr versions, it's only a matter of time before two of
>>> those plugins have branch requirements that conflict.  Unless I'm
>>> missing something here?
>>>
>>> > I'd prefer that a module only leave our "contribs" area when the
>>> concerns/limitations become real.  Doing it prematurely could lead to
>>> atrophy of the module
>>>
>>> +1 to David's comment.   I def hadn't considered the branching-scheme
>>> issues that Ishan brought up in his last reply, and they seem like
>>> valid concerns to me.  But the risk and the downsides of "atrophy" are
>>> serious enough that I'd vote to not risk them until this starts to
>>> cause us issues in practice.  Even if, for now, that means we won't be
>>> able to build a single plugin jar that supports (e.g.) 3 major Solr
>>> versions.  IMO that's a much smaller loss.
>>>
>>> On Tue, Feb 16, 2021 at 9:40 AM David Smiley  wrote:
>>> >
>>> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
>>> ep...@opensourceconnections.com> wrote:
>>> >>
>>> >> Testing across multiple versions is always very difficult ;-).  I
>>> recently saw this very interesting approach to using our Dockerized Solr’s
>>> to test a component against a number of previous versions of Solr.
>>> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
>>> model for other packages to adopt.
>>> >
>>> >
>>> > Thanks for the link to that Querqy PR.  That is *very* similar to what
>>> I do at work (minus multi-version testing), and also similar to how I test
>>> multiple versions in one of my pet projects by using a CI build matrix of a
>>> configurable dependency.  I didn't know Testcontainer.org has its own Solr
>>> module -- https://www.testcontainers.org/modules/solr/ but we
>>> implemented that ourselves; not hard.
>>> >
>>> >>
>>> >> Trying to maintain across multiple versions is kind of a Sisyphean
>>> task, and I don’t think plays to open source communities strengths.   It’s
>>> hard enough to keep all components of Solr up to date with the latest
>>> Lucene and the latest Solr….  (Try out wt=xlsx Response Writer, it’s
>>> currently broken on master) .  Reminds me of the Apache Gump project ;-)
>>> >>
>>> >> If something is really going to be backcompatible across certain
>>> versions, then maybe having it’s own repo makes sense,
>>> >
>>> >
>>> > I'd prefer that a module only leave our "contribs" area when the
>>> concerns/limitations become real.  Doing it prematurely could lead to
>>> atrophy of the module
>>> >
>>> >>
>>> >> but I suspect it means those components may go stale….   Look at DIH
>>> and Velocity components that are moved over to their own repos, both are
>>> getting stale, and I’d argue it’s because they don’t live in the main
>>> project where all of us have oversight and easy access.
>>> >
>>> >
>>> > Agreed :-(
>>> >
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>>
>>
>> --
>> -
>> Noble Paul
>>
>

-- 
-
Noble Paul

Re: Separate git repo(s) for Solr modules

[Houston Putman]

In the future we wont be able to “work on both at the same time”, once
Lucene 9 is cut. Why not pull that bandaid now?

On Mon, May 3, 2021 at 11:32 PM Noble Paul  wrote:

> I'm still struggling to understand the workflow when I'm working on a
> feature that spans lucene and solr.
>
> I'm yet to see an argument against sub-modules
>
> On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski 
> wrote:
>
>> > Shoving such a component into lucene-solr repo makes no sense, given
>> its branching strategy is based on master / branch_8x
>>
>> I get how this could cause issues - I def hadn't thought much about
>> multi-version support and branching.  But I don't think moving plugins
>> to a separate repo solves that problem for us.  If our first class
>> plugins are set up to have different release "lines" that don't line
>> up with major Solr versions, it's only a matter of time before two of
>> those plugins have branch requirements that conflict.  Unless I'm
>> missing something here?
>>
>> > I'd prefer that a module only leave our "contribs" area when the
>> concerns/limitations become real.  Doing it prematurely could lead to
>> atrophy of the module
>>
>> +1 to David's comment.   I def hadn't considered the branching-scheme
>> issues that Ishan brought up in his last reply, and they seem like
>> valid concerns to me.  But the risk and the downsides of "atrophy" are
>> serious enough that I'd vote to not risk them until this starts to
>> cause us issues in practice.  Even if, for now, that means we won't be
>> able to build a single plugin jar that supports (e.g.) 3 major Solr
>> versions.  IMO that's a much smaller loss.
>>
>> On Tue, Feb 16, 2021 at 9:40 AM David Smiley  wrote:
>> >
>> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
>> ep...@opensourceconnections.com> wrote:
>> >>
>> >> Testing across multiple versions is always very difficult ;-).  I
>> recently saw this very interesting approach to using our Dockerized Solr’s
>> to test a component against a number of previous versions of Solr.
>> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
>> model for other packages to adopt.
>> >
>> >
>> > Thanks for the link to that Querqy PR.  That is *very* similar to what
>> I do at work (minus multi-version testing), and also similar to how I test
>> multiple versions in one of my pet projects by using a CI build matrix of a
>> configurable dependency.  I didn't know Testcontainer.org has its own Solr
>> module -- https://www.testcontainers.org/modules/solr/ but we
>> implemented that ourselves; not hard.
>> >
>> >>
>> >> Trying to maintain across multiple versions is kind of a Sisyphean
>> task, and I don’t think plays to open source communities strengths.   It’s
>> hard enough to keep all components of Solr up to date with the latest
>> Lucene and the latest Solr….  (Try out wt=xlsx Response Writer, it’s
>> currently broken on master) .  Reminds me of the Apache Gump project ;-)
>> >>
>> >> If something is really going to be backcompatible across certain
>> versions, then maybe having it’s own repo makes sense,
>> >
>> >
>> > I'd prefer that a module only leave our "contribs" area when the
>> concerns/limitations become real.  Doing it prematurely could lead to
>> atrophy of the module
>> >
>> >>
>> >> but I suspect it means those components may go stale….   Look at DIH
>> and Velocity components that are moved over to their own repos, both are
>> getting stale, and I’d argue it’s because they don’t live in the main
>> project where all of us have oversight and easy access.
>> >
>> >
>> > Agreed :-(
>> >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
>
> --
> -
> Noble Paul
>

Re: Separate git repo(s) for Solr modules

[Noble Paul]

I'm still struggling to understand the workflow when I'm working on a
feature that spans lucene and solr.

I'm yet to see an argument against sub-modules

On Wed, Feb 17, 2021 at 3:18 AM Jason Gerlowski 
wrote:

> > Shoving such a component into lucene-solr repo makes no sense, given its
> branching strategy is based on master / branch_8x
>
> I get how this could cause issues - I def hadn't thought much about
> multi-version support and branching.  But I don't think moving plugins
> to a separate repo solves that problem for us.  If our first class
> plugins are set up to have different release "lines" that don't line
> up with major Solr versions, it's only a matter of time before two of
> those plugins have branch requirements that conflict.  Unless I'm
> missing something here?
>
> > I'd prefer that a module only leave our "contribs" area when the
> concerns/limitations become real.  Doing it prematurely could lead to
> atrophy of the module
>
> +1 to David's comment.   I def hadn't considered the branching-scheme
> issues that Ishan brought up in his last reply, and they seem like
> valid concerns to me.  But the risk and the downsides of "atrophy" are
> serious enough that I'd vote to not risk them until this starts to
> cause us issues in practice.  Even if, for now, that means we won't be
> able to build a single plugin jar that supports (e.g.) 3 major Solr
> versions.  IMO that's a much smaller loss.
>
> On Tue, Feb 16, 2021 at 9:40 AM David Smiley  wrote:
> >
> > On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh <
> ep...@opensourceconnections.com> wrote:
> >>
> >> Testing across multiple versions is always very difficult ;-).  I
> recently saw this very interesting approach to using our Dockerized Solr’s
> to test a component against a number of previous versions of Solr.
> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
> model for other packages to adopt.
> >
> >
> > Thanks for the link to that Querqy PR.  That is *very* similar to what I
> do at work (minus multi-version testing), and also similar to how I test
> multiple versions in one of my pet projects by using a CI build matrix of a
> configurable dependency.  I didn't know Testcontainer.org has its own Solr
> module -- https://www.testcontainers.org/modules/solr/ but we implemented
> that ourselves; not hard.
> >
> >>
> >> Trying to maintain across multiple versions is kind of a Sisyphean
> task, and I don’t think plays to open source communities strengths.   It’s
> hard enough to keep all components of Solr up to date with the latest
> Lucene and the latest Solr….  (Try out wt=xlsx Response Writer, it’s
> currently broken on master) .  Reminds me of the Apache Gump project ;-)
> >>
> >> If something is really going to be backcompatible across certain
> versions, then maybe having it’s own repo makes sense,
> >
> >
> > I'd prefer that a module only leave our "contribs" area when the
> concerns/limitations become real.  Doing it prematurely could lead to
> atrophy of the module
> >
> >>
> >> but I suspect it means those components may go stale….   Look at DIH
> and Velocity components that are moved over to their own repos, both are
> getting stale, and I’d argue it’s because they don’t live in the main
> project where all of us have oversight and easy access.
> >
> >
> > Agreed :-(
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-- 
-
Noble Paul

Re: Separate git repo(s) for Solr modules

[Jason Gerlowski]

> Shoving such a component into lucene-solr repo makes no sense, given its 
> branching strategy is based on master / branch_8x

I get how this could cause issues - I def hadn't thought much about
multi-version support and branching.  But I don't think moving plugins
to a separate repo solves that problem for us.  If our first class
plugins are set up to have different release "lines" that don't line
up with major Solr versions, it's only a matter of time before two of
those plugins have branch requirements that conflict.  Unless I'm
missing something here?

> I'd prefer that a module only leave our "contribs" area when the 
> concerns/limitations become real.  Doing it prematurely could lead to atrophy 
> of the module

+1 to David's comment.   I def hadn't considered the branching-scheme
issues that Ishan brought up in his last reply, and they seem like
valid concerns to me.  But the risk and the downsides of "atrophy" are
serious enough that I'd vote to not risk them until this starts to
cause us issues in practice.  Even if, for now, that means we won't be
able to build a single plugin jar that supports (e.g.) 3 major Solr
versions.  IMO that's a much smaller loss.

On Tue, Feb 16, 2021 at 9:40 AM David Smiley  wrote:
>
> On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh  
> wrote:
>>
>> Testing across multiple versions is always very difficult ;-).  I recently 
>> saw this very interesting approach to using our Dockerized Solr’s to test a 
>> component against a number of previous versions of Solr.  
>> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a model 
>> for other packages to adopt.
>
>
> Thanks for the link to that Querqy PR.  That is *very* similar to what I do 
> at work (minus multi-version testing), and also similar to how I test 
> multiple versions in one of my pet projects by using a CI build matrix of a 
> configurable dependency.  I didn't know Testcontainer.org has its own Solr 
> module -- https://www.testcontainers.org/modules/solr/ but we implemented 
> that ourselves; not hard.
>
>>
>> Trying to maintain across multiple versions is kind of a Sisyphean task, and 
>> I don’t think plays to open source communities strengths.   It’s hard enough 
>> to keep all components of Solr up to date with the latest Lucene and the 
>> latest Solr….  (Try out wt=xlsx Response Writer, it’s currently broken on 
>> master) .  Reminds me of the Apache Gump project ;-)
>>
>> If something is really going to be backcompatible across certain versions, 
>> then maybe having it’s own repo makes sense,
>
>
> I'd prefer that a module only leave our "contribs" area when the 
> concerns/limitations become real.  Doing it prematurely could lead to atrophy 
> of the module
>
>>
>> but I suspect it means those components may go stale….   Look at DIH and 
>> Velocity components that are moved over to their own repos, both are getting 
>> stale, and I’d argue it’s because they don’t live in the main project where 
>> all of us have oversight and easy access.
>
>
> Agreed :-(
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Separate git repo(s) for Solr modules

[David Smiley]

On Tue, Feb 16, 2021 at 8:38 AM Eric Pugh 
wrote:

> Testing across multiple versions is always very difficult ;-).  I recently
> saw this very interesting approach to using our Dockerized Solr’s to test a
> component against a number of previous versions of Solr.
> https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a
> model for other packages to adopt.
>

Thanks for the link to that Querqy PR.  That is *very* similar to what I do
at work (minus multi-version testing), and also similar to how I test
multiple versions in one of my pet projects by using a CI build matrix of a
configurable dependency.  I didn't know Testcontainer.org has its own Solr
module -- https://www.testcontainers.org/modules/solr/ but we implemented
that ourselves; not hard.


> Trying to maintain across multiple versions is kind of a Sisyphean task,
> and I don’t think plays to open source communities strengths.   It’s hard
> enough to keep all components of Solr up to date with the latest Lucene and
> the latest Solr….  (Try out wt=xlsx Response Writer, it’s currently broken
> on master) .  Reminds me of the Apache Gump project ;-)
>
> If something is really going to be backcompatible across certain versions,
> then maybe having it’s own repo makes sense,
>

I'd prefer that a module only leave our "contribs" area when the
concerns/limitations become real.  Doing it prematurely could lead to
atrophy of the module


> but I suspect it means those components may go stale….   Look at DIH and
> Velocity components that are moved over to their own repos, both are
> getting stale, and I’d argue it’s because they don’t live in the main
> project where all of us have oversight and easy access.
>

Agreed :-(

Re: Separate git repo(s) for Solr modules

[Eric Pugh]

Testing across multiple versions is always very difficult ;-).  I recently saw 
this very interesting approach to using our Dockerized Solr’s to test a 
component against a number of previous versions of Solr.  
https://github.com/querqy/querqy/pull/147. I’m hopeful it could be a model for 
other packages to adopt.

Trying to maintain across multiple versions is kind of a Sisyphean task, and I 
don’t think plays to open source communities strengths.   It’s hard enough to 
keep all components of Solr up to date with the latest Lucene and the latest 
Solr….  (Try out wt=xlsx Response Writer, it’s currently broken on master) .  
Reminds me of the Apache Gump project ;-)

If something is really going to be backcompatible across certain versions, then 
maybe having it’s own repo makes sense, but I suspect it means those components 
may go stale….   Look at DIH and Velocity components that are moved over to 
their own repos, both are getting stale, and I’d argue it’s because they don’t 
live in the main project where all of us have oversight and easy access.


> On Feb 16, 2021, at 7:44 AM, Ishan Chattopadhyaya  
> wrote:
> 
> Imagine a component that wants to have its own release cadence. Say, a 1.x 
> line of the component that supports certain features (across Solr 6x to 9x) 
> and 2.x line that supports certain other features (which are a big upgrade 
> from previous 1.x). A Solr UI can be a good example, where say 1.x uses React 
> and 2.x uses AngularJS.
> Shoving such a component into lucene-solr repo makes no sense, given its 
> branching strategy is based on master / branch_8x (and later it will become 
> branch_9x and master).
> 
> On Tue, Feb 16, 2021 at 5:19 PM Jason Gerlowski  > wrote:
> > separate repo can be used to test against many Solr versions. Imagine a 
> > component that works across Solr versions 6x through 9x from day one. I 
> > can't imagine such a component being part of the lucene-solr repo itself.
> 
> It'll be great to test contrib's on multiple Solr versions, but I
> can't see any blockers for this that are same-repo specific.  Can you
> elaborate on what makes this tricky in a same-repo setup?  AFAICT you
> can set up multiple gradle test tasks that rely on different
> solr-test-framework versions and run tests that way - regardless of
> where the code lives.   (Or alternatively, docker images for various
> Solr versions as David suggested.)
> 
> I think Jan's got a good point about how do-able it'll be in practice
> to have plugins support multiple major release versions.  That said
> though, a lot of these questions (docker tests vs otherwise, cross
> major-version plugins?) will likely sort themselves out when it comes
> time to implement.
> 
> On Tue, Feb 16, 2021 at 12:21 AM David Smiley  > wrote:
> >
> > I think embracing Docker is how we can identify if a plugin works in 
> > multiple versions of Solr.  The setup of such a test would start Solr via 
> > Docker and install the plugin into a Solr server using the package manager. 
> >  The test code itself would interact with Solr purely via SolrClient.  This 
> > would be an integration / smoke level test for the plugin.  I test my 
> > plugins at work similar to this way at a "smoke" level, which follows 
> > "integration".  What's cool is that we can take the very same test and 
> > either have it run via an embedded Solr (which we call an integration test) 
> > or a remote'ed Solr (run via Docker, which we call our "smoke" test).  
> > Naturally, if a test makes assumptions that only work in embedded, then it 
> > won't run in smoke mode, so we have annotations to categorize the tests.  
> > I'd like to work on some JUnit "Rule" for Solr, similar to what I have at 
> > work (and have done in the past) -- 
> > https://issues.apache.org/jira/browse/SOLR-11872 
> > 
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley 
> > 
> >
> >
> > On Mon, Feb 15, 2021 at 9:17 AM Ishan Chattopadhyaya 
> > mailto:ichattopadhy...@gmail.com>> wrote:
> >>
> >> Hoss,
> >> One important reason why separate repo for solr-extras is a good idea, as 
> >> opposed to conrib modules, is that separate repo can be used to test 
> >> against many Solr versions. Imagine a component that works across Solr 
> >> versions 6x through 9x from day one. I can't imagine such a component 
> >> being part of the lucene-solr repo itself.
> >>
> >> Also, today, contrib modules are shipped with Solr, which we don't want 
> >> for new things we might want to build.
> >>
> >> On Mon, Jan 25, 2021 at 4:25 PM Ishan Chattopadhyaya 
> >> mailto:ichattopadhy...@gmail.com>> wrote:
> >>>
> >>> I haven't been able to follow up on creation of the extras repo, but more 
> >>> importantly I wanted to respond to Hoss. I'm out on an emergency for a 
> >>> week or so, shall resume 

Re: Separate git repo(s) for Solr modules

[Ishan Chattopadhyaya]

Imagine a component that wants to have its own release cadence. Say, a 1.x
line of the component that supports certain features (across Solr 6x to 9x)
and 2.x line that supports certain other features (which are a big upgrade
from previous 1.x). A Solr UI can be a good example, where say 1.x uses
React and 2.x uses AngularJS.
Shoving such a component into lucene-solr repo makes no sense, given its
branching strategy is based on master / branch_8x (and later it will become
branch_9x and master).

On Tue, Feb 16, 2021 at 5:19 PM Jason Gerlowski 
wrote:

> > separate repo can be used to test against many Solr versions. Imagine a
> component that works across Solr versions 6x through 9x from day one. I
> can't imagine such a component being part of the lucene-solr repo itself.
>
> It'll be great to test contrib's on multiple Solr versions, but I
> can't see any blockers for this that are same-repo specific.  Can you
> elaborate on what makes this tricky in a same-repo setup?  AFAICT you
> can set up multiple gradle test tasks that rely on different
> solr-test-framework versions and run tests that way - regardless of
> where the code lives.   (Or alternatively, docker images for various
> Solr versions as David suggested.)
>
> I think Jan's got a good point about how do-able it'll be in practice
> to have plugins support multiple major release versions.  That said
> though, a lot of these questions (docker tests vs otherwise, cross
> major-version plugins?) will likely sort themselves out when it comes
> time to implement.
>
> On Tue, Feb 16, 2021 at 12:21 AM David Smiley  wrote:
> >
> > I think embracing Docker is how we can identify if a plugin works in
> multiple versions of Solr.  The setup of such a test would start Solr via
> Docker and install the plugin into a Solr server using the package
> manager.  The test code itself would interact with Solr purely via
> SolrClient.  This would be an integration / smoke level test for the
> plugin.  I test my plugins at work similar to this way at a "smoke" level,
> which follows "integration".  What's cool is that we can take the very same
> test and either have it run via an embedded Solr (which we call an
> integration test) or a remote'ed Solr (run via Docker, which we call our
> "smoke" test).  Naturally, if a test makes assumptions that only work in
> embedded, then it won't run in smoke mode, so we have annotations to
> categorize the tests.  I'd like to work on some JUnit "Rule" for Solr,
> similar to what I have at work (and have done in the past) --
> https://issues.apache.org/jira/browse/SOLR-11872
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Mon, Feb 15, 2021 at 9:17 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >>
> >> Hoss,
> >> One important reason why separate repo for solr-extras is a good idea,
> as opposed to conrib modules, is that separate repo can be used to test
> against many Solr versions. Imagine a component that works across Solr
> versions 6x through 9x from day one. I can't imagine such a component being
> part of the lucene-solr repo itself.
> >>
> >> Also, today, contrib modules are shipped with Solr, which we don't want
> for new things we might want to build.
> >>
> >> On Mon, Jan 25, 2021 at 4:25 PM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >>>
> >>> I haven't been able to follow up on creation of the extras repo, but
> more importantly I wanted to respond to Hoss. I'm out on an emergency for a
> week or so, shall resume then. If there's a decision on this until then, I
> shall accept it.
> >>>
> >>> On Mon, 25 Jan, 2021, 9:04 am Jason Gerlowski, 
> wrote:
> 
>  Tentative +1 to Hoss' questions.  I agree with his summary of the
>  potential risks here, and I share his ignorance of the perceived
>  benefits.
> 
>  (I thought for a time that this was driven by interest in having
>  release cadences independent from Solr-core releases.  I'm all for
>  that goal, but if that's the motivation I'm not sure what the obstacle
>  is to doing that with a single repo - all build systems these days
>  support versioning and releasing modules independent of one another.
>  But maybe that was never a driving factor here.)
> 
>  I know there have been a lot of discussions about this, and I know the
>  repo has already been created.  So maybe it's too late to object even
>  if I wanted to, which I'm not sure I do.  But can someone that
>  understands the motivation please summarize what multiple-repos gets
>  us over a single repo that outweighs the "cons" that Hoss mentioned?
> 
>  Jason
> 
>  On Thu, Jan 14, 2021 at 12:34 PM Chris Hostetter
>   wrote:
>  >
>  >
>  > : As we discussed over the last few months, there seems a need to
> move
>  > : non-core pieces away from the Solr core module. The contribs are
> presently
>  > : 

Re: Separate git repo(s) for Solr modules

[Jason Gerlowski]

> separate repo can be used to test against many Solr versions. Imagine a 
> component that works across Solr versions 6x through 9x from day one. I can't 
> imagine such a component being part of the lucene-solr repo itself.

It'll be great to test contrib's on multiple Solr versions, but I
can't see any blockers for this that are same-repo specific.  Can you
elaborate on what makes this tricky in a same-repo setup?  AFAICT you
can set up multiple gradle test tasks that rely on different
solr-test-framework versions and run tests that way - regardless of
where the code lives.   (Or alternatively, docker images for various
Solr versions as David suggested.)

I think Jan's got a good point about how do-able it'll be in practice
to have plugins support multiple major release versions.  That said
though, a lot of these questions (docker tests vs otherwise, cross
major-version plugins?) will likely sort themselves out when it comes
time to implement.

On Tue, Feb 16, 2021 at 12:21 AM David Smiley  wrote:
>
> I think embracing Docker is how we can identify if a plugin works in multiple 
> versions of Solr.  The setup of such a test would start Solr via Docker and 
> install the plugin into a Solr server using the package manager.  The test 
> code itself would interact with Solr purely via SolrClient.  This would be an 
> integration / smoke level test for the plugin.  I test my plugins at work 
> similar to this way at a "smoke" level, which follows "integration".  What's 
> cool is that we can take the very same test and either have it run via an 
> embedded Solr (which we call an integration test) or a remote'ed Solr (run 
> via Docker, which we call our "smoke" test).  Naturally, if a test makes 
> assumptions that only work in embedded, then it won't run in smoke mode, so 
> we have annotations to categorize the tests.  I'd like to work on some JUnit 
> "Rule" for Solr, similar to what I have at work (and have done in the past) 
> -- https://issues.apache.org/jira/browse/SOLR-11872
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Mon, Feb 15, 2021 at 9:17 AM Ishan Chattopadhyaya 
>  wrote:
>>
>> Hoss,
>> One important reason why separate repo for solr-extras is a good idea, as 
>> opposed to conrib modules, is that separate repo can be used to test against 
>> many Solr versions. Imagine a component that works across Solr versions 6x 
>> through 9x from day one. I can't imagine such a component being part of the 
>> lucene-solr repo itself.
>>
>> Also, today, contrib modules are shipped with Solr, which we don't want for 
>> new things we might want to build.
>>
>> On Mon, Jan 25, 2021 at 4:25 PM Ishan Chattopadhyaya 
>>  wrote:
>>>
>>> I haven't been able to follow up on creation of the extras repo, but more 
>>> importantly I wanted to respond to Hoss. I'm out on an emergency for a week 
>>> or so, shall resume then. If there's a decision on this until then, I shall 
>>> accept it.
>>>
>>> On Mon, 25 Jan, 2021, 9:04 am Jason Gerlowski,  
>>> wrote:

 Tentative +1 to Hoss' questions.  I agree with his summary of the
 potential risks here, and I share his ignorance of the perceived
 benefits.

 (I thought for a time that this was driven by interest in having
 release cadences independent from Solr-core releases.  I'm all for
 that goal, but if that's the motivation I'm not sure what the obstacle
 is to doing that with a single repo - all build systems these days
 support versioning and releasing modules independent of one another.
 But maybe that was never a driving factor here.)

 I know there have been a lot of discussions about this, and I know the
 repo has already been created.  So maybe it's too late to object even
 if I wanted to, which I'm not sure I do.  But can someone that
 understands the motivation please summarize what multiple-repos gets
 us over a single repo that outweighs the "cons" that Hoss mentioned?

 Jason

 On Thu, Jan 14, 2021 at 12:34 PM Chris Hostetter
  wrote:
 >
 >
 > : As we discussed over the last few months, there seems a need to move
 > : non-core pieces away from the Solr core module. The contribs are 
 > presently
 > : a good place, but it makes sense to have a separate git repository 
 > hosting
 > : such modules. Some candidates that come to mind are the present day 
 > contrib
 >
 > can you explain why it makes sense to have a separate git repo for these
 > things?
 >
 > I can think of lots of reasons why it makes sense to have a single
 > repo for all things solr (unified CI that quickly identifies if core
 > changes break "first order" plugins, shared feature branches & monotomic
 > commits of code that affects APIs and impls of those APIs, etc...) but
 > I've yet to see any concrete specifics of why multiple git repos are
 > "better" then just 

Re: Separate git repo(s) for Solr modules

[David Smiley]

I think embracing Docker is how we can identify if a plugin works in
multiple versions of Solr.  The setup of such a test would start Solr via
Docker and install the plugin into a Solr server using the package
manager.  The test code itself would interact with Solr purely via
SolrClient.  This would be an integration / smoke level test for the
plugin.  I test my plugins at work similar to this way at a "smoke"
level, which follows "integration".  What's cool is that we can take the
very same test and either have it run via an embedded Solr (which we call
an integration test) or a remote'ed Solr (run via Docker, which we call our
"smoke" test).  Naturally, if a test makes assumptions that only work in
embedded, then it won't run in smoke mode, so we have annotations to
categorize the tests.  I'd like to work on some JUnit "Rule" for Solr,
similar to what I have at work (and have done in the past) --
https://issues.apache.org/jira/browse/SOLR-11872

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Mon, Feb 15, 2021 at 9:17 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Hoss,
> One important reason why separate repo for solr-extras is a good idea, as
> opposed to conrib modules, is that separate repo can be used to test
> against many Solr versions. Imagine a component that works across Solr
> versions 6x through 9x from day one. I can't imagine such a component being
> part of the lucene-solr repo itself.
>
> Also, today, contrib modules are shipped with Solr, which we don't want
> for new things we might want to build.
>
> On Mon, Jan 25, 2021 at 4:25 PM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> I haven't been able to follow up on creation of the extras repo, but more
>> importantly I wanted to respond to Hoss. I'm out on an emergency for a week
>> or so, shall resume then. If there's a decision on this until then, I shall
>> accept it.
>>
>> On Mon, 25 Jan, 2021, 9:04 am Jason Gerlowski, 
>> wrote:
>>
>>> Tentative +1 to Hoss' questions.  I agree with his summary of the
>>> potential risks here, and I share his ignorance of the perceived
>>> benefits.
>>>
>>> (I thought for a time that this was driven by interest in having
>>> release cadences independent from Solr-core releases.  I'm all for
>>> that goal, but if that's the motivation I'm not sure what the obstacle
>>> is to doing that with a single repo - all build systems these days
>>> support versioning and releasing modules independent of one another.
>>> But maybe that was never a driving factor here.)
>>>
>>> I know there have been a lot of discussions about this, and I know the
>>> repo has already been created.  So maybe it's too late to object even
>>> if I wanted to, which I'm not sure I do.  But can someone that
>>> understands the motivation please summarize what multiple-repos gets
>>> us over a single repo that outweighs the "cons" that Hoss mentioned?
>>>
>>> Jason
>>>
>>> On Thu, Jan 14, 2021 at 12:34 PM Chris Hostetter
>>>  wrote:
>>> >
>>> >
>>> > : As we discussed over the last few months, there seems a need to move
>>> > : non-core pieces away from the Solr core module. The contribs are
>>> presently
>>> > : a good place, but it makes sense to have a separate git repository
>>> hosting
>>> > : such modules. Some candidates that come to mind are the present day
>>> contrib
>>> >
>>> > can you explain why it makes sense to have a separate git repo for
>>> these
>>> > things?
>>> >
>>> > I can think of lots of reasons why it makes sense to have a single
>>> > repo for all things solr (unified CI that quickly identifies if core
>>> > changes break "first order" plugins, shared feature branches &
>>> monotomic
>>> > commits of code that affects APIs and impls of those APIs, etc...) but
>>> > I've yet to see any concrete specifics of why multiple git repos are
>>> > "better" then just having distinct sub-projects (with distinct
>>> artifacts)
>>> > in the same repo other then "it makes sense"
>>> >
>>> > why does it make sense?
>>> >
>>> > why can't the ideas of "solr-sandbox" and "solr-extras" just be
>>> > directories in the "solr repo" ? ... what value is gained by making
>>> them
>>> > new repos?
>>> >
>>> >
>>> > -Hoss
>>> > http://www.lucidworks.com/
>>> >
>>> > -
>>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>>> >
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>>

Re: Separate git repo(s) for Solr modules

[Jan Høydahl]

Hi,

> One important reason why separate repo for solr-extras is a good idea, as 
> opposed to conrib modules, is that separate repo can be used to test against 
> many Solr versions. Imagine a component that works across Solr versions 6x 
> through 9x from day one. I can't imagine such a component being part of the 
> lucene-solr repo itself.

For independent tools such as solr-operator or prometheus-exporter, or some 
future standalone DIH, it makes sense with separate repo and release cadence. 
However it will be much easier for our 1st party packages to remain in the 
monorepo, keeping all tests in-sync with core.

> Also, today, contrib modules are shipped with Solr, which we don't want for 
> new things we might want to build.

Remember that Solr main tarball does not need to include all our packages. We 
can easily extend our releae process to release e.g. solr-cell-9.5, 
solr-analysis-9.5, solr-auth-xyz-9.5 in our release repo together with the 9.5 
release, i.e. release every package with every Solr version. Then we do not 
need a compatibility matrix, since users will be able to install the latest 
package. And each release can ship with a pre-installed package repo for that 
version.

I imagine it will be very very hard to provide cross-major-version 
compatibility for some plugins. It may get easier as more higher-level APIs are 
build to shield plugins from low-level inner code, but we're not there.

Jan


> 15. feb. 2021 kl. 15:17 skrev Ishan Chattopadhyaya 
> :
> 
> Hoss,
> One important reason why separate repo for solr-extras is a good idea, as 
> opposed to conrib modules, is that separate repo can be used to test against 
> many Solr versions. Imagine a component that works across Solr versions 6x 
> through 9x from day one. I can't imagine such a component being part of the 
> lucene-solr repo itself.
> 
> Also, today, contrib modules are shipped with Solr, which we don't want for 
> new things we might want to build.
> 
> On Mon, Jan 25, 2021 at 4:25 PM Ishan Chattopadhyaya 
> mailto:ichattopadhy...@gmail.com>> wrote:
> I haven't been able to follow up on creation of the extras repo, but more 
> importantly I wanted to respond to Hoss. I'm out on an emergency for a week 
> or so, shall resume then. If there's a decision on this until then, I shall 
> accept it.
> 
> On Mon, 25 Jan, 2021, 9:04 am Jason Gerlowski,  > wrote:
> Tentative +1 to Hoss' questions.  I agree with his summary of the
> potential risks here, and I share his ignorance of the perceived
> benefits.
> 
> (I thought for a time that this was driven by interest in having
> release cadences independent from Solr-core releases.  I'm all for
> that goal, but if that's the motivation I'm not sure what the obstacle
> is to doing that with a single repo - all build systems these days
> support versioning and releasing modules independent of one another.
> But maybe that was never a driving factor here.)
> 
> I know there have been a lot of discussions about this, and I know the
> repo has already been created.  So maybe it's too late to object even
> if I wanted to, which I'm not sure I do.  But can someone that
> understands the motivation please summarize what multiple-repos gets
> us over a single repo that outweighs the "cons" that Hoss mentioned?
> 
> Jason
> 
> On Thu, Jan 14, 2021 at 12:34 PM Chris Hostetter
> mailto:hossman_luc...@fucit.org>> wrote:
> >
> >
> > : As we discussed over the last few months, there seems a need to move
> > : non-core pieces away from the Solr core module. The contribs are presently
> > : a good place, but it makes sense to have a separate git repository hosting
> > : such modules. Some candidates that come to mind are the present day 
> > contrib
> >
> > can you explain why it makes sense to have a separate git repo for these
> > things?
> >
> > I can think of lots of reasons why it makes sense to have a single
> > repo for all things solr (unified CI that quickly identifies if core
> > changes break "first order" plugins, shared feature branches & monotomic
> > commits of code that affects APIs and impls of those APIs, etc...) but
> > I've yet to see any concrete specifics of why multiple git repos are
> > "better" then just having distinct sub-projects (with distinct artifacts)
> > in the same repo other then "it makes sense"
> >
> > why does it make sense?
> >
> > why can't the ideas of "solr-sandbox" and "solr-extras" just be
> > directories in the "solr repo" ? ... what value is gained by making them
> > new repos?
> >
> >
> > -Hoss
> > http://www.lucidworks.com/ 
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org 
> > 
> > For additional commands, e-mail: dev-h...@lucene.apache.org 
> > 
> >
> 
> -

Re: Separate git repo(s) for Solr modules

[Ishan Chattopadhyaya]

Hoss,
One important reason why separate repo for solr-extras is a good idea, as
opposed to conrib modules, is that separate repo can be used to test
against many Solr versions. Imagine a component that works across Solr
versions 6x through 9x from day one. I can't imagine such a component being
part of the lucene-solr repo itself.

Also, today, contrib modules are shipped with Solr, which we don't want for
new things we might want to build.

On Mon, Jan 25, 2021 at 4:25 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I haven't been able to follow up on creation of the extras repo, but more
> importantly I wanted to respond to Hoss. I'm out on an emergency for a week
> or so, shall resume then. If there's a decision on this until then, I shall
> accept it.
>
> On Mon, 25 Jan, 2021, 9:04 am Jason Gerlowski, 
> wrote:
>
>> Tentative +1 to Hoss' questions.  I agree with his summary of the
>> potential risks here, and I share his ignorance of the perceived
>> benefits.
>>
>> (I thought for a time that this was driven by interest in having
>> release cadences independent from Solr-core releases.  I'm all for
>> that goal, but if that's the motivation I'm not sure what the obstacle
>> is to doing that with a single repo - all build systems these days
>> support versioning and releasing modules independent of one another.
>> But maybe that was never a driving factor here.)
>>
>> I know there have been a lot of discussions about this, and I know the
>> repo has already been created.  So maybe it's too late to object even
>> if I wanted to, which I'm not sure I do.  But can someone that
>> understands the motivation please summarize what multiple-repos gets
>> us over a single repo that outweighs the "cons" that Hoss mentioned?
>>
>> Jason
>>
>> On Thu, Jan 14, 2021 at 12:34 PM Chris Hostetter
>>  wrote:
>> >
>> >
>> > : As we discussed over the last few months, there seems a need to move
>> > : non-core pieces away from the Solr core module. The contribs are
>> presently
>> > : a good place, but it makes sense to have a separate git repository
>> hosting
>> > : such modules. Some candidates that come to mind are the present day
>> contrib
>> >
>> > can you explain why it makes sense to have a separate git repo for these
>> > things?
>> >
>> > I can think of lots of reasons why it makes sense to have a single
>> > repo for all things solr (unified CI that quickly identifies if core
>> > changes break "first order" plugins, shared feature branches & monotomic
>> > commits of code that affects APIs and impls of those APIs, etc...) but
>> > I've yet to see any concrete specifics of why multiple git repos are
>> > "better" then just having distinct sub-projects (with distinct
>> artifacts)
>> > in the same repo other then "it makes sense"
>> >
>> > why does it make sense?
>> >
>> > why can't the ideas of "solr-sandbox" and "solr-extras" just be
>> > directories in the "solr repo" ? ... what value is gained by making them
>> > new repos?
>> >
>> >
>> > -Hoss
>> > http://www.lucidworks.com/
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>> >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>

Re: Separate git repo(s) for Solr modules

[Ishan Chattopadhyaya]

I haven't been able to follow up on creation of the extras repo, but more
importantly I wanted to respond to Hoss. I'm out on an emergency for a week
or so, shall resume then. If there's a decision on this until then, I shall
accept it.

On Mon, 25 Jan, 2021, 9:04 am Jason Gerlowski, 
wrote:

> Tentative +1 to Hoss' questions.  I agree with his summary of the
> potential risks here, and I share his ignorance of the perceived
> benefits.
>
> (I thought for a time that this was driven by interest in having
> release cadences independent from Solr-core releases.  I'm all for
> that goal, but if that's the motivation I'm not sure what the obstacle
> is to doing that with a single repo - all build systems these days
> support versioning and releasing modules independent of one another.
> But maybe that was never a driving factor here.)
>
> I know there have been a lot of discussions about this, and I know the
> repo has already been created.  So maybe it's too late to object even
> if I wanted to, which I'm not sure I do.  But can someone that
> understands the motivation please summarize what multiple-repos gets
> us over a single repo that outweighs the "cons" that Hoss mentioned?
>
> Jason
>
> On Thu, Jan 14, 2021 at 12:34 PM Chris Hostetter
>  wrote:
> >
> >
> > : As we discussed over the last few months, there seems a need to move
> > : non-core pieces away from the Solr core module. The contribs are
> presently
> > : a good place, but it makes sense to have a separate git repository
> hosting
> > : such modules. Some candidates that come to mind are the present day
> contrib
> >
> > can you explain why it makes sense to have a separate git repo for these
> > things?
> >
> > I can think of lots of reasons why it makes sense to have a single
> > repo for all things solr (unified CI that quickly identifies if core
> > changes break "first order" plugins, shared feature branches & monotomic
> > commits of code that affects APIs and impls of those APIs, etc...) but
> > I've yet to see any concrete specifics of why multiple git repos are
> > "better" then just having distinct sub-projects (with distinct artifacts)
> > in the same repo other then "it makes sense"
> >
> > why does it make sense?
> >
> > why can't the ideas of "solr-sandbox" and "solr-extras" just be
> > directories in the "solr repo" ? ... what value is gained by making them
> > new repos?
> >
> >
> > -Hoss
> > http://www.lucidworks.com/
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Separate git repo(s) for Solr modules

[Jason Gerlowski]

Tentative +1 to Hoss' questions.  I agree with his summary of the
potential risks here, and I share his ignorance of the perceived
benefits.

(I thought for a time that this was driven by interest in having
release cadences independent from Solr-core releases.  I'm all for
that goal, but if that's the motivation I'm not sure what the obstacle
is to doing that with a single repo - all build systems these days
support versioning and releasing modules independent of one another.
But maybe that was never a driving factor here.)

I know there have been a lot of discussions about this, and I know the
repo has already been created.  So maybe it's too late to object even
if I wanted to, which I'm not sure I do.  But can someone that
understands the motivation please summarize what multiple-repos gets
us over a single repo that outweighs the "cons" that Hoss mentioned?

Jason

On Thu, Jan 14, 2021 at 12:34 PM Chris Hostetter
 wrote:
>
>
> : As we discussed over the last few months, there seems a need to move
> : non-core pieces away from the Solr core module. The contribs are presently
> : a good place, but it makes sense to have a separate git repository hosting
> : such modules. Some candidates that come to mind are the present day contrib
>
> can you explain why it makes sense to have a separate git repo for these
> things?
>
> I can think of lots of reasons why it makes sense to have a single
> repo for all things solr (unified CI that quickly identifies if core
> changes break "first order" plugins, shared feature branches & monotomic
> commits of code that affects APIs and impls of those APIs, etc...) but
> I've yet to see any concrete specifics of why multiple git repos are
> "better" then just having distinct sub-projects (with distinct artifacts)
> in the same repo other then "it makes sense"
>
> why does it make sense?
>
> why can't the ideas of "solr-sandbox" and "solr-extras" just be
> directories in the "solr repo" ? ... what value is gained by making them
> new repos?
>
>
> -Hoss
> http://www.lucidworks.com/
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Separate git repo(s) for Solr modules

[Chris Hostetter]

: As we discussed over the last few months, there seems a need to move
: non-core pieces away from the Solr core module. The contribs are presently
: a good place, but it makes sense to have a separate git repository hosting
: such modules. Some candidates that come to mind are the present day contrib

can you explain why it makes sense to have a separate git repo for these 
things?

I can think of lots of reasons why it makes sense to have a single 
repo for all things solr (unified CI that quickly identifies if core 
changes break "first order" plugins, shared feature branches & monotomic
commits of code that affects APIs and impls of those APIs, etc...) but 
I've yet to see any concrete specifics of why multiple git repos are 
"better" then just having distinct sub-projects (with distinct artifacts) 
in the same repo other then "it makes sense"

why does it make sense?

why can't the ideas of "solr-sandbox" and "solr-extras" just be 
directories in the "solr repo" ? ... what value is gained by making them 
new repos?


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Separate git repo(s) for Solr modules

[Ishan Chattopadhyaya]

Sounds good, thanks Houston for bringing it up. I'll take a stab at a few
first party packages soon and then we can take a call where they should
belong (maybe on a case to case basis).

Anshum has applied for the sandbox repo, I'll request for the extras repo
soon.

Thanks!

On Thu, 14 Jan, 2021, 4:04 am David Smiley,  wrote:

> Both paths have an appeal -- do contribs stay or move?  I recommend not
> moving existing contribs there until we explore how we can maintain
> compatibility with new Solr releases. (what Houston & Tomas point to
> below).  For example, *perhaps* a "git sub-module" pointing to Solr might
> be a key mechanism?  Could CI automatically keep it up to date?  Someone
> needs to try before we really know.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Wed, Jan 13, 2021 at 10:30 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> Hi Devs,
>>
>> As we discussed over the last few months, there seems a need to move
>> non-core pieces away from the Solr core module. The contribs are presently
>> a good place, but it makes sense to have a separate git repository hosting
>> such modules. Some candidates that come to mind are the present day contrib
>> modules, upcoming HDFS support module (separated away from solr-core),
>> other first party packages. Along with that, there is also a need for a
>> repository for hosting WIP modules/sub-projects.
>>
>> I propose that we apply for the creation of two new git repositories:
>> 1. solr-extras (or lucene-solr-extras)
>> 2. solr-sandbox (or lucene-solr-sandbox)
>>
>> Well tested, well supported modules/sub-projects can be released straight
>> away from *solr-extras*. The first party packages can be built from this
>> location and shipped with Solr (or be available for install using the
>> package manager CLI).
>>
>> New, unproved, beta, unstable modules can be hosted on *solr-sandbox*
>> (and graduate to solr-extras once stable).
>>
>> Please let me know if there are any questions/concerns with this approach.
>>
>> Thanks and regards,
>> Ishan
>>
>

Re: Separate git repo(s) for Solr modules

[David Smiley]

Both paths have an appeal -- do contribs stay or move?  I recommend not
moving existing contribs there until we explore how we can maintain
compatibility with new Solr releases. (what Houston & Tomas point to
below).  For example, *perhaps* a "git sub-module" pointing to Solr might
be a key mechanism?  Could CI automatically keep it up to date?  Someone
needs to try before we really know.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Wed, Jan 13, 2021 at 10:30 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Hi Devs,
>
> As we discussed over the last few months, there seems a need to move
> non-core pieces away from the Solr core module. The contribs are presently
> a good place, but it makes sense to have a separate git repository hosting
> such modules. Some candidates that come to mind are the present day contrib
> modules, upcoming HDFS support module (separated away from solr-core),
> other first party packages. Along with that, there is also a need for a
> repository for hosting WIP modules/sub-projects.
>
> I propose that we apply for the creation of two new git repositories:
> 1. solr-extras (or lucene-solr-extras)
> 2. solr-sandbox (or lucene-solr-sandbox)
>
> Well tested, well supported modules/sub-projects can be released straight
> away from *solr-extras*. The first party packages can be built from this
> location and shipped with Solr (or be available for install using the
> package manager CLI).
>
> New, unproved, beta, unstable modules can be hosted on *solr-sandbox*
> (and graduate to solr-extras once stable).
>
> Please let me know if there are any questions/concerns with this approach.
>
> Thanks and regards,
> Ishan
>

Re: Separate git repo(s) for Solr modules

[Tomás Fernández Löbbe]

Agere with Houston 100%. I think it's a good idea to have an official repo
for things that are related to Solr and, at the same time, don't belong
within the Solr codebase such as the prometheus exporter or, eventually,
the cross-dc work that Anshum is working on (once it "graduates" from
sandbox I guess). For the "first party" plugins (things like
analysis-extra, langid, ltr), it's better to keep them together with the
codebase, so that we can easily guarantee compatibility and releases
together with Solr.

On Wed, Jan 13, 2021 at 8:26 AM Atri Sharma  wrote:

> +1
>
> This is also an opportunity to create the distinction between first party
> supported packages and the other plug-ins.
>
>
>
> On Wed, 13 Jan 2021, 21:00 Ishan Chattopadhyaya, <
> ichattopadhy...@gmail.com> wrote:
>
>> Hi Devs,
>>
>> As we discussed over the last few months, there seems a need to move
>> non-core pieces away from the Solr core module. The contribs are presently
>> a good place, but it makes sense to have a separate git repository hosting
>> such modules. Some candidates that come to mind are the present day contrib
>> modules, upcoming HDFS support module (separated away from solr-core),
>> other first party packages. Along with that, there is also a need for a
>> repository for hosting WIP modules/sub-projects.
>>
>> I propose that we apply for the creation of two new git repositories:
>> 1. solr-extras (or lucene-solr-extras)
>> 2. solr-sandbox (or lucene-solr-sandbox)
>>
>> Well tested, well supported modules/sub-projects can be released straight
>> away from *solr-extras*. The first party packages can be built from this
>> location and shipped with Solr (or be available for install using the
>> package manager CLI).
>>
>> New, unproved, beta, unstable modules can be hosted on *solr-sandbox*
>> (and graduate to solr-extras once stable).
>>
>> Please let me know if there are any questions/concerns with this approach.
>>
>> Thanks and regards,
>> Ishan
>>
>

Re: Separate git repo(s) for Solr modules

[Atri Sharma]

+1

This is also an opportunity to create the distinction between first party
supported packages and the other plug-ins.



On Wed, 13 Jan 2021, 21:00 Ishan Chattopadhyaya, 
wrote:

> Hi Devs,
>
> As we discussed over the last few months, there seems a need to move
> non-core pieces away from the Solr core module. The contribs are presently
> a good place, but it makes sense to have a separate git repository hosting
> such modules. Some candidates that come to mind are the present day contrib
> modules, upcoming HDFS support module (separated away from solr-core),
> other first party packages. Along with that, there is also a need for a
> repository for hosting WIP modules/sub-projects.
>
> I propose that we apply for the creation of two new git repositories:
> 1. solr-extras (or lucene-solr-extras)
> 2. solr-sandbox (or lucene-solr-sandbox)
>
> Well tested, well supported modules/sub-projects can be released straight
> away from *solr-extras*. The first party packages can be built from this
> location and shipped with Solr (or be available for install using the
> package manager CLI).
>
> New, unproved, beta, unstable modules can be hosted on *solr-sandbox*
> (and graduate to solr-extras once stable).
>
> Please let me know if there are any questions/concerns with this approach.
>
> Thanks and regards,
> Ishan
>

Re: Separate git repo(s) for Solr modules

[Houston Putman]

I think it is a good idea to have solr-extras and solr-sandbox. However, I
think it's fine if some projects in solr-extras need to be migrated into
their own repos at some point.

I don't necessarily agree with moving all contrib modules out of the main
repo however. I think it makes the most sense to leave the "first-party"
plugin contribs inside the repo, so that they are released at the same
cadence as Solr itself, and compatibility is guaranteed whenever changes
are made to Solr core. Anything not in the main repo or released in the
same cadence as Solr is guaranteed to become stale with time. And if we do
have "first-party" plugins, then it stands to reason that those are
supposed to be supported on day-1 of a Solr release.

Some contrib modules aren't plugins, such as the Prometheus Exporter, so
this would make sense to be moved to something like solr-extras.

- Houston

On Wed, Jan 13, 2021 at 10:30 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Hi Devs,
>
> As we discussed over the last few months, there seems a need to move
> non-core pieces away from the Solr core module. The contribs are presently
> a good place, but it makes sense to have a separate git repository hosting
> such modules. Some candidates that come to mind are the present day contrib
> modules, upcoming HDFS support module (separated away from solr-core),
> other first party packages. Along with that, there is also a need for a
> repository for hosting WIP modules/sub-projects.
>
> I propose that we apply for the creation of two new git repositories:
> 1. solr-extras (or lucene-solr-extras)
> 2. solr-sandbox (or lucene-solr-sandbox)
>
> Well tested, well supported modules/sub-projects can be released straight
> away from *solr-extras*. The first party packages can be built from this
> location and shipped with Solr (or be available for install using the
> package manager CLI).
>
> New, unproved, beta, unstable modules can be hosted on *solr-sandbox*
> (and graduate to solr-extras once stable).
>
> Please let me know if there are any questions/concerns with this approach.
>
> Thanks and regards,
> Ishan
>

Separate git repo(s) for Solr modules

[Ishan Chattopadhyaya]

Hi Devs,

As we discussed over the last few months, there seems a need to move
non-core pieces away from the Solr core module. The contribs are presently
a good place, but it makes sense to have a separate git repository hosting
such modules. Some candidates that come to mind are the present day contrib
modules, upcoming HDFS support module (separated away from solr-core),
other first party packages. Along with that, there is also a need for a
repository for hosting WIP modules/sub-projects.

I propose that we apply for the creation of two new git repositories:
1. solr-extras (or lucene-solr-extras)
2. solr-sandbox (or lucene-solr-sandbox)

Well tested, well supported modules/sub-projects can be released straight
away from *solr-extras*. The first party packages can be built from this
location and shipped with Solr (or be available for install using the
package manager CLI).

New, unproved, beta, unstable modules can be hosted on *solr-sandbox* (and
graduate to solr-extras once stable).

Please let me know if there are any questions/concerns with this approach.

Thanks and regards,
Ishan