Re: Solr Alpha (EA) release of Reference Branch

[David Smiley]

I think whatever changes can be safely brought from that branch into master
should be done now independently of whatever happens to that branch.  It
may not be a lot of this branch, but it's something.  For example off the
top of my head -- pre-compiling XPath expressions.  AIUI, many changes
cannot be ported because they purterb tests even if the change itself is
fine.  So those can't be ported unless someone's willing to alo address the
test failures that go with it.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Wed, Oct 7, 2020 at 9:36 AM Ilan Ginzburg  wrote:

> TBH, a PR with more than 1400 changed files is hard to look at. How many
> of us will invest a few weeks at least to really understand it? We should
> assume that if we don't bring these changes piece by piece, we risk having
> an unstable version of SolrCloud for a while.
> When I look at some ongoing (unrelated) PR's judged too complex and being
> asked to split and simplify, this one would be at least two orders of
> magnitude "worse".
>
> Maybe this means releasing 9 without this branch but with all recent
> changes so we have a relatively stable release and merging the branch into
> master right afterwards as the base for Solr 10, then accepting a
> relatively long "baking" and bug fixing period?
> A beta release of 10 might then make more sense, since it will be "release
> candidate" type code and not a separate branch that has diverged.
>
> Ilan
>
> Le mer. 7 oct. 2020 à 13:43, Jan Høydahl  a écrit :
>
>> We want/need these improvements, for sure!
>> Agree treating this whole thing as a black box is dangerous. At the same
>> time I realize that this will be one or a few huge merges anyway!
>> Rob’s suggestion is interesting! If at all possible to get the branch up
>> to date with master, and make a PR, then it’s a great way for all of us to
>> have a deeper look.
>> And, who knows, perhaps end of this month there is consensus for merging
>> into master sooner rather than later?
>> Especially if we have had reliably passing Jenkins runs of the new
>> branch, with all tests enabled, for some time!
>> Risk is of course that 9.0.0 could be an unusually unstable release, but
>> people don’t expect bug free x.0.0 releases. And to avoid cherry-pick hell
>> for 6-12 months, perhaps that’s not a bad option after all.
>>
>> Jan
>>
>> 7. okt. 2020 kl. 07:48 skrev Robert Muir :
>>
>>
>> On Tue, Oct 6, 2020 at 10:45 PM Anshum Gupta 
>> wrote:
>>
>>>
>>> I haven’t looked at the current ref branch recently, but the folks who
>>> have looked at it, if you think that this code can be merged into master
>>> even as big chunks, that’d be the most confidence building way forward.
>>>
>>>
>> +1 for considering this approach. merge up master into the branch, and
>> make a big-ass PR to merge back. let people help review (maybe improve) the
>> change as a whole. It's just a big PR, some huge ones like this have been
>> done in lucene before too, unofficially called "unfuck" branches (sorry if
>> you are offended at my terminology). Sometimes you just fix, refactor,
>> cleanup, and keep iterating and see where it can lead. sometimes you revert
>> a bunch of commits because you followed the wrong rabbit-hole, etc.
>> Sometimes it may seem inconvenient, but I think we can all agree It's
>> important to have folks that want to not just take the small fix, but see
>> where it can go and make the whole thing better. Remember Mike's flexible
>> indexing branch?
>>
>> So why not try this way, look at actual code changes and try to get into
>> the master branch? Of course Uwe is willing to point build resources at it
>> either way, but if you want to maximize testing, start with the devs and
>> everyone's jenkins first before throwing at users. Master branch will get
>> you more testing, for sure.
>>
>>
>>

Re: Solr Alpha (EA) release of Reference Branch

[Ilan Ginzburg]

TBH, a PR with more than 1400 changed files is hard to look at. How many of
us will invest a few weeks at least to really understand it? We should
assume that if we don't bring these changes piece by piece, we risk having
an unstable version of SolrCloud for a while.
When I look at some ongoing (unrelated) PR's judged too complex and being
asked to split and simplify, this one would be at least two orders of
magnitude "worse".

Maybe this means releasing 9 without this branch but with all recent
changes so we have a relatively stable release and merging the branch into
master right afterwards as the base for Solr 10, then accepting a
relatively long "baking" and bug fixing period?
A beta release of 10 might then make more sense, since it will be "release
candidate" type code and not a separate branch that has diverged.

Ilan

Le mer. 7 oct. 2020 à 13:43, Jan Høydahl  a écrit :

> We want/need these improvements, for sure!
> Agree treating this whole thing as a black box is dangerous. At the same
> time I realize that this will be one or a few huge merges anyway!
> Rob’s suggestion is interesting! If at all possible to get the branch up
> to date with master, and make a PR, then it’s a great way for all of us to
> have a deeper look.
> And, who knows, perhaps end of this month there is consensus for merging
> into master sooner rather than later?
> Especially if we have had reliably passing Jenkins runs of the new branch,
> with all tests enabled, for some time!
> Risk is of course that 9.0.0 could be an unusually unstable release, but
> people don’t expect bug free x.0.0 releases. And to avoid cherry-pick hell
> for 6-12 months, perhaps that’s not a bad option after all.
>
> Jan
>
> 7. okt. 2020 kl. 07:48 skrev Robert Muir :
>
>
> On Tue, Oct 6, 2020 at 10:45 PM Anshum Gupta 
> wrote:
>
>>
>> I haven’t looked at the current ref branch recently, but the folks who
>> have looked at it, if you think that this code can be merged into master
>> even as big chunks, that’d be the most confidence building way forward.
>>
>>
> +1 for considering this approach. merge up master into the branch, and
> make a big-ass PR to merge back. let people help review (maybe improve) the
> change as a whole. It's just a big PR, some huge ones like this have been
> done in lucene before too, unofficially called "unfuck" branches (sorry if
> you are offended at my terminology). Sometimes you just fix, refactor,
> cleanup, and keep iterating and see where it can lead. sometimes you revert
> a bunch of commits because you followed the wrong rabbit-hole, etc.
> Sometimes it may seem inconvenient, but I think we can all agree It's
> important to have folks that want to not just take the small fix, but see
> where it can go and make the whole thing better. Remember Mike's flexible
> indexing branch?
>
> So why not try this way, look at actual code changes and try to get into
> the master branch? Of course Uwe is willing to point build resources at it
> either way, but if you want to maximize testing, start with the devs and
> everyone's jenkins first before throwing at users. Master branch will get
> you more testing, for sure.
>
>
>

Re: Solr Alpha (EA) release of Reference Branch

[Jan Høydahl]

We want/need these improvements, for sure!
Agree treating this whole thing as a black box is dangerous. At the same time I 
realize that this will be one or a few huge merges anyway!
Rob’s suggestion is interesting! If at all possible to get the branch up to 
date with master, and make a PR, then it’s a great way for all of us to have a 
deeper look. 
And, who knows, perhaps end of this month there is consensus for merging into 
master sooner rather than later?
Especially if we have had reliably passing Jenkins runs of the new branch, with 
all tests enabled, for some time!
Risk is of course that 9.0.0 could be an unusually unstable release, but people 
don’t expect bug free x.0.0 releases. And to avoid cherry-pick hell for 6-12 
months, perhaps that’s not a bad option after all.

Jan

> 7. okt. 2020 kl. 07:48 skrev Robert Muir :
> 
> 
> On Tue, Oct 6, 2020 at 10:45 PM Anshum Gupta  > wrote:
> 
> I haven’t looked at the current ref branch recently, but the folks who have 
> looked at it, if you think that this code can be merged into master even as 
> big chunks, that’d be the most confidence building way forward. 
> 
> 
> +1 for considering this approach. merge up master into the branch, and make a 
> big-ass PR to merge back. let people help review (maybe improve) the change 
> as a whole. It's just a big PR, some huge ones like this have been done in 
> lucene before too, unofficially called "unfuck" branches (sorry if you are 
> offended at my terminology). Sometimes you just fix, refactor, cleanup, and 
> keep iterating and see where it can lead. sometimes you revert a bunch of 
> commits because you followed the wrong rabbit-hole, etc.  Sometimes it may 
> seem inconvenient, but I think we can all agree It's important to have folks 
> that want to not just take the small fix, but see where it can go and make 
> the whole thing better. Remember Mike's flexible indexing branch?
> 
> So why not try this way, look at actual code changes and try to get into the 
> master branch? Of course Uwe is willing to point build resources at it either 
> way, but if you want to maximize testing, start with the devs and everyone's 
> jenkins first before throwing at users. Master branch will get you more 
> testing, for sure. 

Re: Solr Alpha (EA) release of Reference Branch

[Robert Muir]

On Tue, Oct 6, 2020 at 10:45 PM Anshum Gupta  wrote:

>
> I haven’t looked at the current ref branch recently, but the folks who
> have looked at it, if you think that this code can be merged into master
> even as big chunks, that’d be the most confidence building way forward.
>
>
+1 for considering this approach. merge up master into the branch, and make
a big-ass PR to merge back. let people help review (maybe improve) the
change as a whole. It's just a big PR, some huge ones like this have been
done in lucene before too, unofficially called "unfuck" branches (sorry if
you are offended at my terminology). Sometimes you just fix, refactor,
cleanup, and keep iterating and see where it can lead. sometimes you revert
a bunch of commits because you followed the wrong rabbit-hole, etc.
Sometimes it may seem inconvenient, but I think we can all agree It's
important to have folks that want to not just take the small fix, but see
where it can go and make the whole thing better. Remember Mike's flexible
indexing branch?

So why not try this way, look at actual code changes and try to get into
the master branch? Of course Uwe is willing to point build resources at it
either way, but if you want to maximize testing, start with the devs and
everyone's jenkins first before throwing at users. Master branch will get
you more testing, for sure.

Re: Solr Alpha (EA) release of Reference Branch

[Varun Thacker]

On Tue, Oct 6, 2020 at 7:45 PM Anshum Gupta  wrote:

> Thanks for initiating this discussion, Ishan.
>
> For the sake of making sure that we are all on the same page, let me
> summarize my understanding and take on this thread.
>
> The current situation
> Mark has a reference branch, which the folks who have looked at the
> branch, feel that it’s a much better, improved, reliable, and sustainable
> version of the current master i.e. take the same baseline and make it
> better. We would like to get those changes to the project, but aren’t sure
> about how to do so. Releasing the branch when it’s ready to go, as an alpha
> release will allow users to test it.
>
> 1. Is releasing the branch officially going to help us achieve the goal of
> having a well tested branch ?
> 2. Assuming #1 is true, do we as a community want to release the branch
> officially and assume responsibility?
>

I think so! We should all try to help out to the best we can.

3. What is our path forward after the release I.e. do we merge the branch
> into master or swap out current master.
>
> What do we plan to do (options).
> I feel there is a consensus on everyone wanting the best for the project
> and wanting Marks’ changes released.
>
> #1 - There are differing opinions, and I personally think we can have our
> test harnesses test the new branch, but I think most companies running Solr
> at scale would have concerns with taking up an alpha release and deploying
> it in production. The various tests that a bunch of folks are working on is
> our best bet at testing out the branch, in which case I’m not sure if we
> want an official release.
>
> #2 - I feel that having an official release and having artifacts show up
> in maven central will confuse people. The 4.0 alpha release was very
> different in the sense that it was the same branch, the code wasn’t
> replacing anything existing but introducing a completely new feature i.e.
> SolrCloud.
>
> #3 - I’m still unclear on how these changes will be released in terms of
> the community consensus. I’ve tried to merge parts of Marks’ effort from
> another time into master, but it’s very difficult, almost impossible to
> isolate and extract commits on the basis of coverage/features/etc. This is
> a lot of really great effort and after having spoken with Mark multiple
> times, I really feel we should figure out a way to absorb this but I do
> have concerns around replacing the master branch completely.
>
> While I do like the idea that Tomás proposed, I also feel that maintaining
> and managing cherry-picking across 9x, master, and ref branch will only
> make it difficult for people to work though the duration of 9x.
>
> I haven’t looked at the current ref branch recently, but the folks who
> have looked at it, if you think that this code can be merged into master
> even as big chunks, that’d be the most confidence building way forward.
>
>
>
>
> On Tue, Oct 6, 2020 at 11:37 AM Ilan Ginzburg  wrote:
>
>> Copying below Mark's posts from ASF Slack #solr-next-big-thing channel.
>>
>> The Solr Reference Branch.
>> Document 1, a quick intro.
>> You can think of the Solr Reference Branch as a remaster of Solr. It
>> is not an attempt to redesign Solr or make it more fancy. The goal of
>> the Solr Reference Branch is to be a better incarnation of the current
>> Apache Solr, which will provide a base for future development and
>> design.
>> There are a variety of problems with Solr today that make it difficult
>> to adopt and run. This is me being as honest and objective as I can
>> be, though no doubt, many will see it as an exaggeration or negative
>> focus. I just see it as the way it is and has been, it's just taken me
>> a real long time to actually get all the way under the rug to find the
>> really hardened nasty cockroaches burrowed in there.
>> 1. Resource usage and management is wasteful, inefficient, buggy, and
>> haphazard.
>> 2. SolrCloud is not long term reliable. Exceptional cases will
>> frequently flummox the system, and exceptional cases are supposed to
>> be our wheelhouse and primary focus. Leaders will be lost and not
>> recover, the Overseer will go away, GC storms will hit, tight loops in
>> a bad case will crank up resources, and retries will be abundant and
>> overaggressive.
>> 3. Our blocking and locking is generally not efficient, especially in key
>> paths.
>> 4. We get thread safety wrong (too often) in some important spots.
>> 5. Distributed updates have to be added locally before they are
>> distributed, and then that distribution is generally inefficient,
>> prone to blocking and/or timeouts, and hobbled by HTTP1.1 and our need
>> for packing updates into a single request to achieve any kind of
>> performance, losing proper error handling and eating the many rough
>> edges of the ConcurrentUpdateSolrClient.
>> 6. Our Zookeeper foundation code is often inefficient, buggy,
>> unreliable, and improperly used (we don’t always use async or multi
>> where we should, we 

Re: Solr Alpha (EA) release of Reference Branch

[Anshum Gupta]

Thanks for initiating this discussion, Ishan.

For the sake of making sure that we are all on the same page, let me
summarize my understanding and take on this thread.

The current situation
Mark has a reference branch, which the folks who have looked at the branch,
feel that it’s a much better, improved, reliable, and sustainable version
of the current master i.e. take the same baseline and make it better. We
would like to get those changes to the project, but aren’t sure about how
to do so. Releasing the branch when it’s ready to go, as an alpha release
will allow users to test it.

1. Is releasing the branch officially going to help us achieve the goal of
having a well tested branch ?
2. Assuming #1 is true, do we as a community want to release the branch
officially and assume responsibility?
3. What is our path forward after the release I.e. do we merge the branch
into master or swap out current master.

What do we plan to do (options).
I feel there is a consensus on everyone wanting the best for the project
and wanting Marks’ changes released.

#1 - There are differing opinions, and I personally think we can have our
test harnesses test the new branch, but I think most companies running Solr
at scale would have concerns with taking up an alpha release and deploying
it in production. The various tests that a bunch of folks are working on is
our best bet at testing out the branch, in which case I’m not sure if we
want an official release.

#2 - I feel that having an official release and having artifacts show up in
maven central will confuse people. The 4.0 alpha release was very different
in the sense that it was the same branch, the code wasn’t replacing
anything existing but introducing a completely new feature i.e. SolrCloud.

#3 - I’m still unclear on how these changes will be released in terms of
the community consensus. I’ve tried to merge parts of Marks’ effort from
another time into master, but it’s very difficult, almost impossible to
isolate and extract commits on the basis of coverage/features/etc. This is
a lot of really great effort and after having spoken with Mark multiple
times, I really feel we should figure out a way to absorb this but I do
have concerns around replacing the master branch completely.

While I do like the idea that Tomás proposed, I also feel that maintaining
and managing cherry-picking across 9x, master, and ref branch will only
make it difficult for people to work though the duration of 9x.

I haven’t looked at the current ref branch recently, but the folks who have
looked at it, if you think that this code can be merged into master even as
big chunks, that’d be the most confidence building way forward.




On Tue, Oct 6, 2020 at 11:37 AM Ilan Ginzburg  wrote:

> Copying below Mark's posts from ASF Slack #solr-next-big-thing channel.
>
> The Solr Reference Branch.
> Document 1, a quick intro.
> You can think of the Solr Reference Branch as a remaster of Solr. It
> is not an attempt to redesign Solr or make it more fancy. The goal of
> the Solr Reference Branch is to be a better incarnation of the current
> Apache Solr, which will provide a base for future development and
> design.
> There are a variety of problems with Solr today that make it difficult
> to adopt and run. This is me being as honest and objective as I can
> be, though no doubt, many will see it as an exaggeration or negative
> focus. I just see it as the way it is and has been, it's just taken me
> a real long time to actually get all the way under the rug to find the
> really hardened nasty cockroaches burrowed in there.
> 1. Resource usage and management is wasteful, inefficient, buggy, and
> haphazard.
> 2. SolrCloud is not long term reliable. Exceptional cases will
> frequently flummox the system, and exceptional cases are supposed to
> be our wheelhouse and primary focus. Leaders will be lost and not
> recover, the Overseer will go away, GC storms will hit, tight loops in
> a bad case will crank up resources, and retries will be abundant and
> overaggressive.
> 3. Our blocking and locking is generally not efficient, especially in key
> paths.
> 4. We get thread safety wrong (too often) in some important spots.
> 5. Distributed updates have to be added locally before they are
> distributed, and then that distribution is generally inefficient,
> prone to blocking and/or timeouts, and hobbled by HTTP1.1 and our need
> for packing updates into a single request to achieve any kind of
> performance, losing proper error handling and eating the many rough
> edges of the ConcurrentUpdateSolrClient.
> 6. Our Zookeeper foundation code is often inefficient, buggy,
> unreliable, and improperly used (we don’t always use async or multi
> where we should, we force updates from zk instead of being notified,
> we don’t handle session expiration as well as we should, our
> algorithms are slow and buggy, we make a multitude more calls than we
> should (especially on cluster startup), etc, etc)
> 7. We have circular 

Re: Solr Alpha (EA) release of Reference Branch

[Ilan Ginzburg]

Copying below Mark's posts from ASF Slack #solr-next-big-thing channel.

The Solr Reference Branch.
Document 1, a quick intro.
You can think of the Solr Reference Branch as a remaster of Solr. It
is not an attempt to redesign Solr or make it more fancy. The goal of
the Solr Reference Branch is to be a better incarnation of the current
Apache Solr, which will provide a base for future development and
design.
There are a variety of problems with Solr today that make it difficult
to adopt and run. This is me being as honest and objective as I can
be, though no doubt, many will see it as an exaggeration or negative
focus. I just see it as the way it is and has been, it's just taken me
a real long time to actually get all the way under the rug to find the
really hardened nasty cockroaches burrowed in there.
1. Resource usage and management is wasteful, inefficient, buggy, and haphazard.
2. SolrCloud is not long term reliable. Exceptional cases will
frequently flummox the system, and exceptional cases are supposed to
be our wheelhouse and primary focus. Leaders will be lost and not
recover, the Overseer will go away, GC storms will hit, tight loops in
a bad case will crank up resources, and retries will be abundant and
overaggressive.
3. Our blocking and locking is generally not efficient, especially in key paths.
4. We get thread safety wrong (too often) in some important spots.
5. Distributed updates have to be added locally before they are
distributed, and then that distribution is generally inefficient,
prone to blocking and/or timeouts, and hobbled by HTTP1.1 and our need
for packing updates into a single request to achieve any kind of
performance, losing proper error handling and eating the many rough
edges of the ConcurrentUpdateSolrClient.
6. Our Zookeeper foundation code is often inefficient, buggy,
unreliable, and improperly used (we don’t always use async or multi
where we should, we force updates from zk instead of being notified,
we don’t handle session expiration as well as we should, our
algorithms are slow and buggy, we make a multitude more calls than we
should (especially on cluster startup), etc, etc)
7. We have circular dependencies between major classes that can start
threads in their constructors that start interacting with the other
classes before construction is complete.
8. Our XML handling is abysmally outdated and slow for multiple
reasons. Our heavy Xpath usage is incredibly wasteful and expensive.
9. Our thread management is not understandable, not properly tunable,
not efficient, sometimes buggy, not always consistent, and difficult
to understand fundamentally.
10. Our Jetty configuration is lacking in a variety of ways,
especially around shutdown and http2.
11. The dynamic schema feature can be very expensive and not fully thread safe.
12. The Overseer is extremely inefficient, can be extremely slow to
stop, had a buggy leader election algorithm, doesn’t handle session
expiration as well as it should, can keep trying to come back from the
dead, and the list goes on.
13. Our connection resuse is often very poor or non existent, when
it’s improved, it always reverts back to bad or worse.
14. HTTP1.1 is not great for our type of application in a variety of
ways that HTTP2 solves – but we still use a lot of HTTP1.1 and HTTP2
is not configured well and the client needs some work.
15. Lifecycle of important objects is often off, most things can and
will leak (SolrCores, SolrIndexSearchers, Directory’s, Solr clients),
some things will close objects more than once or that don’t belong to
them, or close things in a bad order.
16. There is often sleeps and/or pulling that is a magnitude slower
than proper event driven waits.
17. Our tests are actually pretty unstable and making them stable is
way, way more difficult than most people realize. I’m quite sure I’ve
spent much, much more time on this than anyone out there, and I can
tell you, the tests are not stable in a 1,000 shifting ways that have
and will continue to cause lots of damage.
18. We don’t have good async update/search support for scaling and
better resource usage.
19. We often duplicate resources or create new pools instead of sharing.
20. We don’t do tons of parallelizable stuff in parallel, when we do
it’s inconsistent.
21. Our Collections API can often not wait correctly for the proper
state for what it did to be ready before returning. Even if it gets it
right, a cloud client that made the request won’t necessarily have the
updated state local when the request returns. Things often still work,
but with a variety of interesting and slow results possible.
22. We don’t often holistically look at what we have built and how it
fits together and so often there are silly things, bad fits, one off
bad patterns, lazy attempts at something, etc.
24. Close and shutdown are inefficient and slow across a huge swatch
of our object tree. These issues tend to be growy and breed less
concern over time.
25. There are a variety of ways and 

Re: Solr Alpha (EA) release of Reference Branch

[Tomás Fernández Löbbe]

> Let's say we cut 9x and now there is a new master taken from the
reference branch.
I never said “make a new master”, I said merge changes in ref branch into
master. If things are broken into pieces like Ishan is suggesting, those
changes can be merged into 9.x too. I only suggested this because you felt
unsure about merging to master now and I guess this is due to fear of
introducing bugs so close to a potential 9.0 release, is that not right?


> We will never be able to reconcile these 2 branches
Sorry, but how is that different if we do an alpha release from the branch
now? What would be the process after that? Let's say people don't find
issues and we want to merge those changes, what’s the plan then?

> Choice 1:
I’m fine with choice 1 if that’s what you want, as long as it’s not an
official release for the reasons stated above.


> I promise to do code review & cleanup as much as possible. But I'm
hesitant to give a stamp of approval to make it THE official release
What do you mean? I thought this is what you were suggesting, make an
official release from the reference_impl branch?


I think Ilan’s last email is on spot, and I agree 100% with what he can
express much better than I can :)

> Mark's descriptions in Slack go in the right way but are still too high
level
Can someone share those here? or in Jira?

On Tue, Oct 6, 2020 at 5:09 AM Noble Paul  wrote:

> > I think the danger is high to treat this branch as a black box (or an
> "all or nothing").
>
> True Ilan.  Ideally, I would like a few of us to study the code &
> start pulling in changes we are confident of (even to 8x branch, why
> not). We cannot burden a single developer to do everything.
>
> This cannot be a task just for one or 2 devs. We all will have to work
> together to decompose the changes and digest them into master. I can
> do my bit.
>
> But, I'm sure we may hit a point where certain changes cannot be
> isolated and absorbed. We will have to collectively make a call, how
> to absorb them
>
> On Tue, Oct 6, 2020 at 9:00 PM Ishan Chattopadhyaya
>  wrote:
> >
> >
> > I'm willing to help and I believe others will too if the amount of work
> for contributing is reasonable (i.e. not a three months effort).
> >
> > I looked into the possibility of doing so. To me, it seemed to be that
> it is very hard to do so: possibly 1 year project for me. Problem is that
> it is hard to pull out a particular class of improvements (say thread
> management improvement) and have all tests pass with it (because tests have
> gotten extensive improvements of their own) and also observe the effect of
> the improvement. IIUC, every improvement to Solr seemed to require many
> iterations to get the tests happy. I remember Mark telling me that it may
> not even be possible for him to do something like that (i.e. bring all
> changes into master as tiny pieces).
> >
> > What I volunteered to do, however, is to decompose roughly all the
> general improvements into smaller, manageable commits. However, making sure
> all tests pass at every commit point is beyond my capability.
> >
> > On Tue, 6 Oct, 2020, 3:10 pm Ilan Ginzburg,  wrote:
> >>
> >> Another option to integrate this work into the main code line would be
> to understand what changes have been made and where (Mark's descriptions in
> Slack go in the right way but are still too high level), and then port or
> even redo them in main, one by one.
> >>
> >> I think the danger is high to treat this branch as a black box (or an
> "all or nothing"). Using the merging itself to change our understanding and
> increase our knowledge of what was done can greatly reduce the risk.
> >>
> >> We do develop new features in Solr 9 without beta releasing them, so if
> we port Mark's improvements by small chunks (and maybe in the process
> decide that some should not be ported or not now) I don't see why this
> can't integrate to become like other improvements done to the code. If
> specific changes do require a beta release, do that release from master and
> pick the right moment.
> >>
> >> I'm willing to help and I believe others will too if the amount of work
> for contributing is reasonable (i.e. not a three months effort). This
> requires documenting the changes done in that branch, pointing to where
> these changes happened and then picking them up one by one and porting them
> more or less independently of each other. We might only port a subset of
> changes by the time 9.0 is released, that's fine we can continue in
> following releases.
> >>
> >> My 2 cents...
> >> Ilan
> >>
> >> Le mar. 6 oct. 2020 à 09:56, Noble Paul  a écrit
> :
> >>>
> >>> Yes, A docker image will definitely help. I wasn't trying to downplay
> that
> >>>
> >>> On Tue, Oct 6, 2020 at 6:55 PM Ishan Chattopadhyaya
> >>>  wrote:
> >>> >
> >>> >
> >>> > > Docker is not a big requirement for large scale installations.
> Most of them already have their own install scripts. Availability of docker
> is not important for them. If a user is only 

Re: Solr Alpha (EA) release of Reference Branch

[Noble Paul]

> I think the danger is high to treat this branch as a black box (or an "all or 
> nothing").

True Ilan.  Ideally, I would like a few of us to study the code &
start pulling in changes we are confident of (even to 8x branch, why
not). We cannot burden a single developer to do everything.

This cannot be a task just for one or 2 devs. We all will have to work
together to decompose the changes and digest them into master. I can
do my bit.

But, I'm sure we may hit a point where certain changes cannot be
isolated and absorbed. We will have to collectively make a call, how
to absorb them

On Tue, Oct 6, 2020 at 9:00 PM Ishan Chattopadhyaya
 wrote:
>
>
> I'm willing to help and I believe others will too if the amount of work for 
> contributing is reasonable (i.e. not a three months effort).
>
> I looked into the possibility of doing so. To me, it seemed to be that it is 
> very hard to do so: possibly 1 year project for me. Problem is that it is 
> hard to pull out a particular class of improvements (say thread management 
> improvement) and have all tests pass with it (because tests have gotten 
> extensive improvements of their own) and also observe the effect of the 
> improvement. IIUC, every improvement to Solr seemed to require many 
> iterations to get the tests happy. I remember Mark telling me that it may not 
> even be possible for him to do something like that (i.e. bring all changes 
> into master as tiny pieces).
>
> What I volunteered to do, however, is to decompose roughly all the general 
> improvements into smaller, manageable commits. However, making sure all tests 
> pass at every commit point is beyond my capability.
>
> On Tue, 6 Oct, 2020, 3:10 pm Ilan Ginzburg,  wrote:
>>
>> Another option to integrate this work into the main code line would be to 
>> understand what changes have been made and where (Mark's descriptions in 
>> Slack go in the right way but are still too high level), and then port or 
>> even redo them in main, one by one.
>>
>> I think the danger is high to treat this branch as a black box (or an "all 
>> or nothing"). Using the merging itself to change our understanding and 
>> increase our knowledge of what was done can greatly reduce the risk.
>>
>> We do develop new features in Solr 9 without beta releasing them, so if we 
>> port Mark's improvements by small chunks (and maybe in the process decide 
>> that some should not be ported or not now) I don't see why this can't 
>> integrate to become like other improvements done to the code. If specific 
>> changes do require a beta release, do that release from master and pick the 
>> right moment.
>>
>> I'm willing to help and I believe others will too if the amount of work for 
>> contributing is reasonable (i.e. not a three months effort). This requires 
>> documenting the changes done in that branch, pointing to where these changes 
>> happened and then picking them up one by one and porting them more or less 
>> independently of each other. We might only port a subset of changes by the 
>> time 9.0 is released, that's fine we can continue in following releases.
>>
>> My 2 cents...
>> Ilan
>>
>> Le mar. 6 oct. 2020 à 09:56, Noble Paul  a écrit :
>>>
>>> Yes, A docker image will definitely help. I wasn't trying to downplay that
>>>
>>> On Tue, Oct 6, 2020 at 6:55 PM Ishan Chattopadhyaya
>>>  wrote:
>>> >
>>> >
>>> > > Docker is not a big requirement for large scale installations. Most of 
>>> > > them already have their own install scripts. Availability of docker is 
>>> > > not important for them. If a user is only encouraged to install Solr 
>>> > > because of a docker image , most likely they are not running a large 
>>> > > enough cluster
>>> >
>>> > I disagree, Noble. Having a docker image us going to be useful to some 
>>> > clients, with complex usecases. Great point, David!
>>> >
>>> > On Tue, 6 Oct, 2020, 1:09 pm Ishan Chattopadhyaya, 
>>> >  wrote:
>>> >>
>>> >> As I said, I'm *personally* not confident in putting such a big 
>>> >> changeset into master that wasn't vetted in a real user environment 
>>> >> widely. I have, in the past, done enough bad things to Solr (directly or 
>>> >> indirectly), and I don't want to repeat the same. Also, I'll be very 
>>> >> uncomfortable if someone else did so.
>>> >>
>>> >> Having said this, if someone else wants to port the changes over to 
>>> >> master *without first getting enough real world testing*, feel free to 
>>> >> do so, and I can focus my efforts elsewhere.
>>> >>
>>> >> On Tue, 6 Oct, 2020, 9:22 am Tomás Fernández Löbbe, 
>>> >>  wrote:
>>> >>>
>>> >>> I was thinking (and I haven’t flushed it out completely but will throw 
>>> >>> the idea) that an alternative approach with this timeline could be to 
>>> >>> cut 9x branch around November/December? And then you could merge into 
>>> >>> master, it would have the latest  changes from master plus the ref 
>>> >>> branch changes. From there any nightly build could be use to help 
>>> >>> test/debug.
>>> >>>

Re: Solr Alpha (EA) release of Reference Branch

[Ishan Chattopadhyaya]

I'm willing to help and I believe others will too if the amount of work for
contributing is reasonable (i.e. not a three months effort).

I looked into the possibility of doing so. To me, it seemed to be that it
is very hard to do so: possibly 1 year project for me. Problem is that it
is hard to pull out a particular class of improvements (say thread
management improvement) and have all tests pass with it (because tests have
gotten extensive improvements of their own) and also observe the effect of
the improvement. IIUC, every improvement to Solr seemed to require many
iterations to get the tests happy. I remember Mark telling me that it may
not even be possible for him to do something like that (i.e. bring all
changes into master as tiny pieces).

What I volunteered to do, however, is to decompose roughly all the general
improvements into smaller, manageable commits. However, making sure all
tests pass at every commit point is beyond my capability.

On Tue, 6 Oct, 2020, 3:10 pm Ilan Ginzburg,  wrote:

> Another option to integrate this work into the main code line would be to
> understand what changes have been made and where (Mark's descriptions in
> Slack go in the right way but are still too high level), and then port or
> even redo them in main, one by one.
>
> I think the danger is high to treat this branch as a black box (or an "all
> or nothing"). Using the merging itself to change our understanding and
> increase our knowledge of what was done can greatly reduce the risk.
>
> We do develop new features in Solr 9 without beta releasing them, so if we
> port Mark's improvements by small chunks (and maybe in the process decide
> that some should not be ported or not now) I don't see why this can't
> integrate to become like other improvements done to the code. If specific
> changes do require a beta release, do that release from master and pick the
> right moment.
>
> I'm willing to help and I believe others will too if the amount of work
> for contributing is reasonable (i.e. not a three months effort). This
> requires documenting the changes done in that branch, pointing to where
> these changes happened and then picking them up one by one and porting them
> more or less independently of each other. We might only port a subset of
> changes by the time 9.0 is released, that's fine we can continue in
> following releases.
>
> My 2 cents...
> Ilan
>
> Le mar. 6 oct. 2020 à 09:56, Noble Paul  a écrit :
>
>> Yes, A docker image will definitely help. I wasn't trying to downplay that
>>
>> On Tue, Oct 6, 2020 at 6:55 PM Ishan Chattopadhyaya
>>  wrote:
>> >
>> >
>> > > Docker is not a big requirement for large scale installations. Most
>> of them already have their own install scripts. Availability of docker is
>> not important for them. If a user is only encouraged to install Solr
>> because of a docker image , most likely they are not running a large enough
>> cluster
>> >
>> > I disagree, Noble. Having a docker image us going to be useful to some
>> clients, with complex usecases. Great point, David!
>> >
>> > On Tue, 6 Oct, 2020, 1:09 pm Ishan Chattopadhyaya, <
>> ichattopadhy...@gmail.com> wrote:
>> >>
>> >> As I said, I'm *personally* not confident in putting such a big
>> changeset into master that wasn't vetted in a real user environment widely.
>> I have, in the past, done enough bad things to Solr (directly or
>> indirectly), and I don't want to repeat the same. Also, I'll be very
>> uncomfortable if someone else did so.
>> >>
>> >> Having said this, if someone else wants to port the changes over to
>> master *without first getting enough real world testing*, feel free to do
>> so, and I can focus my efforts elsewhere.
>> >>
>> >> On Tue, 6 Oct, 2020, 9:22 am Tomás Fernández Löbbe, <
>> tomasflo...@gmail.com> wrote:
>> >>>
>> >>> I was thinking (and I haven’t flushed it out completely but will
>> throw the idea) that an alternative approach with this timeline could be to
>> cut 9x branch around November/December? And then you could merge into
>> master, it would have the latest  changes from master plus the ref branch
>> changes. From there any nightly build could be use to help test/debug.
>> >>>
>> >>> That said I don’t know for sure what are the changes in the branch
>> that do not belong in 9. The problem with them being 10x only is that
>> backports would potentially be more difficult for all the life of 9.
>> >>>
>> >>> On Mon, Oct 5, 2020 at 4:54 PM Noble Paul 
>> wrote:
>> 
>>  >I don't think it can be said what committers do and don't do with
>> regards to running Solr.  All of us would answer this differently and at
>> different points in time.
>> 
>>  " I have run it in one large cluster, so it is certified to be bug
>> free/stable" I don't think it's a reasonable approach. We need as much
>> feedback from our users because each of them stress Solr in a different
>> way. This is not to suggest that committers are not doing testing or their
>> tests are not valid. When 

Re: Solr Alpha (EA) release of Reference Branch

[Ilan Ginzburg]

Another option to integrate this work into the main code line would be to
understand what changes have been made and where (Mark's descriptions in
Slack go in the right way but are still too high level), and then port or
even redo them in main, one by one.

I think the danger is high to treat this branch as a black box (or an "all
or nothing"). Using the merging itself to change our understanding and
increase our knowledge of what was done can greatly reduce the risk.

We do develop new features in Solr 9 without beta releasing them, so if we
port Mark's improvements by small chunks (and maybe in the process decide
that some should not be ported or not now) I don't see why this can't
integrate to become like other improvements done to the code. If specific
changes do require a beta release, do that release from master and pick the
right moment.

I'm willing to help and I believe others will too if the amount of work for
contributing is reasonable (i.e. not a three months effort). This requires
documenting the changes done in that branch, pointing to where these
changes happened and then picking them up one by one and porting them more
or less independently of each other. We might only port a subset of changes
by the time 9.0 is released, that's fine we can continue in following
releases.

My 2 cents...
Ilan

Le mar. 6 oct. 2020 à 09:56, Noble Paul  a écrit :

> Yes, A docker image will definitely help. I wasn't trying to downplay that
>
> On Tue, Oct 6, 2020 at 6:55 PM Ishan Chattopadhyaya
>  wrote:
> >
> >
> > > Docker is not a big requirement for large scale installations. Most of
> them already have their own install scripts. Availability of docker is not
> important for them. If a user is only encouraged to install Solr because of
> a docker image , most likely they are not running a large enough cluster
> >
> > I disagree, Noble. Having a docker image us going to be useful to some
> clients, with complex usecases. Great point, David!
> >
> > On Tue, 6 Oct, 2020, 1:09 pm Ishan Chattopadhyaya, <
> ichattopadhy...@gmail.com> wrote:
> >>
> >> As I said, I'm *personally* not confident in putting such a big
> changeset into master that wasn't vetted in a real user environment widely.
> I have, in the past, done enough bad things to Solr (directly or
> indirectly), and I don't want to repeat the same. Also, I'll be very
> uncomfortable if someone else did so.
> >>
> >> Having said this, if someone else wants to port the changes over to
> master *without first getting enough real world testing*, feel free to do
> so, and I can focus my efforts elsewhere.
> >>
> >> On Tue, 6 Oct, 2020, 9:22 am Tomás Fernández Löbbe, <
> tomasflo...@gmail.com> wrote:
> >>>
> >>> I was thinking (and I haven’t flushed it out completely but will throw
> the idea) that an alternative approach with this timeline could be to cut
> 9x branch around November/December? And then you could merge into master,
> it would have the latest  changes from master plus the ref branch changes.
> From there any nightly build could be use to help test/debug.
> >>>
> >>> That said I don’t know for sure what are the changes in the branch
> that do not belong in 9. The problem with them being 10x only is that
> backports would potentially be more difficult for all the life of 9.
> >>>
> >>> On Mon, Oct 5, 2020 at 4:54 PM Noble Paul 
> wrote:
> 
>  >I don't think it can be said what committers do and don't do with
> regards to running Solr.  All of us would answer this differently and at
> different points in time.
> 
>  " I have run it in one large cluster, so it is certified to be bug
> free/stable" I don't think it's a reasonable approach. We need as much
> feedback from our users because each of them stress Solr in a different
> way. This is not to suggest that committers are not doing testing or their
> tests are not valid. When I talk to the committers out here they say they
> do not see any performance stability issues at all. But, my client reports
> issues on a day to day basis.
> 
> 
> 
>  > Definitely publish a Docker image BTW -- it's the best way to try
> out any software.
> 
>  Docker is not a big requirement for large scale installations. Most
> of them already have their own install scripts. Availability of docker is
> not important for them. If a user is only encouraged to install Solr
> because of a docker image , most likely they are not running a large enough
> cluster
> 
> 
> 
>  On Tue, Oct 6, 2020, 6:30 AM David Smiley  wrote:
> >
> > Thanks so much for your responses Ishan... I'm getting much more
> information in this thread than my attempts to get questions answered on
> the JIRA issue months ago.  And especially,  thank you for volunteering for
> the difficult porting efforts!
> >
> > Tomas said:
> >>
> >>  I do agree with the previous comments that calling it "Solr 10"
> (even with the "-alpha") would confuse users, maybe use "reference"? or
> 

Re: Solr Alpha (EA) release of Reference Branch

[Ishan Chattopadhyaya]

> Docker is not a big requirement for large scale installations. Most of
them already have their own install scripts. Availability of docker is not
important for them. If a user is only encouraged to install Solr because of
a docker image , most likely they are not running a large enough cluster

I disagree, Noble. Having a docker image us going to be useful to some
clients, with complex usecases. Great point, David!

On Tue, 6 Oct, 2020, 1:09 pm Ishan Chattopadhyaya, <
ichattopadhy...@gmail.com> wrote:

> As I said, I'm *personally* not confident in putting such a big changeset
> into master that wasn't vetted in a real user environment widely. I have,
> in the past, done enough bad things to Solr (directly or indirectly), and I
> don't want to repeat the same. Also, I'll be very uncomfortable if someone
> else did so.
>
> Having said this, if someone else wants to port the changes over to master
> *without first getting enough real world testing*, feel free to do so, and
> I can focus my efforts elsewhere.
>
> On Tue, 6 Oct, 2020, 9:22 am Tomás Fernández Löbbe, 
> wrote:
>
>> I was thinking (and I haven’t flushed it out completely but will throw
>> the idea) that an alternative approach with this timeline could be to cut
>> 9x branch around November/December? And then you could merge into master,
>> it would have the latest  changes from master plus the ref branch changes.
>> From there any nightly build could be use to help test/debug.
>>
>> That said I don’t know for sure what are the changes in the branch that
>> do not belong in 9. The problem with them being 10x only is that backports
>> would potentially be more difficult for all the life of 9.
>>
>> On Mon, Oct 5, 2020 at 4:54 PM Noble Paul  wrote:
>>
>>> >I don't think it can be said what committers do and don't do with
>>> regards to running Solr.  All of us would answer this differently and at
>>> different points in time.
>>>
>>> " I have run it in one large cluster, so it is certified to be bug
>>> free/stable" I don't think it's a reasonable approach. We need as much
>>> feedback from our users because each of them stress Solr in a
>>> different way. This is not to suggest that committers are not doing testing
>>> or their tests are not valid. When I talk to the committers out here they
>>> say they do not see any performance stability issues at all. But, my client
>>> reports issues on a day to day basis.
>>>
>>>
>>>
>>> > Definitely publish a Docker image BTW -- it's the best way to try out
>>> any software.
>>>
>>> Docker is not a big requirement for large scale installations. Most of
>>> them already have their own install scripts. Availability of docker is not
>>> important for them. If a user is only encouraged to install Solr because of
>>> a docker image , most likely they are not running a large enough cluster
>>>
>>>
>>>
>>> On Tue, Oct 6, 2020, 6:30 AM David Smiley  wrote:
>>>
 Thanks so much for your responses Ishan... I'm getting much more
 information in this thread than my attempts to get questions answered on
 the JIRA issue months ago.  And especially,  thank you for volunteering for
 the difficult porting efforts!

 Tomas said:

>  I do agree with the previous comments that calling it "Solr 10" (even
> with the "-alpha") would confuse users, maybe use "reference"? or maybe
> something in reference to SOLR-14788?
>

 I have the opposite opinion.  This word "reference" is baffling to me
 despite whatever Mark's explanation is.  I like the justification Ishan
 gave for 10-alpha and I don't think I could re-phrase his justification any
 better.  *If* the release was _not_ official (thus wouldn't show up in the
 usual places anyone would look for a release), I think it would alleviate
 that confusion concern even more, although I think "alpha" ought to be
 enough of a signal not to use it without digging deeper on what's going on.

 Alex then Ishan said:

> > Maybe we could release it to
> > committers community first and dogfood it "internally"?
>
> Alex: It is meaningless. Committers don't run large scale
> installations. We barely even have time to take care of running unit tests
> before destabilizing our builds. We are not the right audience. However, 
> we
> all can anyway check out the branch and start playing with it, even 
> without
> a release. There are orgs that don't want to install any code that wasn't
> officially released; this release is geared towards them (to help us test
> this at their scale).
>

 I don't think it can be said what committers do and don't do with
 regards to running Solr.  All of us would answer this differently and at
 different points in time.  From time to time, though not at present, I've
 been well positioned to try out a new version of Solr in a stage/test
 environment to see how it goes.  (Putting on my Salesforce 

Re: Solr Alpha (EA) release of Reference Branch

[Noble Paul]

Yes, A docker image will definitely help. I wasn't trying to downplay that

On Tue, Oct 6, 2020 at 6:55 PM Ishan Chattopadhyaya
 wrote:
>
>
> > Docker is not a big requirement for large scale installations. Most of them 
> > already have their own install scripts. Availability of docker is not 
> > important for them. If a user is only encouraged to install Solr because of 
> > a docker image , most likely they are not running a large enough cluster
>
> I disagree, Noble. Having a docker image us going to be useful to some 
> clients, with complex usecases. Great point, David!
>
> On Tue, 6 Oct, 2020, 1:09 pm Ishan Chattopadhyaya, 
>  wrote:
>>
>> As I said, I'm *personally* not confident in putting such a big changeset 
>> into master that wasn't vetted in a real user environment widely. I have, in 
>> the past, done enough bad things to Solr (directly or indirectly), and I 
>> don't want to repeat the same. Also, I'll be very uncomfortable if someone 
>> else did so.
>>
>> Having said this, if someone else wants to port the changes over to master 
>> *without first getting enough real world testing*, feel free to do so, and I 
>> can focus my efforts elsewhere.
>>
>> On Tue, 6 Oct, 2020, 9:22 am Tomás Fernández Löbbe,  
>> wrote:
>>>
>>> I was thinking (and I haven’t flushed it out completely but will throw the 
>>> idea) that an alternative approach with this timeline could be to cut 9x 
>>> branch around November/December? And then you could merge into master, it 
>>> would have the latest  changes from master plus the ref branch changes. 
>>> From there any nightly build could be use to help test/debug.
>>>
>>> That said I don’t know for sure what are the changes in the branch that do 
>>> not belong in 9. The problem with them being 10x only is that backports 
>>> would potentially be more difficult for all the life of 9.
>>>
>>> On Mon, Oct 5, 2020 at 4:54 PM Noble Paul  wrote:

 >I don't think it can be said what committers do and don't do with regards 
 >to running Solr.  All of us would answer this differently and at 
 >different points in time.

 " I have run it in one large cluster, so it is certified to be bug 
 free/stable" I don't think it's a reasonable approach. We need as much 
 feedback from our users because each of them stress Solr in a different 
 way. This is not to suggest that committers are not doing testing or their 
 tests are not valid. When I talk to the committers out here they say they 
 do not see any performance stability issues at all. But, my client reports 
 issues on a day to day basis.



 > Definitely publish a Docker image BTW -- it's the best way to try out 
 > any software.

 Docker is not a big requirement for large scale installations. Most of 
 them already have their own install scripts. Availability of docker is not 
 important for them. If a user is only encouraged to install Solr because 
 of a docker image , most likely they are not running a large enough cluster



 On Tue, Oct 6, 2020, 6:30 AM David Smiley  wrote:
>
> Thanks so much for your responses Ishan... I'm getting much more 
> information in this thread than my attempts to get questions answered on 
> the JIRA issue months ago.  And especially,  thank you for volunteering 
> for the difficult porting efforts!
>
> Tomas said:
>>
>>  I do agree with the previous comments that calling it "Solr 10" (even 
>> with the "-alpha") would confuse users, maybe use "reference"? or maybe 
>> something in reference to SOLR-14788?
>
>
> I have the opposite opinion.  This word "reference" is baffling to me 
> despite whatever Mark's explanation is.  I like the justification Ishan 
> gave for 10-alpha and I don't think I could re-phrase his justification 
> any better.  *If* the release was _not_ official (thus wouldn't show up 
> in the usual places anyone would look for a release), I think it would 
> alleviate that confusion concern even more, although I think "alpha" 
> ought to be enough of a signal not to use it without digging deeper on 
> what's going on.
>
> Alex then Ishan said:
>>
>> > Maybe we could release it to
>> > committers community first and dogfood it "internally"?
>>
>> Alex: It is meaningless. Committers don't run large scale installations. 
>> We barely even have time to take care of running unit tests before 
>> destabilizing our builds. We are not the right audience. However, we all 
>> can anyway check out the branch and start playing with it, even without 
>> a release. There are orgs that don't want to install any code that 
>> wasn't officially released; this release is geared towards them (to help 
>> us test this at their scale).
>
>
> I don't think it can be said what committers do and don't do with regards 
> to running Solr.  

Re: Solr Alpha (EA) release of Reference Branch

[Noble Paul]

@Tomas Fernandez Lobbe
This is a very risky proposition. Let's say we cut 9x and now there is
 a new master taken from the reference branch. That master is now
going to be 10.0.

* We never get it to the hands of the users anytime soon. We will
never be able to reconcile these 2 branches
* The master & 9x branches will diverge so much that we cannot make
any meaningful commits to any of these branches

Reference branch will have all the features of Solr 9.0 because it is
forked from master (some latest changes are yet to be merged). So,
reference branch is 100% compatible with master. A user should be able
to do a drop in replacement between 9.0 & reference branch. It is also
possible to have  a cluster with nodes running 9x and reference
together( A rolling restart should be possible).
Let's take stock of what we have today.

Choice 1:
* Mark has made some significant changes to improve performance stability
* Mark has promised to make all tests pass over the next 2-3 weeks
* Ishan has promised to do the work to make a release possible
* Ishan has promised to port changes to master as soon as we get
enough user feedback. He has also expressed his reservations on doing
this before it is tested in the wild
* I promise to do code review & cleanup as much as possible. But I'm
hesitant to give a stamp of approval to make it THE official release

Choice: 2
* Tomas promise to port changes from the branch to master
* Tomas make a release

Choice 3:
* Status quo
* We throw away all the good work that is done & demotivate the developers
* Deprive users of all the promised goodness

I'm not sure what else is a choice and who is volunteering to work on
that. We need people who are willing to offer to get their hands
dirty. Personally, I'm totally against choice #3 and I'm willing to
collaborate with anyone who is has a plan to make all the goodness in
the branch available to our users

On Tue, Oct 6, 2020 at 6:39 PM Ishan Chattopadhyaya
 wrote:
>
> As I said, I'm *personally* not confident in putting such a big changeset 
> into master that wasn't vetted in a real user environment widely. I have, in 
> the past, done enough bad things to Solr (directly or indirectly), and I 
> don't want to repeat the same. Also, I'll be very uncomfortable if someone 
> else did so.
>
> Having said this, if someone else wants to port the changes over to master 
> *without first getting enough real world testing*, feel free to do so, and I 
> can focus my efforts elsewhere.
>
> On Tue, 6 Oct, 2020, 9:22 am Tomás Fernández Löbbe,  
> wrote:
>>
>> I was thinking (and I haven’t flushed it out completely but will throw the 
>> idea) that an alternative approach with this timeline could be to cut 9x 
>> branch around November/December? And then you could merge into master, it 
>> would have the latest  changes from master plus the ref branch changes. From 
>> there any nightly build could be use to help test/debug.
>>
>> That said I don’t know for sure what are the changes in the branch that do 
>> not belong in 9. The problem with them being 10x only is that backports 
>> would potentially be more difficult for all the life of 9.
>>
>> On Mon, Oct 5, 2020 at 4:54 PM Noble Paul  wrote:
>>>
>>> >I don't think it can be said what committers do and don't do with regards 
>>> >to running Solr.  All of us would answer this differently and at different 
>>> >points in time.
>>>
>>> " I have run it in one large cluster, so it is certified to be bug 
>>> free/stable" I don't think it's a reasonable approach. We need as much 
>>> feedback from our users because each of them stress Solr in a different 
>>> way. This is not to suggest that committers are not doing testing or their 
>>> tests are not valid. When I talk to the committers out here they say they 
>>> do not see any performance stability issues at all. But, my client reports 
>>> issues on a day to day basis.
>>>
>>>
>>>
>>> > Definitely publish a Docker image BTW -- it's the best way to try out any 
>>> > software.
>>>
>>> Docker is not a big requirement for large scale installations. Most of them 
>>> already have their own install scripts. Availability of docker is not 
>>> important for them. If a user is only encouraged to install Solr because of 
>>> a docker image , most likely they are not running a large enough cluster
>>>
>>>
>>>
>>> On Tue, Oct 6, 2020, 6:30 AM David Smiley  wrote:

 Thanks so much for your responses Ishan... I'm getting much more 
 information in this thread than my attempts to get questions answered on 
 the JIRA issue months ago.  And especially,  thank you for volunteering 
 for the difficult porting efforts!

 Tomas said:
>
>  I do agree with the previous comments that calling it "Solr 10" (even 
> with the "-alpha") would confuse users, maybe use "reference"? or maybe 
> something in reference to SOLR-14788?


 I have the opposite opinion.  This word "reference" is baffling to me 
 despite 

Re: Solr Alpha (EA) release of Reference Branch

[Ishan Chattopadhyaya]

As I said, I'm *personally* not confident in putting such a big changeset
into master that wasn't vetted in a real user environment widely. I have,
in the past, done enough bad things to Solr (directly or indirectly), and I
don't want to repeat the same. Also, I'll be very uncomfortable if someone
else did so.

Having said this, if someone else wants to port the changes over to master
*without first getting enough real world testing*, feel free to do so, and
I can focus my efforts elsewhere.

On Tue, 6 Oct, 2020, 9:22 am Tomás Fernández Löbbe, 
wrote:

> I was thinking (and I haven’t flushed it out completely but will throw the
> idea) that an alternative approach with this timeline could be to cut 9x
> branch around November/December? And then you could merge into master, it
> would have the latest  changes from master plus the ref branch changes.
> From there any nightly build could be use to help test/debug.
>
> That said I don’t know for sure what are the changes in the branch that do
> not belong in 9. The problem with them being 10x only is that backports
> would potentially be more difficult for all the life of 9.
>
> On Mon, Oct 5, 2020 at 4:54 PM Noble Paul  wrote:
>
>> >I don't think it can be said what committers do and don't do with
>> regards to running Solr.  All of us would answer this differently and at
>> different points in time.
>>
>> " I have run it in one large cluster, so it is certified to be bug
>> free/stable" I don't think it's a reasonable approach. We need as much
>> feedback from our users because each of them stress Solr in a
>> different way. This is not to suggest that committers are not doing testing
>> or their tests are not valid. When I talk to the committers out here they
>> say they do not see any performance stability issues at all. But, my client
>> reports issues on a day to day basis.
>>
>>
>>
>> > Definitely publish a Docker image BTW -- it's the best way to try out
>> any software.
>>
>> Docker is not a big requirement for large scale installations. Most of
>> them already have their own install scripts. Availability of docker is not
>> important for them. If a user is only encouraged to install Solr because of
>> a docker image , most likely they are not running a large enough cluster
>>
>>
>>
>> On Tue, Oct 6, 2020, 6:30 AM David Smiley  wrote:
>>
>>> Thanks so much for your responses Ishan... I'm getting much more
>>> information in this thread than my attempts to get questions answered on
>>> the JIRA issue months ago.  And especially,  thank you for volunteering for
>>> the difficult porting efforts!
>>>
>>> Tomas said:
>>>
  I do agree with the previous comments that calling it "Solr 10" (even
 with the "-alpha") would confuse users, maybe use "reference"? or maybe
 something in reference to SOLR-14788?

>>>
>>> I have the opposite opinion.  This word "reference" is baffling to me
>>> despite whatever Mark's explanation is.  I like the justification Ishan
>>> gave for 10-alpha and I don't think I could re-phrase his justification any
>>> better.  *If* the release was _not_ official (thus wouldn't show up in the
>>> usual places anyone would look for a release), I think it would alleviate
>>> that confusion concern even more, although I think "alpha" ought to be
>>> enough of a signal not to use it without digging deeper on what's going on.
>>>
>>> Alex then Ishan said:
>>>
 > Maybe we could release it to
 > committers community first and dogfood it "internally"?

 Alex: It is meaningless. Committers don't run large scale
 installations. We barely even have time to take care of running unit tests
 before destabilizing our builds. We are not the right audience. However, we
 all can anyway check out the branch and start playing with it, even without
 a release. There are orgs that don't want to install any code that wasn't
 officially released; this release is geared towards them (to help us test
 this at their scale).

>>>
>>> I don't think it can be said what committers do and don't do with
>>> regards to running Solr.  All of us would answer this differently and at
>>> different points in time.  From time to time, though not at present, I've
>>> been well positioned to try out a new version of Solr in a stage/test
>>> environment to see how it goes.  (Putting on my Salesforce metaphorical
>>> hat...) Even though I'm not able to deploy it in a realistic way today, I'm
>>> able to run a battery of tests to see if one of the features we depend on
>>> have changed or is broken.  That's useful feedback to an alpha release!
>>> And even though I'm saying I'm not well positioned to try out some new Solr
>>> release in a production-ish setting now, it's something I could make a good
>>> case for internally since upgrades take a lot of effort where I work.  It's
>>> in our interest for SolrCloud to be very stable (of course).
>>>
>>> Regardless, I think what you're driving at Ishan is that you want an
>>> 

Re: Solr Alpha (EA) release of Reference Branch

[Tomás Fernández Löbbe]

I was thinking (and I haven’t flushed it out completely but will throw the
idea) that an alternative approach with this timeline could be to cut 9x
branch around November/December? And then you could merge into master, it
would have the latest  changes from master plus the ref branch changes.
>From there any nightly build could be use to help test/debug.

That said I don’t know for sure what are the changes in the branch that do
not belong in 9. The problem with them being 10x only is that backports
would potentially be more difficult for all the life of 9.

On Mon, Oct 5, 2020 at 4:54 PM Noble Paul  wrote:

> >I don't think it can be said what committers do and don't do with regards
> to running Solr.  All of us would answer this differently and at different
> points in time.
>
> " I have run it in one large cluster, so it is certified to be bug
> free/stable" I don't think it's a reasonable approach. We need as much
> feedback from our users because each of them stress Solr in a
> different way. This is not to suggest that committers are not doing testing
> or their tests are not valid. When I talk to the committers out here they
> say they do not see any performance stability issues at all. But, my client
> reports issues on a day to day basis.
>
>
>
> > Definitely publish a Docker image BTW -- it's the best way to try out
> any software.
>
> Docker is not a big requirement for large scale installations. Most of
> them already have their own install scripts. Availability of docker is not
> important for them. If a user is only encouraged to install Solr because of
> a docker image , most likely they are not running a large enough cluster
>
>
>
> On Tue, Oct 6, 2020, 6:30 AM David Smiley  wrote:
>
>> Thanks so much for your responses Ishan... I'm getting much more
>> information in this thread than my attempts to get questions answered on
>> the JIRA issue months ago.  And especially,  thank you for volunteering for
>> the difficult porting efforts!
>>
>> Tomas said:
>>
>>>  I do agree with the previous comments that calling it "Solr 10" (even
>>> with the "-alpha") would confuse users, maybe use "reference"? or maybe
>>> something in reference to SOLR-14788?
>>>
>>
>> I have the opposite opinion.  This word "reference" is baffling to me
>> despite whatever Mark's explanation is.  I like the justification Ishan
>> gave for 10-alpha and I don't think I could re-phrase his justification any
>> better.  *If* the release was _not_ official (thus wouldn't show up in the
>> usual places anyone would look for a release), I think it would alleviate
>> that confusion concern even more, although I think "alpha" ought to be
>> enough of a signal not to use it without digging deeper on what's going on.
>>
>> Alex then Ishan said:
>>
>>> > Maybe we could release it to
>>> > committers community first and dogfood it "internally"?
>>>
>>> Alex: It is meaningless. Committers don't run large scale installations.
>>> We barely even have time to take care of running unit tests before
>>> destabilizing our builds. We are not the right audience. However, we all
>>> can anyway check out the branch and start playing with it, even without a
>>> release. There are orgs that don't want to install any code that wasn't
>>> officially released; this release is geared towards them (to help us test
>>> this at their scale).
>>>
>>
>> I don't think it can be said what committers do and don't do with regards
>> to running Solr.  All of us would answer this differently and at different
>> points in time.  From time to time, though not at present, I've been well
>> positioned to try out a new version of Solr in a stage/test environment to
>> see how it goes.  (Putting on my Salesforce metaphorical hat...) Even
>> though I'm not able to deploy it in a realistic way today, I'm able to run
>> a battery of tests to see if one of the features we depend on have changed
>> or is broken.  That's useful feedback to an alpha release!  And even though
>> I'm saying I'm not well positioned to try out some new Solr release in a
>> production-ish setting now, it's something I could make a good case for
>> internally since upgrades take a lot of effort where I work.  It's in our
>> interest for SolrCloud to be very stable (of course).
>>
>> Regardless, I think what you're driving at Ishan is that you want an
>> "official" release -- one that goes through the whole ceremony.  You
>> believe that people would be more likely to use it.  I think all we need to
>> do is announce (similar to a real release) that there is some unofficial
>> alpha distribution and that we want to solicit your feedback -- basically,
>> help us find bugs.  Definitely publish a Docker image BTW -- it's the best
>> way to try out any software.  I'm -0 on doing an official release for alpha
>> software because it's unnecessary to achieve the goals and somewhat
>> confusing.  I think the Solr 4 alpha/beta situation was different -- it was
>> not some fork a committer was 

Re: Solr Alpha (EA) release of Reference Branch

[Noble Paul]

>I don't think it can be said what committers do and don't do with regards
to running Solr.  All of us would answer this differently and at different
points in time.

" I have run it in one large cluster, so it is certified to be bug
free/stable" I don't think it's a reasonable approach. We need as much
feedback from our users because each of them stress Solr in a
different way. This is not to suggest that committers are not doing testing
or their tests are not valid. When I talk to the committers out here they
say they do not see any performance stability issues at all. But, my client
reports issues on a day to day basis.



> Definitely publish a Docker image BTW -- it's the best way to try out any
software.

Docker is not a big requirement for large scale installations. Most of them
already have their own install scripts. Availability of docker is not
important for them. If a user is only encouraged to install Solr because of
a docker image , most likely they are not running a large enough cluster



On Tue, Oct 6, 2020, 6:30 AM David Smiley  wrote:

> Thanks so much for your responses Ishan... I'm getting much more
> information in this thread than my attempts to get questions answered on
> the JIRA issue months ago.  And especially,  thank you for volunteering for
> the difficult porting efforts!
>
> Tomas said:
>
>>  I do agree with the previous comments that calling it "Solr 10" (even
>> with the "-alpha") would confuse users, maybe use "reference"? or maybe
>> something in reference to SOLR-14788?
>>
>
> I have the opposite opinion.  This word "reference" is baffling to me
> despite whatever Mark's explanation is.  I like the justification Ishan
> gave for 10-alpha and I don't think I could re-phrase his justification any
> better.  *If* the release was _not_ official (thus wouldn't show up in the
> usual places anyone would look for a release), I think it would alleviate
> that confusion concern even more, although I think "alpha" ought to be
> enough of a signal not to use it without digging deeper on what's going on.
>
> Alex then Ishan said:
>
>> > Maybe we could release it to
>> > committers community first and dogfood it "internally"?
>>
>> Alex: It is meaningless. Committers don't run large scale installations.
>> We barely even have time to take care of running unit tests before
>> destabilizing our builds. We are not the right audience. However, we all
>> can anyway check out the branch and start playing with it, even without a
>> release. There are orgs that don't want to install any code that wasn't
>> officially released; this release is geared towards them (to help us test
>> this at their scale).
>>
>
> I don't think it can be said what committers do and don't do with regards
> to running Solr.  All of us would answer this differently and at different
> points in time.  From time to time, though not at present, I've been well
> positioned to try out a new version of Solr in a stage/test environment to
> see how it goes.  (Putting on my Salesforce metaphorical hat...) Even
> though I'm not able to deploy it in a realistic way today, I'm able to run
> a battery of tests to see if one of the features we depend on have changed
> or is broken.  That's useful feedback to an alpha release!  And even though
> I'm saying I'm not well positioned to try out some new Solr release in a
> production-ish setting now, it's something I could make a good case for
> internally since upgrades take a lot of effort where I work.  It's in our
> interest for SolrCloud to be very stable (of course).
>
> Regardless, I think what you're driving at Ishan is that you want an
> "official" release -- one that goes through the whole ceremony.  You
> believe that people would be more likely to use it.  I think all we need to
> do is announce (similar to a real release) that there is some unofficial
> alpha distribution and that we want to solicit your feedback -- basically,
> help us find bugs.  Definitely publish a Docker image BTW -- it's the best
> way to try out any software.  I'm -0 on doing an official release for alpha
> software because it's unnecessary to achieve the goals and somewhat
> confusing.  I think the Solr 4 alpha/beta situation was different -- it was
> not some fork a committer was maintaining; it was the master branch of its
> time, and it was destined to be the very next release, not some possible
> future release.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>

Re: Solr Alpha (EA) release of Reference Branch

[Tomás Fernández Löbbe]

> I think the Solr 4 alpha/beta situation was different -- it was not some
fork a committer was maintaining; it was the master branch of its time, and
it was destined to be the very next release, not some possible future
release.

Agree, 4’s alpha/beta was a very different situation.

> I believe it has to be an official release to have enough credibility.
People trust the Apache brand and the community. This will ensure that we
get enough people to test this out. The very objective of this release is
to get help from our users to uncover any bugs. Most big shops will not
deploy unofficial releases in their prod/staging environments. We wish to
tick all the boxes for our users

I think this is fooling ourselves/our users. They trust Apache releases
because we take them seriously, because they have the community behind,
etc. This is a release from a feature branch, so we have to be cautious and
very upfront. While I hope many go and test it and maybe deploy it to some
of their environments, I feel we need to be careful not to send the wrong
message.

> I'm -0 on doing an official release for alpha software because it's
unnecessary to achieve the goals and somewhat confusing

I’d say I’m -1 to make it an official release for those reasons.

I think we should try to merge into master (whenever you are comfortable?
Does it need to wait until 9.0 is out? What's the main reason for that? can
we split the parts that do need to wait vs the ones that don’t?) and then
people can be encouraged to run a nightly version in their test
environments to help debug possible instability. If we need to do
alpha/beta versions from master then I think that would make more sense.



On Mon, Oct 5, 2020 at 12:30 PM David Smiley  wrote:

> Thanks so much for your responses Ishan... I'm getting much more
> information in this thread than my attempts to get questions answered on
> the JIRA issue months ago.  And especially,  thank you for volunteering for
> the difficult porting efforts!
>
> Tomas said:
>
>>  I do agree with the previous comments that calling it "Solr 10" (even
>> with the "-alpha") would confuse users, maybe use "reference"? or maybe
>> something in reference to SOLR-14788?
>>
>
> I have the opposite opinion.  This word "reference" is baffling to me
> despite whatever Mark's explanation is.  I like the justification Ishan
> gave for 10-alpha and I don't think I could re-phrase his justification any
> better.  *If* the release was _not_ official (thus wouldn't show up in the
> usual places anyone would look for a release), I think it would alleviate
> that confusion concern even more, although I think "alpha" ought to be
> enough of a signal not to use it without digging deeper on what's going on.
>
> Alex then Ishan said:
>
>> > Maybe we could release it to
>> > committers community first and dogfood it "internally"?
>>
>> Alex: It is meaningless. Committers don't run large scale installations.
>> We barely even have time to take care of running unit tests before
>> destabilizing our builds. We are not the right audience. However, we all
>> can anyway check out the branch and start playing with it, even without a
>> release. There are orgs that don't want to install any code that wasn't
>> officially released; this release is geared towards them (to help us test
>> this at their scale).
>>
>
> I don't think it can be said what committers do and don't do with regards
> to running Solr.  All of us would answer this differently and at different
> points in time.  From time to time, though not at present, I've been well
> positioned to try out a new version of Solr in a stage/test environment to
> see how it goes.  (Putting on my Salesforce metaphorical hat...) Even
> though I'm not able to deploy it in a realistic way today, I'm able to run
> a battery of tests to see if one of the features we depend on have changed
> or is broken.  That's useful feedback to an alpha release!  And even though
> I'm saying I'm not well positioned to try out some new Solr release in a
> production-ish setting now, it's something I could make a good case for
> internally since upgrades take a lot of effort where I work.  It's in our
> interest for SolrCloud to be very stable (of course).
>
> Regardless, I think what you're driving at Ishan is that you want an
> "official" release -- one that goes through the whole ceremony.  You
> believe that people would be more likely to use it.  I think all we need to
> do is announce (similar to a real release) that there is some unofficial
> alpha distribution and that we want to solicit your feedback -- basically,
> help us find bugs.  Definitely publish a Docker image BTW -- it's the best
> way to try out any software.  I'm -0 on doing an official release for alpha
> software because it's unnecessary to achieve the goals and somewhat
> confusing.  I think the Solr 4 alpha/beta situation was different -- it was
> not some fork a committer was maintaining; it was the master branch of its
> time, 

Re: Solr Alpha (EA) release of Reference Branch

[David Smiley]

Thanks so much for your responses Ishan... I'm getting much more
information in this thread than my attempts to get questions answered on
the JIRA issue months ago.  And especially,  thank you for volunteering for
the difficult porting efforts!

Tomas said:

>  I do agree with the previous comments that calling it "Solr 10" (even
> with the "-alpha") would confuse users, maybe use "reference"? or maybe
> something in reference to SOLR-14788?
>

I have the opposite opinion.  This word "reference" is baffling to me
despite whatever Mark's explanation is.  I like the justification Ishan
gave for 10-alpha and I don't think I could re-phrase his justification any
better.  *If* the release was _not_ official (thus wouldn't show up in the
usual places anyone would look for a release), I think it would alleviate
that confusion concern even more, although I think "alpha" ought to be
enough of a signal not to use it without digging deeper on what's going on.

Alex then Ishan said:

> > Maybe we could release it to
> > committers community first and dogfood it "internally"?
>
> Alex: It is meaningless. Committers don't run large scale installations.
> We barely even have time to take care of running unit tests before
> destabilizing our builds. We are not the right audience. However, we all
> can anyway check out the branch and start playing with it, even without a
> release. There are orgs that don't want to install any code that wasn't
> officially released; this release is geared towards them (to help us test
> this at their scale).
>

I don't think it can be said what committers do and don't do with regards
to running Solr.  All of us would answer this differently and at different
points in time.  From time to time, though not at present, I've been well
positioned to try out a new version of Solr in a stage/test environment to
see how it goes.  (Putting on my Salesforce metaphorical hat...) Even
though I'm not able to deploy it in a realistic way today, I'm able to run
a battery of tests to see if one of the features we depend on have changed
or is broken.  That's useful feedback to an alpha release!  And even though
I'm saying I'm not well positioned to try out some new Solr release in a
production-ish setting now, it's something I could make a good case for
internally since upgrades take a lot of effort where I work.  It's in our
interest for SolrCloud to be very stable (of course).

Regardless, I think what you're driving at Ishan is that you want an
"official" release -- one that goes through the whole ceremony.  You
believe that people would be more likely to use it.  I think all we need to
do is announce (similar to a real release) that there is some unofficial
alpha distribution and that we want to solicit your feedback -- basically,
help us find bugs.  Definitely publish a Docker image BTW -- it's the best
way to try out any software.  I'm -0 on doing an official release for alpha
software because it's unnecessary to achieve the goals and somewhat
confusing.  I think the Solr 4 alpha/beta situation was different -- it was
not some fork a committer was maintaining; it was the master branch of its
time, and it was destined to be the very next release, not some possible
future release.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley

Re: Solr Alpha (EA) release of Reference Branch

[Ishan Chattopadhyaya]

To conclude this thread, seems that there's general consensus for a release
around early December timeframe. If there are any outstanding concerns
against the release itself, please chime in.

The naming is something we can discuss separately, as we go along. State of
the branch can be re-evaluated at the time of the release to see if we need
to delay it, it still makes sense to go with the release, or any major
shifts in direction is needed.

Thanks everyone for your suggestions, thoughts and concerns.

On Mon, 5 Oct, 2020, 6:43 pm Ishan Chattopadhyaya, <
ichattopadhy...@gmail.com> wrote:

> IIUC, reference_impl is the branch where the changes are stable. *_dev is
> the branch where active development is going on. Once changes are baked in
> there, they are promoted to reference_impl. We want to release from
> reference_impl.
>
> On Mon, 5 Oct, 2020, 6:16 pm Erick Erickson, 
> wrote:
>
>> Uwe:
>>
>> I think it’s reference_impl_dev...
>>
>> > On Oct 4, 2020, at 6:00 PM, Uwe Schindler  wrote:
>> >
>> > Hi,
>> > I enabled the „gradlew check” runs on ASF Jenkins and Policeman Jenkins
>> (Linux, Windows, MacOS).
>> >
>> > I used branch (“reference_impl”). Is this correct, because it’s about a
>> month old?
>> > There’s also a much newer “reference_impl_dev” branch. Which one is
>> correct?
>> >
>> > I will go sleeping now, sorry for failure mails during the night.
>> Builds seem to fail, but it’s too late to do anything against it.
>> >
>> > Uwe
>> >
>> > -
>> > Uwe Schindler
>> > Achterdiek 19, D-28357 Bremen
>> > https://www.thetaphi.de
>> > eMail: u...@thetaphi.de
>> >
>> > From: Ishan Chattopadhyaya 
>> > Sent: Sunday, October 4, 2020 6:32 AM
>> > To: Uwe Schindler 
>> > Cc: Lucene Dev 
>> > Subject: Re: Solr Alpha (EA) release of Reference Branch
>> >
>> > Yes Uwe, it is ready for Jenkins testing. The Solr tests should run
>> very fast now.
>> >
>> > As for naming, our package manager in Solr will break if we don't
>> specify a major and a minor version. There's a concept of version
>> constraints for packages that need to compare against Solr version. I think
>> release process will also be much simpler if we have a major and minor
>> version. We have done a similar release in the past, 4.0.0-beta iirc.
>> >
>> > Why I favour 10.0-alpha or something like that is because users would
>> clearly know it is something that isn't coming right now (and hence very
>> early access), and it is logically a major version change (that comes after
>> 8x). With calling it 10x instead of 9x, we have the scope of abandoning the
>> effort as a whole if the early access releases have some serious problem or
>> we decide to take some other release strategy later.
>> >
>> > If the 10.0-alpha succeeds, we can always fold the changes back into a
>> 9.1 or go from 9.0 straight to 10.0, depending on what we decide later.
>> Alternatively, if the release looks good and 9.0 hasn't released yet, we
>> can fold those changes back to master, and either (a) release everything
>> normally as 9.0, or (b) call master as 10x and release from there (going
>> from 8.8 to 10.0 directly, skipping 9x altogether).
>> >
>> > Tldr, we shall have complete flexibility to go in any direction we want
>> to.
>> >
>> > WDYT?
>> >
>> > On Sun, 4 Oct, 2020, 2:31 am Uwe Schindler,  wrote:
>> >> Is the branch ready for Jenkins testing?
>> >>
>> >> If yes and "gradlew check" works, I really would like to set it up.
>> >>
>> >> Uwe
>> >>
>> >> Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com>:
>> >>> Hi Devs,
>> >>>
>> >>> As you might be aware, the reference_impl branch has a lot of
>> improvements that we want to see in Solr master. However, it is currently a
>> large deviation from master and hence the stability and reliability (though
>> improved in certain aspects) remains to be tested in real production
>> environments before we gain confidence in bringing those changes to master.
>> >>>
>> >>> I propose that we do a one off preview release from that branch, say
>> Solr 10 alpha (early access) or any other name that someone suggests, so
>> that users could try it out and report regressions or improvements etc.
>> >>>
>> >>> I volunteer to be the RM and planning to start the process around 1
>> December-15 December timeframe. Until then, we can tighten the loose ends
>> on the branch and plan for such a release.
>> >>>
>> >>> Is there any thoughts, concerns, questions?
>> >>>
>> >>> Regards,
>> >>> Ishan
>> >>
>> >> --
>> >> Uwe Schindler
>> >> Achterdiek 19, 28357 Bremen
>> >> https://www.thetaphi.de
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>

Re: Solr Alpha (EA) release of Reference Branch

[Ishan Chattopadhyaya]

IIUC, reference_impl is the branch where the changes are stable. *_dev is
the branch where active development is going on. Once changes are baked in
there, they are promoted to reference_impl. We want to release from
reference_impl.

On Mon, 5 Oct, 2020, 6:16 pm Erick Erickson, 
wrote:

> Uwe:
>
> I think it’s reference_impl_dev...
>
> > On Oct 4, 2020, at 6:00 PM, Uwe Schindler  wrote:
> >
> > Hi,
> > I enabled the „gradlew check” runs on ASF Jenkins and Policeman Jenkins
> (Linux, Windows, MacOS).
> >
> > I used branch (“reference_impl”). Is this correct, because it’s about a
> month old?
> > There’s also a much newer “reference_impl_dev” branch. Which one is
> correct?
> >
> > I will go sleeping now, sorry for failure mails during the night. Builds
> seem to fail, but it’s too late to do anything against it.
> >
> > Uwe
> >
> > -
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > https://www.thetaphi.de
> > eMail: u...@thetaphi.de
> >
> > From: Ishan Chattopadhyaya 
> > Sent: Sunday, October 4, 2020 6:32 AM
> > To: Uwe Schindler 
> > Cc: Lucene Dev 
> > Subject: Re: Solr Alpha (EA) release of Reference Branch
> >
> > Yes Uwe, it is ready for Jenkins testing. The Solr tests should run very
> fast now.
> >
> > As for naming, our package manager in Solr will break if we don't
> specify a major and a minor version. There's a concept of version
> constraints for packages that need to compare against Solr version. I think
> release process will also be much simpler if we have a major and minor
> version. We have done a similar release in the past, 4.0.0-beta iirc.
> >
> > Why I favour 10.0-alpha or something like that is because users would
> clearly know it is something that isn't coming right now (and hence very
> early access), and it is logically a major version change (that comes after
> 8x). With calling it 10x instead of 9x, we have the scope of abandoning the
> effort as a whole if the early access releases have some serious problem or
> we decide to take some other release strategy later.
> >
> > If the 10.0-alpha succeeds, we can always fold the changes back into a
> 9.1 or go from 9.0 straight to 10.0, depending on what we decide later.
> Alternatively, if the release looks good and 9.0 hasn't released yet, we
> can fold those changes back to master, and either (a) release everything
> normally as 9.0, or (b) call master as 10x and release from there (going
> from 8.8 to 10.0 directly, skipping 9x altogether).
> >
> > Tldr, we shall have complete flexibility to go in any direction we want
> to.
> >
> > WDYT?
> >
> > On Sun, 4 Oct, 2020, 2:31 am Uwe Schindler,  wrote:
> >> Is the branch ready for Jenkins testing?
> >>
> >> If yes and "gradlew check" works, I really would like to set it up.
> >>
> >> Uwe
> >>
> >> Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com>:
> >>> Hi Devs,
> >>>
> >>> As you might be aware, the reference_impl branch has a lot of
> improvements that we want to see in Solr master. However, it is currently a
> large deviation from master and hence the stability and reliability (though
> improved in certain aspects) remains to be tested in real production
> environments before we gain confidence in bringing those changes to master.
> >>>
> >>> I propose that we do a one off preview release from that branch, say
> Solr 10 alpha (early access) or any other name that someone suggests, so
> that users could try it out and report regressions or improvements etc.
> >>>
> >>> I volunteer to be the RM and planning to start the process around 1
> December-15 December timeframe. Until then, we can tighten the loose ends
> on the branch and plan for such a release.
> >>>
> >>> Is there any thoughts, concerns, questions?
> >>>
> >>> Regards,
> >>> Ishan
> >>
> >> --
> >> Uwe Schindler
> >> Achterdiek 19, 28357 Bremen
> >> https://www.thetaphi.de
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Solr Alpha (EA) release of Reference Branch

[Erick Erickson]

Uwe:

I think it’s reference_impl_dev...

> On Oct 4, 2020, at 6:00 PM, Uwe Schindler  wrote:
> 
> Hi,
> I enabled the „gradlew check” runs on ASF Jenkins and Policeman Jenkins 
> (Linux, Windows, MacOS).
>  
> I used branch (“reference_impl”). Is this correct, because it’s about a month 
> old?
> There’s also a much newer “reference_impl_dev” branch. Which one is correct?
>  
> I will go sleeping now, sorry for failure mails during the night. Builds seem 
> to fail, but it’s too late to do anything against it.
>  
> Uwe
>  
> -
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: u...@thetaphi.de
>  
> From: Ishan Chattopadhyaya  
> Sent: Sunday, October 4, 2020 6:32 AM
> To: Uwe Schindler 
> Cc: Lucene Dev 
> Subject: Re: Solr Alpha (EA) release of Reference Branch
>  
> Yes Uwe, it is ready for Jenkins testing. The Solr tests should run very fast 
> now.
>  
> As for naming, our package manager in Solr will break if we don't specify a 
> major and a minor version. There's a concept of version constraints for 
> packages that need to compare against Solr version. I think release process 
> will also be much simpler if we have a major and minor version. We have done 
> a similar release in the past, 4.0.0-beta iirc.
>  
> Why I favour 10.0-alpha or something like that is because users would clearly 
> know it is something that isn't coming right now (and hence very early 
> access), and it is logically a major version change (that comes after 8x). 
> With calling it 10x instead of 9x, we have the scope of abandoning the effort 
> as a whole if the early access releases have some serious problem or we 
> decide to take some other release strategy later.
>  
> If the 10.0-alpha succeeds, we can always fold the changes back into a 9.1 or 
> go from 9.0 straight to 10.0, depending on what we decide later. 
> Alternatively, if the release looks good and 9.0 hasn't released yet, we can 
> fold those changes back to master, and either (a) release everything normally 
> as 9.0, or (b) call master as 10x and release from there (going from 8.8 to 
> 10.0 directly, skipping 9x altogether).
>  
> Tldr, we shall have complete flexibility to go in any direction we want to.
>  
> WDYT?
>  
> On Sun, 4 Oct, 2020, 2:31 am Uwe Schindler,  wrote:
>> Is the branch ready for Jenkins testing?
>> 
>> If yes and "gradlew check" works, I really would like to set it up.
>> 
>> Uwe
>> 
>> Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya 
>> :
>>> Hi Devs,
>>>  
>>> As you might be aware, the reference_impl branch has a lot of improvements 
>>> that we want to see in Solr master. However, it is currently a large 
>>> deviation from master and hence the stability and reliability (though 
>>> improved in certain aspects) remains to be tested in real production 
>>> environments before we gain confidence in bringing those changes to master.
>>>  
>>> I propose that we do a one off preview release from that branch, say Solr 
>>> 10 alpha (early access) or any other name that someone suggests, so that 
>>> users could try it out and report regressions or improvements etc.
>>>  
>>> I volunteer to be the RM and planning to start the process around 1 
>>> December-15 December timeframe. Until then, we can tighten the loose ends 
>>> on the branch and plan for such a release.
>>>  
>>> Is there any thoughts, concerns, questions?
>>>  
>>> Regards,
>>> Ishan
>> 
>> --
>> Uwe Schindler
>> Achterdiek 19, 28357 Bremen
>> https://www.thetaphi.de


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr Alpha (EA) release of Reference Branch

[Shawn Heisey]

On 10/3/2020 1:42 PM, Ishan Chattopadhyaya wrote:
As you might be aware, the reference_impl branch has a lot of 
improvements that we want to see in Solr master. However, it is 
currently a large deviation from master and hence the stability and 
reliability (though improved in certain aspects) remains to be tested in 
real production environments before we gain confidence in bringing those 
changes to master.


I propose that we do a one off preview release from that branch, say 
Solr 10 alpha (early access) or any other name that someone suggests, so 
that users could try it out and report regressions or improvements etc.


How to handle this seems to come down to the answers to a couple of 
questions:


* Is this new code stable enough to work in the wild?
* Do we want to release 9.0 before we merge to master, or after?

Can someone who was involved when 4.0-ALPHA and 4.0-BETA were released 
tell me whether those releases actually did what was intended and made 
4.0.0 better than it would have been without the special releases?  If 
they did, then a special release for this new implementation before 
merging to master would probably also be helpful.  If there was no real 
benefit gained, then maybe we're better off just going ahead with the merge.


If the general feeling is that this new release is looking very solid, 
then I think we should merge to master soon, probably just before 
branch_9x is created.  If we think it needs more work, then maybe we 
should hold on merging until *after* branch_9x is created, so the new 
implementation will release with 10.0 and there will be more time to 
make it bulletproof.


My current impression, which I will admit is made with almost zero 
actual information about the code or its stability, is that we should 
plan on stabilizing the new implementation for the 9.0 release.  There's 
going to be some pain no matter how we handle this, so diving in now 
seems like a good idea.


A tangent:  How robust is our testing?  I know they take a long time to 
run, which Mark's new implementation aims to improve, but do we think 
there's enough being tested?  There has been some discussion in the past 
about benchmarks for Solr.  Benchmarks would be very useful, but I'm 
more interested in making sure that our tests will catch problems before 
they get out into the wild.  The two goals don't have to be mutually 
exclusive, though.


Thanks,
Shawn

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr Alpha (EA) release of Reference Branch

[Noble Paul]

@Tomas Fernandez Lobbe


The naming is up for debate and it doesn't matter at this point in
time. I believe it has to be an official release to have enough
credibility. People trust the Apache brand and the community. This
will ensure that we get enough people to test this out. The very
objective of this release is to get help from our users to uncover any
bugs. Most big shops will not deploy unofficial releases in their
prod/staging environments. We wish to tick all the boxes for our users

On Mon, Oct 5, 2020 at 9:14 AM Uwe Schindler  wrote:
>
> Hi,
>
>
>
> In addition the gradle build scripts in the reference_impl branch seems very 
> outdated and does not support Policeman-Jenkins Multi-JVM testing. Although 
> it will print that it ran with later Java version, in fact it runs with JDK 
> 11 only, as RUNTIME_JAVA_HOME is ignored.
>
>
>
> Uwe
>
>
>
> -
>
> Uwe Schindler
>
> Achterdiek 19, D-28357 Bremen
>
> https://www.thetaphi.de
>
> eMail: u...@thetaphi.de
>
>
>
> From: Uwe Schindler 
> Sent: Monday, October 5, 2020 12:01 AM
> To: 'Ishan Chattopadhyaya' 
> Cc: 'Lucene Dev' 
> Subject: RE: Solr Alpha (EA) release of Reference Branch
>
>
>
> Hi,
>
> I enabled the „gradlew check” runs on ASF Jenkins and Policeman Jenkins 
> (Linux, Windows, MacOS).
>
>
>
> I used branch (“reference_impl”). Is this correct, because it’s about a month 
> old?
>
> There’s also a much newer “reference_impl_dev” branch. Which one is correct?
>
>
>
> I will go sleeping now, sorry for failure mails during the night. Builds seem 
> to fail, but it’s too late to do anything against it.
>
>
>
> Uwe
>
>
>
> -
>
> Uwe Schindler
>
> Achterdiek 19, D-28357 Bremen
>
> https://www.thetaphi.de
>
> eMail: u...@thetaphi.de
>
>
>
> From: Ishan Chattopadhyaya 
> Sent: Sunday, October 4, 2020 6:32 AM
> To: Uwe Schindler 
> Cc: Lucene Dev 
> Subject: Re: Solr Alpha (EA) release of Reference Branch
>
>
>
> Yes Uwe, it is ready for Jenkins testing. The Solr tests should run very fast 
> now.
>
>
>
> As for naming, our package manager in Solr will break if we don't specify a 
> major and a minor version. There's a concept of version constraints for 
> packages that need to compare against Solr version. I think release process 
> will also be much simpler if we have a major and minor version. We have done 
> a similar release in the past, 4.0.0-beta iirc.
>
>
>
> Why I favour 10.0-alpha or something like that is because users would clearly 
> know it is something that isn't coming right now (and hence very early 
> access), and it is logically a major version change (that comes after 8x). 
> With calling it 10x instead of 9x, we have the scope of abandoning the effort 
> as a whole if the early access releases have some serious problem or we 
> decide to take some other release strategy later.
>
>
>
> If the 10.0-alpha succeeds, we can always fold the changes back into a 9.1 or 
> go from 9.0 straight to 10.0, depending on what we decide later. 
> Alternatively, if the release looks good and 9.0 hasn't released yet, we can 
> fold those changes back to master, and either (a) release everything normally 
> as 9.0, or (b) call master as 10x and release from there (going from 8.8 to 
> 10.0 directly, skipping 9x altogether).
>
>
>
> Tldr, we shall have complete flexibility to go in any direction we want to.
>
>
>
> WDYT?
>
>
>
> On Sun, 4 Oct, 2020, 2:31 am Uwe Schindler,  wrote:
>
> Is the branch ready for Jenkins testing?
>
> If yes and "gradlew check" works, I really would like to set it up.
>
> Uwe
>
> Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya 
> :
>
> Hi Devs,
>
>
>
> As you might be aware, the reference_impl branch has a lot of improvements 
> that we want to see in Solr master. However, it is currently a large 
> deviation from master and hence the stability and reliability (though 
> improved in certain aspects) remains to be tested in real production 
> environments before we gain confidence in bringing those changes to master.
>
>
>
> I propose that we do a one off preview release from that branch, say Solr 10 
> alpha (early access) or any other name that someone suggests, so that users 
> could try it out and report regressions or improvements etc.
>
>
>
> I volunteer to be the RM and planning to start the process around 1 
> December-15 December timeframe. Until then, we can tighten the loose ends on 
> the branch and plan for such a release.
>
>
>
> Is there any thoughts, concerns, questions?
>
>
>
> Regards,
>
> Ishan
>
>
> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de



-- 
-
Noble Paul

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Solr Alpha (EA) release of Reference Branch

[Uwe Schindler]

Hi,

 

In addition the gradle build scripts in the reference_impl branch seems very 
outdated and does not support Policeman-Jenkins Multi-JVM testing. Although it 
will print that it ran with later Java version, in fact it runs with JDK 11 
only, as RUNTIME_JAVA_HOME is ignored.

 

Uwe

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Uwe Schindler  
Sent: Monday, October 5, 2020 12:01 AM
To: 'Ishan Chattopadhyaya' 
Cc: 'Lucene Dev' 
Subject: RE: Solr Alpha (EA) release of Reference Branch

 

Hi,

I enabled the „gradlew check” runs on ASF Jenkins and Policeman Jenkins (Linux, 
Windows, MacOS).

 

I used branch (“reference_impl”). Is this correct, because it’s about a month 
old?

There’s also a much newer “reference_impl_dev” branch. Which one is correct?

 

I will go sleeping now, sorry for failure mails during the night. Builds seem 
to fail, but it’s too late to do anything against it.

 

Uwe

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: u...@thetaphi.de <mailto:u...@thetaphi.de> 

 

From: Ishan Chattopadhyaya mailto:ichattopadhy...@gmail.com> > 
Sent: Sunday, October 4, 2020 6:32 AM
To: Uwe Schindler mailto:u...@thetaphi.de> >
Cc: Lucene Dev mailto:dev@lucene.apache.org> >
Subject: Re: Solr Alpha (EA) release of Reference Branch

 

Yes Uwe, it is ready for Jenkins testing. The Solr tests should run very fast 
now.

 

As for naming, our package manager in Solr will break if we don't specify a 
major and a minor version. There's a concept of version constraints for 
packages that need to compare against Solr version. I think release process 
will also be much simpler if we have a major and minor version. We have done a 
similar release in the past, 4.0.0-beta iirc.

 

Why I favour 10.0-alpha or something like that is because users would clearly 
know it is something that isn't coming right now (and hence very early access), 
and it is logically a major version change (that comes after 8x). With calling 
it 10x instead of 9x, we have the scope of abandoning the effort as a whole if 
the early access releases have some serious problem or we decide to take some 
other release strategy later.

 

If the 10.0-alpha succeeds, we can always fold the changes back into a 9.1 or 
go from 9.0 straight to 10.0, depending on what we decide later. Alternatively, 
if the release looks good and 9.0 hasn't released yet, we can fold those 
changes back to master, and either (a) release everything normally as 9.0, or 
(b) call master as 10x and release from there (going from 8.8 to 10.0 directly, 
skipping 9x altogether).

 

Tldr, we shall have complete flexibility to go in any direction we want to.

 

WDYT?

 

On Sun, 4 Oct, 2020, 2:31 am Uwe Schindler, mailto:u...@thetaphi.de> > wrote:

Is the branch ready for Jenkins testing?

If yes and "gradlew check" works, I really would like to set it up.

Uwe

Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya 
mailto:ichattopadhy...@gmail.com> >:

Hi Devs,

 

As you might be aware, the reference_impl branch has a lot of improvements that 
we want to see in Solr master. However, it is currently a large deviation from 
master and hence the stability and reliability (though improved in certain 
aspects) remains to be tested in real production environments before we gain 
confidence in bringing those changes to master.

 

I propose that we do a one off preview release from that branch, say Solr 10 
alpha (early access) or any other name that someone suggests, so that users 
could try it out and report regressions or improvements etc.

 

I volunteer to be the RM and planning to start the process around 1 December-15 
December timeframe. Until then, we can tighten the loose ends on the branch and 
plan for such a release.

 

Is there any thoughts, concerns, questions?

 

Regards,

Ishan


--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de

RE: Solr Alpha (EA) release of Reference Branch

[Uwe Schindler]

Hi,

I enabled the „gradlew check” runs on ASF Jenkins and Policeman Jenkins (Linux, 
Windows, MacOS).

 

I used branch (“reference_impl”). Is this correct, because it’s about a month 
old?

There’s also a much newer “reference_impl_dev” branch. Which one is correct?

 

I will go sleeping now, sorry for failure mails during the night. Builds seem 
to fail, but it’s too late to do anything against it.

 

Uwe

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Ishan Chattopadhyaya  
Sent: Sunday, October 4, 2020 6:32 AM
To: Uwe Schindler 
Cc: Lucene Dev 
Subject: Re: Solr Alpha (EA) release of Reference Branch

 

Yes Uwe, it is ready for Jenkins testing. The Solr tests should run very fast 
now.

 

As for naming, our package manager in Solr will break if we don't specify a 
major and a minor version. There's a concept of version constraints for 
packages that need to compare against Solr version. I think release process 
will also be much simpler if we have a major and minor version. We have done a 
similar release in the past, 4.0.0-beta iirc.

 

Why I favour 10.0-alpha or something like that is because users would clearly 
know it is something that isn't coming right now (and hence very early access), 
and it is logically a major version change (that comes after 8x). With calling 
it 10x instead of 9x, we have the scope of abandoning the effort as a whole if 
the early access releases have some serious problem or we decide to take some 
other release strategy later.

 

If the 10.0-alpha succeeds, we can always fold the changes back into a 9.1 or 
go from 9.0 straight to 10.0, depending on what we decide later. Alternatively, 
if the release looks good and 9.0 hasn't released yet, we can fold those 
changes back to master, and either (a) release everything normally as 9.0, or 
(b) call master as 10x and release from there (going from 8.8 to 10.0 directly, 
skipping 9x altogether).

 

Tldr, we shall have complete flexibility to go in any direction we want to.

 

WDYT?

 

On Sun, 4 Oct, 2020, 2:31 am Uwe Schindler, mailto:u...@thetaphi.de> > wrote:

Is the branch ready for Jenkins testing?

If yes and "gradlew check" works, I really would like to set it up.

Uwe

Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya 
mailto:ichattopadhy...@gmail.com> >:

Hi Devs,

 

As you might be aware, the reference_impl branch has a lot of improvements that 
we want to see in Solr master. However, it is currently a large deviation from 
master and hence the stability and reliability (though improved in certain 
aspects) remains to be tested in real production environments before we gain 
confidence in bringing those changes to master.

 

I propose that we do a one off preview release from that branch, say Solr 10 
alpha (early access) or any other name that someone suggests, so that users 
could try it out and report regressions or improvements etc.

 

I volunteer to be the RM and planning to start the process around 1 December-15 
December timeframe. Until then, we can tighten the loose ends on the branch and 
plan for such a release.

 

Is there any thoughts, concerns, questions?

 

Regards,

Ishan


--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de

Re: Solr Alpha (EA) release of Reference Branch

[Tomás Fernández Löbbe]

I'm glad to see efforts to merge the reference branch changes into master.
I do agree with the previous comments that calling it "Solr 10" (even with
the "-alpha") would confuse users, maybe use "reference"? or maybe
something in reference to SOLR-14788? Don't know what's the issue with the
package manager but It likely can be modified to handle something like
this. Also, why does this need to be an official Apache release? Lets just
make unofficial releases (maybe tags in GH and the binaries in your apache
home directory or something) and ask the community to test those. You can
iterate much faster that way (you may want to have multiple of these
releases , maybe weekly at some point and going through the official
process will be tedious) and it would be much more clear for the users.

On Sun, Oct 4, 2020 at 10:16 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> > I do hope we manage to port to master all these improvements!
>
> Ilan: It is a very important consideration. We should ensure that this
> happens (either these changes are ported to master in chunks, or this
> branch becomes master after fixing the history to decompose in meaningful
> chunks).
>
>
> > What’s your sense of how much effort changing _functionality_ on
> master/8x and porting it to the EA is? I’m sure “It Depends(tm)”, I’m more
> interested in whether you expect most stuff ports pretty easily or very
> little stuff to port easily?
>
> Erick: I feel both branches (master and reference) are mostly on par
> functionality wise (except recent changes in master, post June). AFAICT,
> bulk of the changes in reference branch are fixes, refactorings and
> SolrCloud improvements. Mark, please fill in if I've missed something.
>
> > BTW, how many warnings are there ;)
>
> Erick: Tons! Precommit fails too. That's why we need time till December :-)
>
> > does it become futile to chase down the intermittent failures we see on
> master/8x? One of the major thrusts of the EA is things like race
> conditions and the like. If many/most such errors just disappear in EA, I
> have little incentive to fix them in master/8x. Under any circumstances, I
> suspect that most fixes like this would be totally different between the
> two. That’s a huge positive BTW….
>
> Erick: Depends on how optimistic we are about the success of this branch.
> At this moment, I am not confident enough to have these changes in
> reference branch merged back to master, and hence I want users to do a
> thorough production testing before we are confident. Just because tests run
> fast doesn't necessarily mean the service will run flawlessly in
> production, even though that is definitely the hope Mark started this
> effort with. In my opinion, fixes to 8x/master should definitely not stop
> because of this effort.
>
> > Does it make sense to cut 9.0 coincidentally with the EA being adopted
> as the right future direction? In that case, 9x may be short-lived, more of
> a placeholder that we deprecate methods, backport new changes from EA etc,
> but don’t necessarily expend much effort to backport changes from 10x that
> don’t backport easily.
>
> Erick: +1, definitely one of the many reasonable paths to take, IMO.
>
> > Has Lucene changed much (or at all) in the EA? I’m guessing not. Maybe
> not even touched…
>
> Erick: It is the same Lucene as what master is using. I don't see any
> Lucene level changes.
>
> >  Let's focus on making all the tests pass and get this to the hands of
> our users.
>
> Noble: +1
>
> > Let's say Solr 10 ( or whatever name gets picked ) turns out stable
> enough in the alpha phase - What would the next step be?
>
> Varun: I think at that point we should make sure that branch is the
> master: either (i) by porting over all changes from that branch, piece by
> piece, to master branch (I volunteer to do so), or (ii) fix the commit
> history of that branch itself (decompose all the changes into
> meaningful/logical chunks) and make sure all features in current master are
> on that branch (I volunteer to do so).
>
> > Would we bring back all the changes to master?
>
> Varun: As I explained above, that is one of the possiblities.
>
> > Do you have a sense into how that would end up playing out? Could it be
> brought in chunks or would it have to be wholesale ?
>
> Varun: It can surely be brought in wholesale. As per a conversation with
> Noble and David, we agreed that bringing it in chunks would be best going
> forward. All chunks may not independently work/pass tests, but they will be
> isolated enough to capture the themes.
>
> > Also do you know what features in the reference branch have been removed
> because they were unstable ?
>
> Varun: None that I am aware of. Mark, please help me here. There are many
> features whose tests are disabled, either because they were failing or they
> were taking very long. It is our collective goal to ensure they are
> unignored before this EA release. Mark is working on it, AFAIK.
>
> > Finding out the 

Re: Solr Alpha (EA) release of Reference Branch

[Ishan Chattopadhyaya]

> I do hope we manage to port to master all these improvements!

Ilan: It is a very important consideration. We should ensure that this
happens (either these changes are ported to master in chunks, or this
branch becomes master after fixing the history to decompose in meaningful
chunks).


> What’s your sense of how much effort changing _functionality_ on
master/8x and porting it to the EA is? I’m sure “It Depends(tm)”, I’m more
interested in whether you expect most stuff ports pretty easily or very
little stuff to port easily?

Erick: I feel both branches (master and reference) are mostly on par
functionality wise (except recent changes in master, post June). AFAICT,
bulk of the changes in reference branch are fixes, refactorings and
SolrCloud improvements. Mark, please fill in if I've missed something.

> BTW, how many warnings are there ;)

Erick: Tons! Precommit fails too. That's why we need time till December :-)

> does it become futile to chase down the intermittent failures we see on
master/8x? One of the major thrusts of the EA is things like race
conditions and the like. If many/most such errors just disappear in EA, I
have little incentive to fix them in master/8x. Under any circumstances, I
suspect that most fixes like this would be totally different between the
two. That’s a huge positive BTW….

Erick: Depends on how optimistic we are about the success of this branch.
At this moment, I am not confident enough to have these changes in
reference branch merged back to master, and hence I want users to do a
thorough production testing before we are confident. Just because tests run
fast doesn't necessarily mean the service will run flawlessly in
production, even though that is definitely the hope Mark started this
effort with. In my opinion, fixes to 8x/master should definitely not stop
because of this effort.

> Does it make sense to cut 9.0 coincidentally with the EA being adopted as
the right future direction? In that case, 9x may be short-lived, more of a
placeholder that we deprecate methods, backport new changes from EA etc,
but don’t necessarily expend much effort to backport changes from 10x that
don’t backport easily.

Erick: +1, definitely one of the many reasonable paths to take, IMO.

> Has Lucene changed much (or at all) in the EA? I’m guessing not. Maybe
not even touched…

Erick: It is the same Lucene as what master is using. I don't see any
Lucene level changes.

>  Let's focus on making all the tests pass and get this to the hands of
our users.

Noble: +1

> Let's say Solr 10 ( or whatever name gets picked ) turns out stable
enough in the alpha phase - What would the next step be?

Varun: I think at that point we should make sure that branch is the master:
either (i) by porting over all changes from that branch, piece by piece, to
master branch (I volunteer to do so), or (ii) fix the commit history of
that branch itself (decompose all the changes into meaningful/logical
chunks) and make sure all features in current master are on that branch (I
volunteer to do so).

> Would we bring back all the changes to master?

Varun: As I explained above, that is one of the possiblities.

> Do you have a sense into how that would end up playing out? Could it be
brought in chunks or would it have to be wholesale ?

Varun: It can surely be brought in wholesale. As per a conversation with
Noble and David, we agreed that bringing it in chunks would be best going
forward. All chunks may not independently work/pass tests, but they will be
isolated enough to capture the themes.

> Also do you know what features in the reference branch have been removed
because they were unstable ?

Varun: None that I am aware of. Mark, please help me here. There are many
features whose tests are disabled, either because they were failing or they
were taking very long. It is our collective goal to ensure they are
unignored before this EA release. Mark is working on it, AFAIK.

> Finding out the features/bug-fixes in master that haven't made it to the
reference branch would be easier to find out.

Varun: This is important, so that master and reference are on par with each
other features wise. This is what I was working on, and intend to continue
working on. (SOLR-14830)

>  When we say "let users try", do we mean actual public with a release
published on our website?

Alex: Yes, an official release via Apache process.

> Because I can see the publishing of version 10, however it is tagged
> (alpha, whatever), completely confusing people about the upcoming 9
> version and causing an adoption delay.

Alex: We have to be very clear about messaging that this is an early access
preview release, and by no means what will be there in the final 10.0 (or
howsoever we tag it).

> Especially combined with
> cleanups that we already put in 9.0.

Alex: All 9.0 cleanups (deprecations, removals, examples) etc. should
ideally be mirrored in this reference branch. We should ensure that and
that is part of what I'm working on: 

Re: Solr Alpha (EA) release of Reference Branch

[Alexandre Rafalovitch]

(The comments below are in the context of +1 of getting this working out.)

When we say "let users try", do we mean actual public with a release
published on our website?

Because I can see the publishing of version 10, however it is tagged
(alpha, whatever), completely confusing people about the upcoming 9
version and causing an adoption delay. Especially combined with
cleanups that we already put in 9.0. Maybe we could release it to
committers community first and dogfood it "internally"?

And if the issue with naming it 'not 10.x' is one piece of code
(package manager), maybe we can one-off patch that instead. Or hack
the version to be something ridiculous like 42 (the answer to
everything...) instead of something that is psychologically feasible.
I recall this dual version confusion happening before in other
communities and it really messed things up. Python is a recent
example, but I seem to recall other similar events for
products/communities that no longer exist (hopefully for other
reasons).

And yes, all the questions of forward-porting are there as well,
if/once this succeeds.

Regards,
   Alex.


Regards,
   Alex.

On Sun, 4 Oct 2020 at 11:34, Varun Thacker  wrote:
>
> Hi Ishan,
>
> Let's say Solr 10 ( or whatever name gets picked ) turns out stable enough in 
> the alpha phase - What would the next step be?
>
> Would we bring back all the changes to master? Do you have a sense into how 
> that would end up playing out? Could it be brought in chunks or would it have 
> to be wholesale ?
>
> Also do you know what features in the reference branch have been removed 
> because they were unstable ? Finding out the features/bug-fixes in master 
> that haven't made it to the reference branch would be easier to find out.
>
> On Sat, Oct 3, 2020 at 10:17 PM Ishan Chattopadhyaya 
>  wrote:
>>
>> Erick, I'll answer your questions shortly.
>>
>> On Sun, 4 Oct, 2020, 10:33 am Ishan Chattopadhyaya, 
>>  wrote:
>>>
>>> Agree, Noble. Let's not worry about the naming too much. We can discuss 
>>> that later as well, or in a separate thread.
>>>
>>> On Sun, 4 Oct, 2020, 10:06 am Noble Paul,  wrote:

 +1 Ishan

 It's important that the branch gets some real world testing and
 feedback. At this point we cannot be 100% sure about the stability of
 that branch to port all the changes to master.

 Users don't care what is Solr 9/Solr 10  or even Mark's Solr or even a
 "Crazy Solr". As long as all the tests pass and they can do an upgrade
 of their existing cluster to that release,that IS Solr. I think we do
 not need to worry too much about it now. If/when we reach a point
 where we have a new stable release of Solr that is 100% compatible
 with our other branch, we can resume this discussion.

 As Ilan said, we may get real feedback from our users deploying it on
 production scale but non critical deployments. Our JUnit tests are not
 good enough to uncover stability issues.

 Let's focus on making all the tests pass and get this to the hands of our 
 users.

 On Sun, Oct 4, 2020 at 8:01 AM Uwe Schindler  wrote:
 >
 > Is the branch ready for Jenkins testing?
 >
 > If yes and "gradlew check" works, I really would like to set it up.
 >
 > Uwe
 >
 > Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya 
 > :
 >>
 >> Hi Devs,
 >>
 >> As you might be aware, the reference_impl branch has a lot of 
 >> improvements that we want to see in Solr master. However, it is 
 >> currently a large deviation from master and hence the stability and 
 >> reliability (though improved in certain aspects) remains to be tested 
 >> in real production environments before we gain confidence in bringing 
 >> those changes to master.
 >>
 >> I propose that we do a one off preview release from that branch, say 
 >> Solr 10 alpha (early access) or any other name that someone suggests, 
 >> so that users could try it out and report regressions or improvements 
 >> etc.
 >>
 >> I volunteer to be the RM and planning to start the process around 1 
 >> December-15 December timeframe. Until then, we can tighten the loose 
 >> ends on the branch and plan for such a release.
 >>
 >> Is there any thoughts, concerns, questions?
 >>
 >> Regards,
 >> Ishan
 >
 >
 > --
 > Uwe Schindler
 > Achterdiek 19, 28357 Bremen
 > https://www.thetaphi.de



 --
 -
 Noble Paul

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr Alpha (EA) release of Reference Branch

[Varun Thacker]

Hi Ishan,

Let's say Solr 10 ( or whatever name gets picked ) turns out stable enough
in the alpha phase - What would the next step be?

Would we bring back all the changes to master? Do you have a sense into how
that would end up playing out? Could it be brought in chunks or would it
have to be wholesale ?

Also do you know what features in the reference branch have been removed
because they were unstable ? Finding out the features/bug-fixes in master
that haven't made it to the reference branch would be easier to find out.

On Sat, Oct 3, 2020 at 10:17 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Erick, I'll answer your questions shortly.
>
> On Sun, 4 Oct, 2020, 10:33 am Ishan Chattopadhyaya, <
> ichattopadhy...@gmail.com> wrote:
>
>> Agree, Noble. Let's not worry about the naming too much. We can discuss
>> that later as well, or in a separate thread.
>>
>> On Sun, 4 Oct, 2020, 10:06 am Noble Paul,  wrote:
>>
>>> +1 Ishan
>>>
>>> It's important that the branch gets some real world testing and
>>> feedback. At this point we cannot be 100% sure about the stability of
>>> that branch to port all the changes to master.
>>>
>>> Users don't care what is Solr 9/Solr 10  or even Mark's Solr or even a
>>> "Crazy Solr". As long as all the tests pass and they can do an upgrade
>>> of their existing cluster to that release,that IS Solr. I think we do
>>> not need to worry too much about it now. If/when we reach a point
>>> where we have a new stable release of Solr that is 100% compatible
>>> with our other branch, we can resume this discussion.
>>>
>>> As Ilan said, we may get real feedback from our users deploying it on
>>> production scale but non critical deployments. Our JUnit tests are not
>>> good enough to uncover stability issues.
>>>
>>> Let's focus on making all the tests pass and get this to the hands of
>>> our users.
>>>
>>> On Sun, Oct 4, 2020 at 8:01 AM Uwe Schindler  wrote:
>>> >
>>> > Is the branch ready for Jenkins testing?
>>> >
>>> > If yes and "gradlew check" works, I really would like to set it up.
>>> >
>>> > Uwe
>>> >
>>> > Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya <
>>> ichattopadhy...@gmail.com>:
>>> >>
>>> >> Hi Devs,
>>> >>
>>> >> As you might be aware, the reference_impl branch has a lot of
>>> improvements that we want to see in Solr master. However, it is currently a
>>> large deviation from master and hence the stability and reliability (though
>>> improved in certain aspects) remains to be tested in real production
>>> environments before we gain confidence in bringing those changes to master.
>>> >>
>>> >> I propose that we do a one off preview release from that branch, say
>>> Solr 10 alpha (early access) or any other name that someone suggests, so
>>> that users could try it out and report regressions or improvements etc.
>>> >>
>>> >> I volunteer to be the RM and planning to start the process around 1
>>> December-15 December timeframe. Until then, we can tighten the loose ends
>>> on the branch and plan for such a release.
>>> >>
>>> >> Is there any thoughts, concerns, questions?
>>> >>
>>> >> Regards,
>>> >> Ishan
>>> >
>>> >
>>> > --
>>> > Uwe Schindler
>>> > Achterdiek 19, 28357 Bremen
>>> > https://www.thetaphi.de
>>>
>>>
>>>
>>> --
>>> -
>>> Noble Paul
>>>
>>

Re: Solr Alpha (EA) release of Reference Branch

[Ishan Chattopadhyaya]

Erick, I'll answer your questions shortly.

On Sun, 4 Oct, 2020, 10:33 am Ishan Chattopadhyaya, <
ichattopadhy...@gmail.com> wrote:

> Agree, Noble. Let's not worry about the naming too much. We can discuss
> that later as well, or in a separate thread.
>
> On Sun, 4 Oct, 2020, 10:06 am Noble Paul,  wrote:
>
>> +1 Ishan
>>
>> It's important that the branch gets some real world testing and
>> feedback. At this point we cannot be 100% sure about the stability of
>> that branch to port all the changes to master.
>>
>> Users don't care what is Solr 9/Solr 10  or even Mark's Solr or even a
>> "Crazy Solr". As long as all the tests pass and they can do an upgrade
>> of their existing cluster to that release,that IS Solr. I think we do
>> not need to worry too much about it now. If/when we reach a point
>> where we have a new stable release of Solr that is 100% compatible
>> with our other branch, we can resume this discussion.
>>
>> As Ilan said, we may get real feedback from our users deploying it on
>> production scale but non critical deployments. Our JUnit tests are not
>> good enough to uncover stability issues.
>>
>> Let's focus on making all the tests pass and get this to the hands of our
>> users.
>>
>> On Sun, Oct 4, 2020 at 8:01 AM Uwe Schindler  wrote:
>> >
>> > Is the branch ready for Jenkins testing?
>> >
>> > If yes and "gradlew check" works, I really would like to set it up.
>> >
>> > Uwe
>> >
>> > Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com>:
>> >>
>> >> Hi Devs,
>> >>
>> >> As you might be aware, the reference_impl branch has a lot of
>> improvements that we want to see in Solr master. However, it is currently a
>> large deviation from master and hence the stability and reliability (though
>> improved in certain aspects) remains to be tested in real production
>> environments before we gain confidence in bringing those changes to master.
>> >>
>> >> I propose that we do a one off preview release from that branch, say
>> Solr 10 alpha (early access) or any other name that someone suggests, so
>> that users could try it out and report regressions or improvements etc.
>> >>
>> >> I volunteer to be the RM and planning to start the process around 1
>> December-15 December timeframe. Until then, we can tighten the loose ends
>> on the branch and plan for such a release.
>> >>
>> >> Is there any thoughts, concerns, questions?
>> >>
>> >> Regards,
>> >> Ishan
>> >
>> >
>> > --
>> > Uwe Schindler
>> > Achterdiek 19, 28357 Bremen
>> > https://www.thetaphi.de
>>
>>
>>
>> --
>> -
>> Noble Paul
>>
>

Re: Solr Alpha (EA) release of Reference Branch

[Ishan Chattopadhyaya]

Agree, Noble. Let's not worry about the naming too much. We can discuss
that later as well, or in a separate thread.

On Sun, 4 Oct, 2020, 10:06 am Noble Paul,  wrote:

> +1 Ishan
>
> It's important that the branch gets some real world testing and
> feedback. At this point we cannot be 100% sure about the stability of
> that branch to port all the changes to master.
>
> Users don't care what is Solr 9/Solr 10  or even Mark's Solr or even a
> "Crazy Solr". As long as all the tests pass and they can do an upgrade
> of their existing cluster to that release,that IS Solr. I think we do
> not need to worry too much about it now. If/when we reach a point
> where we have a new stable release of Solr that is 100% compatible
> with our other branch, we can resume this discussion.
>
> As Ilan said, we may get real feedback from our users deploying it on
> production scale but non critical deployments. Our JUnit tests are not
> good enough to uncover stability issues.
>
> Let's focus on making all the tests pass and get this to the hands of our
> users.
>
> On Sun, Oct 4, 2020 at 8:01 AM Uwe Schindler  wrote:
> >
> > Is the branch ready for Jenkins testing?
> >
> > If yes and "gradlew check" works, I really would like to set it up.
> >
> > Uwe
> >
> > Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com>:
> >>
> >> Hi Devs,
> >>
> >> As you might be aware, the reference_impl branch has a lot of
> improvements that we want to see in Solr master. However, it is currently a
> large deviation from master and hence the stability and reliability (though
> improved in certain aspects) remains to be tested in real production
> environments before we gain confidence in bringing those changes to master.
> >>
> >> I propose that we do a one off preview release from that branch, say
> Solr 10 alpha (early access) or any other name that someone suggests, so
> that users could try it out and report regressions or improvements etc.
> >>
> >> I volunteer to be the RM and planning to start the process around 1
> December-15 December timeframe. Until then, we can tighten the loose ends
> on the branch and plan for such a release.
> >>
> >> Is there any thoughts, concerns, questions?
> >>
> >> Regards,
> >> Ishan
> >
> >
> > --
> > Uwe Schindler
> > Achterdiek 19, 28357 Bremen
> > https://www.thetaphi.de
>
>
>
> --
> -
> Noble Paul
>

Re: Solr Alpha (EA) release of Reference Branch

[Noble Paul]

+1 Ishan

It's important that the branch gets some real world testing and
feedback. At this point we cannot be 100% sure about the stability of
that branch to port all the changes to master.

Users don't care what is Solr 9/Solr 10  or even Mark's Solr or even a
"Crazy Solr". As long as all the tests pass and they can do an upgrade
of their existing cluster to that release,that IS Solr. I think we do
not need to worry too much about it now. If/when we reach a point
where we have a new stable release of Solr that is 100% compatible
with our other branch, we can resume this discussion.

As Ilan said, we may get real feedback from our users deploying it on
production scale but non critical deployments. Our JUnit tests are not
good enough to uncover stability issues.

Let's focus on making all the tests pass and get this to the hands of our users.

On Sun, Oct 4, 2020 at 8:01 AM Uwe Schindler  wrote:
>
> Is the branch ready for Jenkins testing?
>
> If yes and "gradlew check" works, I really would like to set it up.
>
> Uwe
>
> Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya 
> :
>>
>> Hi Devs,
>>
>> As you might be aware, the reference_impl branch has a lot of improvements 
>> that we want to see in Solr master. However, it is currently a large 
>> deviation from master and hence the stability and reliability (though 
>> improved in certain aspects) remains to be tested in real production 
>> environments before we gain confidence in bringing those changes to master.
>>
>> I propose that we do a one off preview release from that branch, say Solr 10 
>> alpha (early access) or any other name that someone suggests, so that users 
>> could try it out and report regressions or improvements etc.
>>
>> I volunteer to be the RM and planning to start the process around 1 
>> December-15 December timeframe. Until then, we can tighten the loose ends on 
>> the branch and plan for such a release.
>>
>> Is there any thoughts, concerns, questions?
>>
>> Regards,
>> Ishan
>
>
> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de



-- 
-
Noble Paul

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr Alpha (EA) release of Reference Branch

[Ishan Chattopadhyaya]

Yes Uwe, it is ready for Jenkins testing. The Solr tests should run very
fast now.

As for naming, our package manager in Solr will break if we don't specify a
major and a minor version. There's a concept of version constraints for
packages that need to compare against Solr version. I think release process
will also be much simpler if we have a major and minor version. We have
done a similar release in the past, 4.0.0-beta iirc.

Why I favour 10.0-alpha or something like that is because users would
clearly know it is something that isn't coming right now (and hence very
early access), and it is logically a major version change (that comes after
8x). With calling it 10x instead of 9x, we have the scope of abandoning the
effort as a whole if the early access releases have some serious problem or
we decide to take some other release strategy later.

If the 10.0-alpha succeeds, we can always fold the changes back into a 9.1
or go from 9.0 straight to 10.0, depending on what we decide later.
Alternatively, if the release looks good and 9.0 hasn't released yet, we
can fold those changes back to master, and either (a) release everything
normally as 9.0, or (b) call master as 10x and release from there (going
from 8.8 to 10.0 directly, skipping 9x altogether).

Tldr, we shall have complete flexibility to go in any direction we want to.

WDYT?

On Sun, 4 Oct, 2020, 2:31 am Uwe Schindler,  wrote:

> Is the branch ready for Jenkins testing?
>
> If yes and "gradlew check" works, I really would like to set it up.
>
> Uwe
>
> Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com>:
>>
>> Hi Devs,
>>
>> As you might be aware, the reference_impl branch has a lot of
>> improvements that we want to see in Solr master. However, it is currently a
>> large deviation from master and hence the stability and reliability (though
>> improved in certain aspects) remains to be tested in real production
>> environments before we gain confidence in bringing those changes to master.
>>
>> I propose that we do a one off preview release from that branch, say Solr
>> 10 alpha (early access) or any other name that someone suggests, so that
>> users could try it out and report regressions or improvements etc.
>>
>> I volunteer to be the RM and planning to start the process around 1
>> December-15 December timeframe. Until then, we can tighten the loose ends
>> on the branch and plan for such a release.
>>
>> Is there any thoughts, concerns, questions?
>>
>> Regards,
>> Ishan
>>
>
> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de
>

Re: Solr Alpha (EA) release of Reference Branch

[Erick Erickson]

I agree with Ilan, let’s call it something else. “super special the future of 
Solr” maybe ;) “Marks baby”. “batshit crazy better Solr”.

What I’d like to avoid is confusion about where to fix things. Say I’m working 
on an issue. Should I fix it in this impl then backport to 9.0 and 8x? Do like 
we do now (fix on 9.0 backport to 8x as indicated) then “forward port” to the 
reference impl? Ignore the reference impl other than testing?

Questions:

- What’s your sense of how much effort changing _functionality_ on master/8x 
and porting it to the EA is? I’m sure “It Depends(tm)”, I’m more interested in 
whether you expect most stuff ports pretty easily or very little stuff to port 
easily? BTW, how many warnings are there ;)

- The above notwithstanding, does it become futile to chase down the 
intermittent failures we see on master/8x? One of the major thrusts of the EA 
is things like race conditions and the like. If many/most such errors just 
disappear in EA, I have little incentive to fix them in master/8x. Under any 
circumstances, I suspect that most fixes like this would be totally different 
between the two. That’s a huge positive BTW….

- Does it make sense to cut 9.0 coincidentally with the EA being adopted as the 
right future direction? In that case, 9x may be short-lived, more of a 
placeholder that we deprecate methods, backport new changes from EA etc, but 
don’t necessarily expend much effort to backport changes from 10x that don’t 
backport easily.

- Has Lucene changed much (or at all) in the EA? I’m guessing not. Maybe not 
even touched…

Thanks,
Erick

> On Oct 3, 2020, at 4:27 PM, Ilan Ginzburg  wrote:
> 
> Thanks Ishan for the initiative!
> I think that’s a good idea if it allows testing that branch, assuming some 
> are ready to invest what it takes and run this in production (maybe not with 
> user facing prod traffic?).
> 
> I do not think naming it Solr 10 is a good idea though, as it is likely very 
> different from what will end up being in Solr 10 (and even from what will be 
> in Solr 9.
> 
> I do hope we manage to port to master all these improvements!
> 
> Ilan
> 
> On Sat 3 Oct 2020 at 21:42, Ishan Chattopadhyaya  
> wrote:
> Hi Devs,
> 
> As you might be aware, the reference_impl branch has a lot of improvements 
> that we want to see in Solr master. However, it is currently a large 
> deviation from master and hence the stability and reliability (though 
> improved in certain aspects) remains to be tested in real production 
> environments before we gain confidence in bringing those changes to master.
> 
> I propose that we do a one off preview release from that branch, say Solr 10 
> alpha (early access) or any other name that someone suggests, so that users 
> could try it out and report regressions or improvements etc.
> 
> I volunteer to be the RM and planning to start the process around 1 
> December-15 December timeframe. Until then, we can tighten the loose ends on 
> the branch and plan for such a release.
> 
> Is there any thoughts, concerns, questions?
> 
> Regards,
> Ishan


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr Alpha (EA) release of Reference Branch

[Uwe Schindler]

Is the branch ready for Jenkins testing?

If yes and "gradlew check" works, I really would like to set it up.

Uwe

Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya 
:
>Hi Devs,
>
>As you might be aware, the reference_impl branch has a lot of
>improvements
>that we want to see in Solr master. However, it is currently a large
>deviation from master and hence the stability and reliability (though
>improved in certain aspects) remains to be tested in real production
>environments before we gain confidence in bringing those changes to
>master.
>
>I propose that we do a one off preview release from that branch, say
>Solr
>10 alpha (early access) or any other name that someone suggests, so
>that
>users could try it out and report regressions or improvements etc.
>
>I volunteer to be the RM and planning to start the process around 1
>December-15 December timeframe. Until then, we can tighten the loose
>ends
>on the branch and plan for such a release.
>
>Is there any thoughts, concerns, questions?
>
>Regards,
>Ishan

--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de

Re: Solr Alpha (EA) release of Reference Branch

[Ilan Ginzburg]

Thanks Ishan for the initiative!
I think that’s a good idea if it allows testing that branch, assuming some
are ready to invest what it takes and run this in production (maybe not
with user facing prod traffic?).

I do not think naming it Solr 10 is a good idea though, as it is likely
very different from what will end up being in Solr 10 (and even from what
will be in Solr 9.

I do hope we manage to port to master all these improvements!

Ilan

On Sat 3 Oct 2020 at 21:42, Ishan Chattopadhyaya 
wrote:

> Hi Devs,
>
> As you might be aware, the reference_impl branch has a lot of improvements
> that we want to see in Solr master. However, it is currently a large
> deviation from master and hence the stability and reliability (though
> improved in certain aspects) remains to be tested in real production
> environments before we gain confidence in bringing those changes to master.
>
> I propose that we do a one off preview release from that branch, say Solr
> 10 alpha (early access) or any other name that someone suggests, so that
> users could try it out and report regressions or improvements etc.
>
> I volunteer to be the RM and planning to start the process around 1
> December-15 December timeframe. Until then, we can tighten the loose ends
> on the branch and plan for such a release.
>
> Is there any thoughts, concerns, questions?
>
> Regards,
> Ishan
>

Solr Alpha (EA) release of Reference Branch

[Ishan Chattopadhyaya]

Hi Devs,

As you might be aware, the reference_impl branch has a lot of improvements
that we want to see in Solr master. However, it is currently a large
deviation from master and hence the stability and reliability (though
improved in certain aspects) remains to be tested in real production
environments before we gain confidence in bringing those changes to master.

I propose that we do a one off preview release from that branch, say Solr
10 alpha (early access) or any other name that someone suggests, so that
users could try it out and report regressions or improvements etc.

I volunteer to be the RM and planning to start the process around 1
December-15 December timeframe. Until then, we can tighten the loose ends
on the branch and plan for such a release.

Is there any thoughts, concerns, questions?

Regards,
Ishan