Re: Solr Alpha (EA) release of Reference Branch

2020-10-07 Thread David Smiley
I think whatever changes can be safely brought from that branch into master should be done now independently of whatever happens to that branch. It may not be a lot of this branch, but it's something. For example off the top of my head -- pre-compiling XPath expressions. AIUI, many changes

Re: Solr Alpha (EA) release of Reference Branch

2020-10-07 Thread Ilan Ginzburg
TBH, a PR with more than 1400 changed files is hard to look at. How many of us will invest a few weeks at least to really understand it? We should assume that if we don't bring these changes piece by piece, we risk having an unstable version of SolrCloud for a while. When I look at some ongoing

Re: Solr Alpha (EA) release of Reference Branch

2020-10-07 Thread Jan Høydahl
We want/need these improvements, for sure! Agree treating this whole thing as a black box is dangerous. At the same time I realize that this will be one or a few huge merges anyway! Rob’s suggestion is interesting! If at all possible to get the branch up to date with master, and make a PR, then

Re: Solr Alpha (EA) release of Reference Branch

2020-10-06 Thread Robert Muir
On Tue, Oct 6, 2020 at 10:45 PM Anshum Gupta wrote: > > I haven’t looked at the current ref branch recently, but the folks who > have looked at it, if you think that this code can be merged into master > even as big chunks, that’d be the most confidence building way forward. > > +1 for

Re: Solr Alpha (EA) release of Reference Branch

2020-10-06 Thread Varun Thacker
On Tue, Oct 6, 2020 at 7:45 PM Anshum Gupta wrote: > Thanks for initiating this discussion, Ishan. > > For the sake of making sure that we are all on the same page, let me > summarize my understanding and take on this thread. > > The current situation > Mark has a reference branch, which the

Re: Solr Alpha (EA) release of Reference Branch

2020-10-06 Thread Anshum Gupta
Thanks for initiating this discussion, Ishan. For the sake of making sure that we are all on the same page, let me summarize my understanding and take on this thread. The current situation Mark has a reference branch, which the folks who have looked at the branch, feel that it’s a much better,

Re: Solr Alpha (EA) release of Reference Branch

2020-10-06 Thread Ilan Ginzburg
Copying below Mark's posts from ASF Slack #solr-next-big-thing channel. The Solr Reference Branch. Document 1, a quick intro. You can think of the Solr Reference Branch as a remaster of Solr. It is not an attempt to redesign Solr or make it more fancy. The goal of the Solr Reference Branch is to

Re: Solr Alpha (EA) release of Reference Branch

2020-10-06 Thread Tomás Fernández Löbbe
> Let's say we cut 9x and now there is a new master taken from the reference branch. I never said “make a new master”, I said merge changes in ref branch into master. If things are broken into pieces like Ishan is suggesting, those changes can be merged into 9.x too. I only suggested this because

Re: Solr Alpha (EA) release of Reference Branch

2020-10-06 Thread Noble Paul
> I think the danger is high to treat this branch as a black box (or an "all or > nothing"). True Ilan. Ideally, I would like a few of us to study the code & start pulling in changes we are confident of (even to 8x branch, why not). We cannot burden a single developer to do everything. This

Re: Solr Alpha (EA) release of Reference Branch

2020-10-06 Thread Ishan Chattopadhyaya
I'm willing to help and I believe others will too if the amount of work for contributing is reasonable (i.e. not a three months effort). I looked into the possibility of doing so. To me, it seemed to be that it is very hard to do so: possibly 1 year project for me. Problem is that it is hard to

Re: Solr Alpha (EA) release of Reference Branch

2020-10-06 Thread Ilan Ginzburg
Another option to integrate this work into the main code line would be to understand what changes have been made and where (Mark's descriptions in Slack go in the right way but are still too high level), and then port or even redo them in main, one by one. I think the danger is high to treat this

Re: Solr Alpha (EA) release of Reference Branch

2020-10-06 Thread Ishan Chattopadhyaya
> Docker is not a big requirement for large scale installations. Most of them already have their own install scripts. Availability of docker is not important for them. If a user is only encouraged to install Solr because of a docker image , most likely they are not running a large enough cluster

Re: Solr Alpha (EA) release of Reference Branch

2020-10-06 Thread Noble Paul
Yes, A docker image will definitely help. I wasn't trying to downplay that On Tue, Oct 6, 2020 at 6:55 PM Ishan Chattopadhyaya wrote: > > > > Docker is not a big requirement for large scale installations. Most of them > > already have their own install scripts. Availability of docker is not >

Re: Solr Alpha (EA) release of Reference Branch

2020-10-06 Thread Noble Paul
@Tomas Fernandez Lobbe This is a very risky proposition. Let's say we cut 9x and now there is a new master taken from the reference branch. That master is now going to be 10.0. * We never get it to the hands of the users anytime soon. We will never be able to reconcile these 2 branches * The

Re: Solr Alpha (EA) release of Reference Branch

2020-10-06 Thread Ishan Chattopadhyaya
As I said, I'm *personally* not confident in putting such a big changeset into master that wasn't vetted in a real user environment widely. I have, in the past, done enough bad things to Solr (directly or indirectly), and I don't want to repeat the same. Also, I'll be very uncomfortable if someone

Re: Solr Alpha (EA) release of Reference Branch

2020-10-05 Thread Tomás Fernández Löbbe
I was thinking (and I haven’t flushed it out completely but will throw the idea) that an alternative approach with this timeline could be to cut 9x branch around November/December? And then you could merge into master, it would have the latest changes from master plus the ref branch changes.

Re: Solr Alpha (EA) release of Reference Branch

2020-10-05 Thread Noble Paul
>I don't think it can be said what committers do and don't do with regards to running Solr. All of us would answer this differently and at different points in time. " I have run it in one large cluster, so it is certified to be bug free/stable" I don't think it's a reasonable approach. We need

Re: Solr Alpha (EA) release of Reference Branch

2020-10-05 Thread Tomás Fernández Löbbe
> I think the Solr 4 alpha/beta situation was different -- it was not some fork a committer was maintaining; it was the master branch of its time, and it was destined to be the very next release, not some possible future release. Agree, 4’s alpha/beta was a very different situation. > I believe

Re: Solr Alpha (EA) release of Reference Branch

2020-10-05 Thread David Smiley
Thanks so much for your responses Ishan... I'm getting much more information in this thread than my attempts to get questions answered on the JIRA issue months ago. And especially, thank you for volunteering for the difficult porting efforts! Tomas said: > I do agree with the previous

Re: Solr Alpha (EA) release of Reference Branch

2020-10-05 Thread Ishan Chattopadhyaya
ow, sorry for failure mails during the night. >> Builds seem to fail, but it’s too late to do anything against it. >> > >> > Uwe >> > >> > - >> > Uwe Schindler >> > Achterdiek 19, D-28357 Bremen >> > https://www.thetaphi.de >>

Re: Solr Alpha (EA) release of Reference Branch

2020-10-05 Thread Ishan Chattopadhyaya
nst it. > > > > Uwe > > > > - > > Uwe Schindler > > Achterdiek 19, D-28357 Bremen > > https://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > From: Ishan Chattopadhyaya > > Sent: Sunday, October 4, 2020 6:32 AM > > To: Uwe

Re: Solr Alpha (EA) release of Reference Branch

2020-10-05 Thread Erick Erickson
e Schindler > Achterdiek 19, D-28357 Bremen > https://www.thetaphi.de > eMail: u...@thetaphi.de > > From: Ishan Chattopadhyaya > Sent: Sunday, October 4, 2020 6:32 AM > To: Uwe Schindler > Cc: Lucene Dev > Subject: Re: Solr Alpha (EA) release of Reference Branch >

Re: Solr Alpha (EA) release of Reference Branch

2020-10-05 Thread Shawn Heisey
On 10/3/2020 1:42 PM, Ishan Chattopadhyaya wrote: As you might be aware, the reference_impl branch has a lot of improvements that we want to see in Solr master. However, it is currently a large deviation from master and hence the stability and reliability (though improved in certain aspects)

Re: Solr Alpha (EA) release of Reference Branch

2020-10-04 Thread Noble Paul
gt; Uwe > > > > - > > Uwe Schindler > > Achterdiek 19, D-28357 Bremen > > https://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > From: Uwe Schindler > Sent: Monday, October 5, 2020 12:01 AM > To: 'Ishan Chattopadhyaya' > Cc: 'Lucene Dev' &

RE: Solr Alpha (EA) release of Reference Branch

2020-10-04 Thread Uwe Schindler
- Uwe Schindler Achterdiek 19, D-28357 Bremen https://www.thetaphi.de eMail: u...@thetaphi.de From: Uwe Schindler Sent: Monday, October 5, 2020 12:01 AM To: 'Ishan Chattopadhyaya' Cc: 'Lucene Dev' Subject: RE: Solr Alpha (EA) release of Reference Branch Hi, I enabled the „gradlew

RE: Solr Alpha (EA) release of Reference Branch

2020-10-04 Thread Uwe Schindler
Schindler Cc: Lucene Dev Subject: Re: Solr Alpha (EA) release of Reference Branch Yes Uwe, it is ready for Jenkins testing. The Solr tests should run very fast now. As for naming, our package manager in Solr will break if we don't specify a major and a minor version. There's a concept

Re: Solr Alpha (EA) release of Reference Branch

2020-10-04 Thread Tomás Fernández Löbbe
I'm glad to see efforts to merge the reference branch changes into master. I do agree with the previous comments that calling it "Solr 10" (even with the "-alpha") would confuse users, maybe use "reference"? or maybe something in reference to SOLR-14788? Don't know what's the issue with the

Re: Solr Alpha (EA) release of Reference Branch

2020-10-04 Thread Ishan Chattopadhyaya
> I do hope we manage to port to master all these improvements! Ilan: It is a very important consideration. We should ensure that this happens (either these changes are ported to master in chunks, or this branch becomes master after fixing the history to decompose in meaningful chunks). >

Re: Solr Alpha (EA) release of Reference Branch

2020-10-04 Thread Alexandre Rafalovitch
(The comments below are in the context of +1 of getting this working out.) When we say "let users try", do we mean actual public with a release published on our website? Because I can see the publishing of version 10, however it is tagged (alpha, whatever), completely confusing people about the

Re: Solr Alpha (EA) release of Reference Branch

2020-10-04 Thread Varun Thacker
Hi Ishan, Let's say Solr 10 ( or whatever name gets picked ) turns out stable enough in the alpha phase - What would the next step be? Would we bring back all the changes to master? Do you have a sense into how that would end up playing out? Could it be brought in chunks or would it have to be

Re: Solr Alpha (EA) release of Reference Branch

2020-10-03 Thread Ishan Chattopadhyaya
Erick, I'll answer your questions shortly. On Sun, 4 Oct, 2020, 10:33 am Ishan Chattopadhyaya, < ichattopadhy...@gmail.com> wrote: > Agree, Noble. Let's not worry about the naming too much. We can discuss > that later as well, or in a separate thread. > > On Sun, 4 Oct, 2020, 10:06 am Noble

Re: Solr Alpha (EA) release of Reference Branch

2020-10-03 Thread Ishan Chattopadhyaya
Agree, Noble. Let's not worry about the naming too much. We can discuss that later as well, or in a separate thread. On Sun, 4 Oct, 2020, 10:06 am Noble Paul, wrote: > +1 Ishan > > It's important that the branch gets some real world testing and > feedback. At this point we cannot be 100% sure

Re: Solr Alpha (EA) release of Reference Branch

2020-10-03 Thread Noble Paul
+1 Ishan It's important that the branch gets some real world testing and feedback. At this point we cannot be 100% sure about the stability of that branch to port all the changes to master. Users don't care what is Solr 9/Solr 10 or even Mark's Solr or even a "Crazy Solr". As long as all the

Re: Solr Alpha (EA) release of Reference Branch

2020-10-03 Thread Ishan Chattopadhyaya
Yes Uwe, it is ready for Jenkins testing. The Solr tests should run very fast now. As for naming, our package manager in Solr will break if we don't specify a major and a minor version. There's a concept of version constraints for packages that need to compare against Solr version. I think

Re: Solr Alpha (EA) release of Reference Branch

2020-10-03 Thread Erick Erickson
I agree with Ilan, let’s call it something else. “super special the future of Solr” maybe ;) “Marks baby”. “batshit crazy better Solr”. What I’d like to avoid is confusion about where to fix things. Say I’m working on an issue. Should I fix it in this impl then backport to 9.0 and 8x? Do like

Re: Solr Alpha (EA) release of Reference Branch

2020-10-03 Thread Uwe Schindler
Is the branch ready for Jenkins testing? If yes and "gradlew check" works, I really would like to set it up. Uwe Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya : >Hi Devs, > >As you might be aware, the reference_impl branch has a lot of >improvements >that we want to see in

Re: Solr Alpha (EA) release of Reference Branch

2020-10-03 Thread Ilan Ginzburg
Thanks Ishan for the initiative! I think that’s a good idea if it allows testing that branch, assuming some are ready to invest what it takes and run this in production (maybe not with user facing prod traffic?). I do not think naming it Solr 10 is a good idea though, as it is likely very

Solr Alpha (EA) release of Reference Branch

2020-10-03 Thread Ishan Chattopadhyaya
Hi Devs, As you might be aware, the reference_impl branch has a lot of improvements that we want to see in Solr master. However, it is currently a large deviation from master and hence the stability and reliability (though improved in certain aspects) remains to be tested in real production