I don’t agree with the argument “Solr outgrew being a subproject of Lucene”. I read “promotion to TLP” as if this was some achievement that needs to be celebrated now. Solr didn’t become a TLP years ago because the decision then was to merge with Lucene development, thinking they would progress better together than separated. It’s technically true that Solr is a subproject of Lucene, but so is Lucene Core, and I don’t see Lucene Core being promoted to TLP. They are both part of the same Apache project, which for historical reasons is called Lucene.
> I would be curious if people can make the argument for keeping them together... I think the same arguments that were used 10 years ago to merge the projects are as valid now, some of them presented in Dawid’s email. Faster development, better coverage, code in the right places.[1] IMO, if we need to say “we can’t release X because it breaks Y”, or “we need to release X to be able to release Y”, the projects are not really independent, and “the PMCs will overlap” won’t take us very far. > The big question is this: “Is this the right time to split Solr and Lucene into two independent projects?”. This is not the question we should be asking ourselves right now. It assumes the split is happening, and that’s what we are trying to discuss here. The question in my mind is “Is splitting Lucene and Solr into different project beneficial for them? Is this going to make them both better?" > As it is today, deveopers have had to do necessary Solr changes at the same time when doing changes in Lucene. This is not really fair to the (mainly) Lucene developers. It is not fair to Solr either, as such work might be done in a hasty fashion and/or in a sub optimal way due to lack of familiarity with Solr code base; like we unfortunately have seen a couple of times in the past (not trying to blame anyone). This, I agree, is a pain point for keeping them together. That said, while not all, most currently active committers joined the project while this was already a thing, it’s not something that was imposed later to the majority of us. > With Lucene as a dependency, Solr can choose to stay on same Lucene version for a couple of releases while taking the time to work out the proper way to adapt to changed Lucene APIs or to sort out performance issues. I agree with this and I believe it’s a point in favor of keeping them together (and in part discussed 10 years ago when projects merged). Keeping them on the same repo forces Solr to use the latest Lucene, helping find issues/bugs soon, hopefully before they are released. [1] https://mail-archives.apache.org/mod_mbox/lucene-general/201002.mbox/%3c9ac0c6aa1002240832x1a8e3309k6799d75b8d19d...@mail.gmail.com%3e On Tue, May 5, 2020 at 8:56 AM Michael McCandless <luc...@mikemccandless.com> wrote: > On Tue, May 5, 2020 at 11:41 AM Jan Høydahl <jan....@cominvent.com> wrote: > > As it is today, deveopers have had to do necessary Solr changes at the >> same time when doing changes in Lucene. This is not really fair to the >> (mainly) Lucene developers. It is not fair to Solr either, as such work >> might be done in a hasty fashion and/or in a sub optimal way due to lack of >> familiarity with Solr code base; like we unfortunately have seen a couple >> of times in the past (not trying to blame anyone). With Lucene as a >> dependency, Solr can choose to stay on same Lucene version for a couple of >> releases while taking the time to work out the proper way to adapt to >> changed Lucene APIs or to sort out performance issues. >> > > +1, that is a great point, Jan. > > This will mean that the (any) necessary Solr source code changes that go > along with a Lucene change will (sometimes) be done with higher quality, > more thought, better expertise, etc., which I agree will be good for > ongoing Solr development, help prevent accidental performance regressions, > etc. Net/net that's a big positive for Solr, in addition to having a > stronger independent identity (https://solr.apache.org). > > >> Question: When Lucene no longer has the Solr test suite to help catch >> bugs, how long time would it take from a Lucene commit, before Solr/ES >> Jenkins instances would have had time to produce a build and run tests? >> Would it be possible to setup a trigger in Solr Jenkins? >> > > That's a great question! > > Maybe Elasticsearch developers could chime in, since this already happened > for them many times by now :) I would think there are technical solutions > to let the Solr CI build pull the latest Lucene snapshot build, to keep the > latency lowish, but I do not know the details. > > Mike >
