CDCR does work, kind of. But it requires extensive care and feeding and, as Ishan says, it’s _very_ easy to shoot yourself in the foot. Or run out of disk space. Or get to a state where you have to replicate the index. And “bi-directional” means you can go from A -> B _or_ B -> A, but you can’t index to both A and B at once. Anyone who’s using it invariably rolls their own monitoring to make sure it’s still running. You want “fire and forget” functionality, but that’s not where CDCR is at.
The consequence of not having the monitoring in place is that the tlogs fill up, and then your index can become corrupt. Yes, it’s fixable, but there’s always problem N+1... I think CDCR could be made acceptable _if_ someone was willing to own it and devote a lot of time to maintenance. But nobody is stepping up to do it, certainly not me. And it’s a side issue, Solr is a search engine. There are solutions out there that are built from the start to deal with keeping separate DCs in sync. Let’s use those rather than a “kinda works” solution. One of the problems with Solr is that it’s become a hodgepodge of peripheral stuff that somebody found useful at some point. And in a number of instances, capabilities were added to Solr when no other tools were available. But the state of the art have progressed, it’s time to jettison older stuff... The advantage of CDCR is that it is all contained in Solr, no outside packages required. The disadvantage is that has very few people willing to work on it. So I’m for taking it out of Solr. My prediction is that if it’s made a package, it’ll languish and at some point become unusable with the then-current version of Solr. And nobody who complains will be willing to devote the time and effort to making it work with Solr X.Y. FWIW... > On Aug 27, 2020, at 7:50 PM, Ishan Chattopadhyaya <[email protected]> > wrote: > > It does start. It is broken because it is fraught with dangers of users > shooting themselves in their feet. Some context here: > https://issues.apache.org/jira/browse/SOLR-14616?focusedCommentId=17153129&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17153129 > > On Fri, Aug 28, 2020 at 4:52 AM Alexandre Rafalovitch <[email protected]> > wrote: > If CDCR is actively broken (does not start?), then isn't it > effectively deprecated from the last version that did not work? And if > it is not going to be maintained, then isn't the 'latest' version is > whichever we still did not delete it in. Because a broken feature is > only worth keeping in, if we ever plan to fix it. > > We have been through the same with UIMA, if I recall. It was broken > for a bit and then when I pulled it, ONE person got all upset. > SOLR-11694 > > Regards, > Alex > Ps. I don't know the degree of 'broken' of this specific feature. So, > I am mostly talking practical principles here. > > On Thu, 27 Aug 2020 at 19:03, Ishan Chattopadhyaya > <[email protected]> wrote: > > > > > I find it highly depressing that we can't, *in a major release*, manage > > > to get rid of our deprecations -- particularly for code that has a new > > > home and is packaged in a form that is trivial to install (thanks to our > > > new awesome package manager). > > > > I'm not sure why you think "we can't". I can't even remember a single > > committer standing in the way of removing those *that already have a > > package*. However, there's a backlash against removing CDCR even though > > there is no one volunteering to support it (as a package) and it is clearly > > broken, which is what totally puzzles me. > > https://issues.apache.org/jira/browse/SOLR-14616 > > > > On Fri, Aug 28, 2020 at 4:19 AM Alexandre Rafalovitch <[email protected]> > > wrote: > >> > >> Well, I have created SOLR-14783 (Remove DIH from 9.0) and am busily > >> learning magic gradle commands to make that happen without leaving > >> behind random crumbs. Once that lands, I will do Jira search on all > >> DIH still-open tasks after that and close them pointing to the said > >> Jira. > >> > >> So, I guess somebody better -1 the Jira if they really want that one > >> to stay until ... ? And then read very carefully through SIP-10 of > >> which, this is just a first step. > >> > >> In general, maybe we can manage to do so many new features and cleanup > >> in 9 that will make Solr TLP look like a great Big Bang moment... > >> > >> And it will probably take a little longer to achieve that, so the - > >> effective - deprecation schedule would still be ok. > >> > >> Regards, > >> Alex. > >> > >> On Thu, 27 Aug 2020 at 18:35, David Smiley <[email protected]> wrote: > >> >> > >> >> It has been proposed on the list to NOT rip out all deprecations in > >> >> 9.0, but allow users to upgrade to 9.0 with e.g. SolrCell still > >> >> available, and then have yet some time to change their processes to > >> >> adapt to the new way of doing stuff. I like that proposal. Sure, 9.0 > >> >> will remove lots of deprecated code, but I think it is a mistake to do > >> >> all of the proposed removals at once. We can spread removals out in 9.x > >> >> releases, after users have had a few releases with a choice between old > >> >> and new and the new alternative is solid. > >> > > >> > > >> > I find it highly depressing that we can't, *in a major release*, manage > >> > to get rid of our deprecations -- particularly for code that has a new > >> > home and is packaged in a form that is trivial to install (thanks to our > >> > new awesome package manager). I'm sympathetic to waiting to delete > >> > until *after* there is an actual package ready at that time (rather than > >> > just the promise of one). > >> > > >> > Also, users generally are cautious on performing a major version > >> > upgrade. There's time. > >> > > >> > ~ David Smiley > >> > Apache Lucene/Solr Search Developer > >> > http://www.linkedin.com/in/davidwsmiley > >> > > >> > > >> > On Wed, Aug 12, 2020 at 4:06 AM Jan Høydahl <[email protected]> > >> > wrote: > >> >> > >> >> I edited the page to introduce the (super important) Solr TLP split > >> >> into the roadmap. > >> >> Also added a rough timeframe and a «major theme» for each release above > >> >> the issue table. > >> >> I added 8.8 and 9.1 as I think it is important to track what gets done > >> >> just before 9.0 and what can be deferred to after 9.0. > >> >> > >> >> It has been proposed on the list to NOT rip out all deprecations in > >> >> 9.0, but allow users to upgrade to 9.0 with e.g. SolrCell still > >> >> available, and then have yet some time to change their processes to > >> >> adapt to the new way of doing stuff. I like that proposal. Sure, 9.0 > >> >> will remove lots of deprecated code, but I think it is a mistake to do > >> >> all of the proposed removals at once. We can spread removals out in 9.x > >> >> releases, after users have had a few releases with a choice between old > >> >> and new and the new alternative is solid. > >> >> > >> >> Thanks Gus for taking ownership and suggesting a process! Feel free to > >> >> rework what I edited into a structure you see more fit. > >> >> > >> >> Jan > >> >> > >> >> 11. aug. 2020 kl. 18:51 skrev Gus Heck <[email protected]>: > >> >> > >> >> I was thinking that level of detail is in the Jira... I don't see any > >> >> reason for things to disappear (in fact rejected should go in a > >> >> rejected list for future reference.) > >> >> > >> >> On Tue, Aug 11, 2020 at 12:04 PM Ilan Ginzburg <[email protected]> > >> >> wrote: > >> >>> > >> >>> Maybe also add “in progress”? So items do not disappear suddenly from > >> >>> the page when work really starts on them? > >> >>> > >> >>> On Tue 11 Aug 2020 at 17:15, Gus Heck <[email protected]> wrote: > >> >>>> > >> >>>> Cool, since I brought it up, I can volunteer to help manage the page. > >> >>>> We should get jira issue links in there wherever possible. Do we want > >> >>>> to build an initial list and have some sort of Proposed/Planned > >> >>>> workflow so readers can have confidence (or appropriate lack of > >> >>>> confidence) in what they see there? voting on things seems like too > >> >>>> much but maybe folks who care watch the page, and if something is on > >> >>>> there for a week without objection it can be called accepted? If a > >> >>>> discussion starts here it can be marked "Considering" so... something > >> >>>> like this: > >> >>>> > >> >>>> 4 states: Proposed, Considering, Planned, Rejected > >> >>>> > >> >>>> Workflow like this: > >> >>>> Proposed -------(no objection 1 wk) --> Planned > >> >>>> Proposed -------(discussion)----------> Considering > >> >>>> Considering ----(agreement) ----------> Planned > >> >>>> Considering ----(deferred) -----------> Proposed (later release) > >> >>>> Considering ----(unsuitable) ---------> Rejected > >> >>>> Considering ----(promoted) -----------> Proposed (earlier release) > >> >>>> Planned --------(difficulty found) ---> Considering > >> >>>> > >> >>>> Anything in "Considering" should have an active dev list thread, and > >> >>>> if it didn't happen on the list it didn't happen :). Any of that (or > >> >>>> differences of opinion during Considering) can be overridden by a > >> >>>> formal vote of course > >> >>>> > >> >>>> -Gus > >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> On Tue, Aug 11, 2020 at 10:29 AM Ishan Chattopadhyaya > >> >>>> <[email protected]> wrote: > >> >>>>> > >> >>>>> I've created a placeholder document here: > >> >>>>> https://cwiki.apache.org/confluence/display/SOLR/Roadmap > >> >>>>> Let us put in all our items there. > >> >>>>> > >> >>>>> On Tue, Aug 11, 2020 at 4:45 PM Jan Høydahl <[email protected]> > >> >>>>> wrote: > >> >>>>>> > >> >>>>>> Let’s revive this email thread about Roadmap. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> With so many large initiatives going on, and the TLP split also, I > >> >>>>>> think it makes perfect sense with a Roadmap. > >> >>>>>> > >> >>>>>> > >> >>>>>> I know we’re not used to that kind of thing - we tend to just let > >> >>>>>> things play out as it happens to land in various releases, but this > >> >>>>>> time is special, and I think we’d benefit from more coordination. I > >> >>>>>> don’t know how to enforce such coordination though, other than > >> >>>>>> appealing to all committers to endorse the roadmap and respect it > >> >>>>>> when they merge things. We may not be able to set a release date > >> >>>>>> for 9.0 right now, but we may be able to define preconditions and > >> >>>>>> scope certain features to 9.0 or 9.1 rather than 8.7 or 8.8 - that > >> >>>>>> kind of coarse-grained decisions. We also may need a person that > >> >>>>>> «owns» the Roadmap confluence page and actively promotes it, tries > >> >>>>>> to keep it up to date and reminds the rest of us about its > >> >>>>>> existence. A roadmap must NOT be a brake slowing us down, but a > >> >>>>>> tool helping us avoid silly mistakes. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> Jan > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > 5. jul. 2020 kl. 02:39 skrev Noble Paul <[email protected]>: > >> >>>>>> > >> >>>>>> > >> >>>>>> > > >> >>>>>> > >> >>>>>> > >> >>>>>> > I think the logical thing to do today is completely rip out all > >> >>>>>> > >> >>>>>> > >> >>>>>> > autoscaling code as it exists today. > >> >>>>>> > >> >>>>>> > >> >>>>>> > Let's deprecate that in 8.7 and build something for > >> >>>>>> > "assign-strategy". > >> >>>>>> > >> >>>>>> > >> >>>>>> > Austoscaling , if required, should not be a part of Solr > >> >>>>>> > >> >>>>>> > >> >>>>>> > > >> >>>>>> > >> >>>>>> > >> >>>>>> > > >> >>>>>> > >> >>>>>> > >> >>>>>> > > >> >>>>>> > >> >>>>>> > >> >>>>>> > On Fri, Jul 3, 2020 at 5:48 PM Jan Høydahl > >> >>>>>> > <[email protected]> wrote: > >> >>>>>> > >> >>>>>> > >> >>>>>> >> > >> >>>>>> > >> >>>>>> > >> >>>>>> >> +1 > >> >>>>>> > >> >>>>>> > >> >>>>>> >> > >> >>>>>> > >> >>>>>> > >> >>>>>> >> Why don’t we make a Roadmap wiki page as Cassandra suggests, and > >> >>>>>> >> indicate what major things needs to happen when. > >> >>>>>> > >> >>>>>> > >> >>>>>> >> Perhaps if we can get the Solr TLP and git-split ball rolling as > >> >>>>>> >> a pre-9.0 task, then perhaps 8.8 could be the last joint release > >> >>>>>> >> (6.6, 7.7, 8.8 hehe)? > >> >>>>>> > >> >>>>>> > >> >>>>>> >> That would enable Lucene to ship 9.0 without waiting for a ton > >> >>>>>> >> of alpha-quality Solr features, and Solr could have its own > >> >>>>>> >> Roadmap wiki. > >> >>>>>> > >> >>>>>> > >> >>>>>> >> > >> >>>>>> > >> >>>>>> > >> >>>>>> >> Jan > >> >>>>>> > >> >>>>>> > >> >>>>>> >> > >> >>>>>> > >> >>>>>> > >> >>>>>> >> 3. jul. 2020 kl. 09:19 skrev Dawid Weiss <[email protected]>: > >> >>>>>> > >> >>>>>> > >> >>>>>> >> > >> >>>>>> > >> >>>>>> > >> >>>>>> >> > >> >>>>>> > >> >>>>>> > >> >>>>>> >>> I totally expect some things to bubble up when we try to > >> >>>>>> >>> release with Gradle, the tarball being one. I don’t think > >> >>>>>> >>> that’s a very big issue, but if you have lots of “not very big” > >> >>>>>> >>> issues they do add up. > >> >>>>>> > >> >>>>>> > >> >>>>>> >> > >> >>>>>> > >> >>>>>> > >> >>>>>> >> > >> >>>>>> > >> >>>>>> > >> >>>>>> >> Adding a tarball is literally 3-5 lines of code (you add a task > >> >>>>>> >> that builds a tarball or a zip file from the outputs of > >> >>>>>> >> solr/packaging toDir task)... The bigger issue with gradle is > >> >>>>>> >> that somebody has to step up and try to identify any other > >> >>>>>> >> issues and/or missing bits when trying to do a full release > >> >>>>>> >> cycle. > >> >>>>>> > >> >>>>>> > >> >>>>>> >> > >> >>>>>> > >> >>>>>> > >> >>>>>> >> D. > >> >>>>>> > >> >>>>>> > >> >>>>>> >> > >> >>>>>> > >> >>>>>> > >> >>>>>> >> > >> >>>>>> > >> >>>>>> > >> >>>>>> > > >> >>>>>> > >> >>>>>> > >> >>>>>> > > >> >>>>>> > >> >>>>>> > >> >>>>>> > -- > >> >>>>>> > >> >>>>>> > >> >>>>>> > ----------------------------------------------------- > >> >>>>>> > >> >>>>>> > >> >>>>>> > Noble Paul > >> >>>>>> > >> >>>>>> > >> >>>>>> > > >> >>>>>> > >> >>>>>> > >> >>>>>> > --------------------------------------------------------------------- > >> >>>>>> > >> >>>>>> > >> >>>>>> > To unsubscribe, e-mail: [email protected] > >> >>>>>> > >> >>>>>> > >> >>>>>> > For additional commands, e-mail: [email protected] > >> >>>>>> > >> >>>>>> > >> >>>>>> > > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> --------------------------------------------------------------------- > >> >>>>>> > >> >>>>>> > >> >>>>>> To unsubscribe, e-mail: [email protected] > >> >>>>>> > >> >>>>>> > >> >>>>>> For additional commands, e-mail: [email protected] > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>> > >> >>>>> > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> http://www.needhamsoftware.com (work) > >> >>>> http://www.the111shift.com (play) > >> >>>> > >> >>>> > >> >> > >> >> > >> >> -- > >> >> http://www.needhamsoftware.com (work) > >> >> http://www.the111shift.com (play) > >> >> > >> >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
