And look, we started pretty deep in the hole. Solr started with tons of bug or limitations that hardly mattered to it and hit SolrCloud in the eye like a train. And we were not setup to deal with that.
We never had a nice garden for SolrCloud. We started in a mess, thinking, eventually we clear the overgrowth, and we are all good. And then we started building our house and that garden went wild with a life of it's own. And our development practices, amazingly above many many many groups and standards out there, is woefully inaccurate for what we are doing. "Test pass, I'm not sure about all this but I'm going to commit" (Tests never pass, must be a lie anyway) "Leaving on vacation, going to fire this in" "No one has looked at this huge thing, it's been a while, going to commit" *commit* And comments to that affect pretty much wrap up our careful and thoughtful attitude. And then of course we come and clean up after, careful gardeners that we are ... no, we don't. We are not setup to be gardeners, we are not trying, even if we do, I only like grass and screw the other plants. Without SolrCloud, Solr wold be in trouble as well. Brute that it is, it could go a few more rounds. SolrCloud is a ballerina. Doesn't look it, cause we dont take care of it. But it is, and it cannot take the beating that the brute does. - Mark On Tue, Nov 5, 2019 at 5:19 AM Mark Miller <markrmil...@gmail.com> wrote: > Basically I can fix 99% of this without you guys - with simple care and > effort and time that non of you are likely in the circumstances of being > able to duplicate.. Been there done that, made it 100x-1000x faster to boot > and added all kinds of fun. > > But I can't build the rest of Solr. I don't care about facets. So let's > meet half way. > > On Tue, Nov 5, 2019 at 5:14 AM Mark Miller <markrmil...@gmail.com> wrote: > >> There are 10,000 problems here. >> >> So if you eventually land on one possible solution you agree on, we a >> little closer. >> >> There is no problem with the current design. Design's can always be >> improved, sure. I've made this one fast. You won't believe me fast. The low >> hanging fruit is astronomical, there is more fruit above that. >> >> We never focused on performance. Or at least didn't. That's after we >> harden. >> >> Except performance is the key to everything. >> >> SolrCloud is not the only problem. The design of Solr, of SolrCloud, they >> are fine. Change them, I don't care. Later. They are not a problem. >> >> But Solr has as many problems as SolrCloud at this point. This just >> mater a whole hell of lot less unless they are messing with SolrCloud. >> Standalone is more of a brute. >> >> We have 60 modules that are interconnected. We have a huge code base. >> That is also fine. >> >> We don't tend our garden. That's not fine. I've tended the garden before >> without one - more than once before. It's a great damn garden. You guys >> only get to see it grown over and full of weeds. >> >> Anyway, no redesign, no library, no nothing like that gonna save this. >> >> This is hardly concrete awareness of a problem here. The awareness to >> figure out what actually are the problems and what must be done - that's >> expensive shit these days if you ask me. I've been wrong lots tough. >> >> >> >> >> >> >> On Mon, Nov 4, 2019 at 2:26 PM Jörn Franke <jornfra...@gmail.com> wrote: >> >>> I guess this is also a bit normal with software that grows over the >>> years. >>> One could also say that one writes the current use cases and interesting >>> future use cases for Solr in a document and designs from scratch new - >>> taking only the good pieces out of the existing software. >>> Of course there is a certain amount of time where you need to maintain >>> both - but this will be also the case for a major rewrite. >>> >>> > Am 04.11.2019 um 20:58 schrieb Erick Erickson <erickerick...@gmail.com >>> >: >>> > >>> > If Curator would make that easier and we’re doing major surgery >>> anyway…. >>> > >>> > But yeah, a nifty, new, more modern tool isn’t going to magically help >>> if the design is flawed. >>> > >>> > Or, if I’m putting my philosophical hat on, code doesn’t get gnarly >>> intentionally. It gets gnarly because there are a bunch of problems to be >>> solved and you don’t know what they are until you run into them. And it’s >>> always a tension between fixing it enough to get by and fixing it by >>> refactoring/redesign. >>> > >>> > But eventually “fixing it enough to get by” totters under it’s own >>> weight and becomes increasingly fragile and you must take the hit and redo >>> major portions of it. The questions now are: >>> > 1> are we at that point? >>> > 2> are we going to put the effort into rewriting some of the worst >>> offenders? >>> > >>> > >>> > >>> >> On Nov 4, 2019, at 1:28 PM, Scott Blum <dragonsi...@gmail.com> wrote: >>> >> >>> >> Figuring out a better overall algorithmic & data structure design >>> that's an order of magnitude improvement seems far more important than >>> swapping out libraries. And I say this as a Curator fan and committer. ;) >>> >> >>> >> On Mon, Nov 4, 2019 at 11:44 AM Erick Erickson < >>> erickerick...@gmail.com> wrote: >>> >> Bram: >>> >> >>> >> Using Curator has been proposed before. It would require significant >>> refactoring b/c of how deeply entwined raw ZK is in the code. That said, if >>> we’re going to do major surgery it may be the right time to consider it. >>> >> >>> >> Erick >>> >> >>> >>>> On Nov 4, 2019, at 9:24 AM, Bram Van Dam <bram.van...@intix.eu> >>> wrote: >>> >>> >>> >>>> SolrCloud is sick right now. The way low level Zookeeper is handeled >>> >>> >>> >>> On an unrelated project, I've stopped using "raw" ZK client access >>> and >>> >>> have switched to Curator. The API is a fair bit easier to work with, >>> and >>> >>> it results in less ugly code. I realize that this won't go very far >>> in >>> >>> resolving more fundamental issues, but it might be something that can >>> >>> help improve the shape of the code. >>> >>> >>> >>> - Bram >>> >>> >>> >>> --------------------------------------------------------------------- >>> >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>> >>> >> >>> >> >>> >> --------------------------------------------------------------------- >>> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> >> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >> >>> > >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> > For additional commands, e-mail: dev-h...@lucene.apache.org >>> > >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>> >> >> -- >> - Mark >> >> http://about.me/markrmiller >> > > > -- > - Mark > > http://about.me/markrmiller > -- - Mark http://about.me/markrmiller
