I'll tell you what guys, development right now sucks. I don't enjoy. But when I start to put things in shape? I get this smile, and I start going with the feeling of I don't need you guys, I don't users, I don't need a job, cause just this is figgen nice.
On Tue, Nov 5, 2019 at 9:59 AM Mark Miller <markrmil...@gmail.com> wrote: > I suppose I should toss one more out. > > Hell yes, we will be using curator. > > It's insane for any group larger than 2-3 to directly use ZooKeeper. Even > for that group, you want some damn good reasons to not use curator. We can > start using more assembly too (joke Yonik). > > Curator was an option initially. Then it was yet another project hosted by > Netflix. Now it is essential. > > > - Mark > > On Tue, Nov 5, 2019 at 9:41 AM Mark Miller <markrmil...@gmail.com> wrote: > >> And look, we started pretty deep in the hole. Solr started with tons of >> bug or limitations that hardly mattered to it and hit SolrCloud in the eye >> like a train. And we were not setup to deal with that. >> >> We never had a nice garden for SolrCloud. We started in a mess, thinking, >> eventually we clear the overgrowth, and we are all good. And then we >> started building our house and that garden went wild with a life of it's >> own. >> >> And our development practices, amazingly above many many many groups and >> standards out there, is woefully inaccurate for what we are doing. >> >> "Test pass, I'm not sure about all this but I'm going to commit" (Tests >> never pass, must be a lie anyway) >> "Leaving on vacation, going to fire this in" >> "No one has looked at this huge thing, it's been a while, going to commit" >> *commit* >> >> And comments to that affect pretty much wrap up our careful and >> thoughtful attitude. >> >> And then of course we come and clean up after, careful gardeners that we >> are ... no, we don't. We are not setup to be gardeners, we are not trying, >> even if we do, I only like grass and screw the other plants. >> >> Without SolrCloud, Solr wold be in trouble as well. Brute that it is, it >> could go a few more rounds. SolrCloud is a ballerina. Doesn't look it, >> cause we dont take care of it. But it is, and it cannot take the beating >> that the brute does. >> >> - Mark >> >> On Tue, Nov 5, 2019 at 5:19 AM Mark Miller <markrmil...@gmail.com> wrote: >> >>> Basically I can fix 99% of this without you guys - with simple care and >>> effort and time that non of you are likely in the circumstances of being >>> able to duplicate.. Been there done that, made it 100x-1000x faster to boot >>> and added all kinds of fun. >>> >>> But I can't build the rest of Solr. I don't care about facets. So let's >>> meet half way. >>> >>> On Tue, Nov 5, 2019 at 5:14 AM Mark Miller <markrmil...@gmail.com> >>> wrote: >>> >>>> There are 10,000 problems here. >>>> >>>> So if you eventually land on one possible solution you agree on, we a >>>> little closer. >>>> >>>> There is no problem with the current design. Design's can always be >>>> improved, sure. I've made this one fast. You won't believe me fast. The low >>>> hanging fruit is astronomical, there is more fruit above that. >>>> >>>> We never focused on performance. Or at least didn't. That's after we >>>> harden. >>>> >>>> Except performance is the key to everything. >>>> >>>> SolrCloud is not the only problem. The design of Solr, of SolrCloud, >>>> they are fine. Change them, I don't care. Later. They are not a problem. >>>> >>>> But Solr has as many problems as SolrCloud at this point. This just >>>> mater a whole hell of lot less unless they are messing with SolrCloud. >>>> Standalone is more of a brute. >>>> >>>> We have 60 modules that are interconnected. We have a huge code base. >>>> That is also fine. >>>> >>>> We don't tend our garden. That's not fine. I've tended the garden >>>> before without one - more than once before. It's a great damn garden. You >>>> guys only get to see it grown over and full of weeds. >>>> >>>> Anyway, no redesign, no library, no nothing like that gonna save this. >>>> >>>> This is hardly concrete awareness of a problem here. The awareness to >>>> figure out what actually are the problems and what must be done - that's >>>> expensive shit these days if you ask me. I've been wrong lots tough. >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Mon, Nov 4, 2019 at 2:26 PM Jörn Franke <jornfra...@gmail.com> >>>> wrote: >>>> >>>>> I guess this is also a bit normal with software that grows over the >>>>> years. >>>>> One could also say that one writes the current use cases and >>>>> interesting future use cases for Solr in a document and designs from >>>>> scratch new - taking only the good pieces out of the existing software. >>>>> Of course there is a certain amount of time where you need to maintain >>>>> both - but this will be also the case for a major rewrite. >>>>> >>>>> > Am 04.11.2019 um 20:58 schrieb Erick Erickson < >>>>> erickerick...@gmail.com>: >>>>> > >>>>> > If Curator would make that easier and we’re doing major surgery >>>>> anyway…. >>>>> > >>>>> > But yeah, a nifty, new, more modern tool isn’t going to magically >>>>> help if the design is flawed. >>>>> > >>>>> > Or, if I’m putting my philosophical hat on, code doesn’t get gnarly >>>>> intentionally. It gets gnarly because there are a bunch of problems to be >>>>> solved and you don’t know what they are until you run into them. And it’s >>>>> always a tension between fixing it enough to get by and fixing it by >>>>> refactoring/redesign. >>>>> > >>>>> > But eventually “fixing it enough to get by” totters under it’s own >>>>> weight and becomes increasingly fragile and you must take the hit and redo >>>>> major portions of it. The questions now are: >>>>> > 1> are we at that point? >>>>> > 2> are we going to put the effort into rewriting some of the worst >>>>> offenders? >>>>> > >>>>> > >>>>> > >>>>> >> On Nov 4, 2019, at 1:28 PM, Scott Blum <dragonsi...@gmail.com> >>>>> wrote: >>>>> >> >>>>> >> Figuring out a better overall algorithmic & data structure design >>>>> that's an order of magnitude improvement seems far more important than >>>>> swapping out libraries. And I say this as a Curator fan and committer. ;) >>>>> >> >>>>> >> On Mon, Nov 4, 2019 at 11:44 AM Erick Erickson < >>>>> erickerick...@gmail.com> wrote: >>>>> >> Bram: >>>>> >> >>>>> >> Using Curator has been proposed before. It would require >>>>> significant refactoring b/c of how deeply entwined raw ZK is in the code. >>>>> That said, if we’re going to do major surgery it may be the right time to >>>>> consider it. >>>>> >> >>>>> >> Erick >>>>> >> >>>>> >>>> On Nov 4, 2019, at 9:24 AM, Bram Van Dam <bram.van...@intix.eu> >>>>> wrote: >>>>> >>> >>>>> >>>> SolrCloud is sick right now. The way low level Zookeeper is >>>>> handeled >>>>> >>> >>>>> >>> On an unrelated project, I've stopped using "raw" ZK client access >>>>> and >>>>> >>> have switched to Curator. The API is a fair bit easier to work >>>>> with, and >>>>> >>> it results in less ugly code. I realize that this won't go very >>>>> far in >>>>> >>> resolving more fundamental issues, but it might be something that >>>>> can >>>>> >>> help improve the shape of the code. >>>>> >>> >>>>> >>> - Bram >>>>> >>> >>>>> >>> >>>>> --------------------------------------------------------------------- >>>>> >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>> >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>> >>> >>>>> >> >>>>> >> >>>>> >> >>>>> --------------------------------------------------------------------- >>>>> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>> >> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>> >> >>>>> > >>>>> > >>>>> > --------------------------------------------------------------------- >>>>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>> > For additional commands, e-mail: dev-h...@lucene.apache.org >>>>> > >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>> >>>>> >>>> >>>> -- >>>> - Mark >>>> >>>> http://about.me/markrmiller >>>> >>> >>> >>> -- >>> - Mark >>> >>> http://about.me/markrmiller >>> >> >> >> -- >> - Mark >> >> http://about.me/markrmiller >> > > > -- > - Mark > > http://about.me/markrmiller > -- - Mark http://about.me/markrmiller
