And now we are to meat of it. Fill in here: https://issues.apache.org/jira/browse/SOLR-13888
We can play in a better world, we can have fun, but some of you are going to have to adjust your ways. In the most convenient way possible. You are all great people, I don't want to cause you annoyance, but there are certain requirements to building an aircraft, and there certain requirements to building this. On Tue, Nov 5, 2019 at 10:44 AM Mark Miller <markrmil...@gmail.com> wrote: > If you had any idea how much suffering just that has caused. Not just > users, but us. > > Mark > > On Tue, Nov 5, 2019 at 10:38 AM Mark Miller <markrmil...@gmail.com> wrote: > >> It’s like 6-7 years since I quickly added a shitty collections API in my >> free time because we desperately needed SOMETHING. I don’t know if I tried >> to make it wait for proper state or what , it was a stub to try get things >> moving. That call, to this day, along with all our other checks, until some >> tests ones recently, is garbage. >> >> If I downloaded a database, and a lot the time, after the create a >> database call returned, my database was not ready, I’d saw wow. Terrible >> bug got through. If it was a persistent issue for over half a decade? My >> god. >> >> Look I just spent that half decade upgrading from Solr 4 to whatever. I >> was mostly out of the loop. But this is crazy, me in there too. >> >> Mark >> >> On Tue, Nov 5, 2019 at 10:05 AM Mark Miller <markrmil...@gmail.com> >> wrote: >> >>> I'll tell you what guys, development right now sucks. I don't enjoy. >>> >>> But when I start to put things in shape? I get this smile, and I start >>> going with the feeling of I don't need you guys, I don't users, I don't >>> need a job, cause just this is figgen nice. >>> >>> On Tue, Nov 5, 2019 at 9:59 AM Mark Miller <markrmil...@gmail.com> >>> wrote: >>> >>>> I suppose I should toss one more out. >>>> >>>> Hell yes, we will be using curator. >>>> >>>> It's insane for any group larger than 2-3 to directly use ZooKeeper. >>>> Even for that group, you want some damn good reasons to not use curator. We >>>> can start using more assembly too (joke Yonik). >>>> >>>> Curator was an option initially. Then it was yet another project hosted >>>> by Netflix. Now it is essential. >>>> >>>> >>>> - Mark >>>> >>>> On Tue, Nov 5, 2019 at 9:41 AM Mark Miller <markrmil...@gmail.com> >>>> wrote: >>>> >>>>> And look, we started pretty deep in the hole. Solr started with tons >>>>> of bug or limitations that hardly mattered to it and hit SolrCloud in the >>>>> eye like a train. And we were not setup to deal with that. >>>>> >>>>> We never had a nice garden for SolrCloud. We started in a mess, >>>>> thinking, eventually we clear the overgrowth, and we are all good. And >>>>> then >>>>> we started building our house and that garden went wild with a life of >>>>> it's >>>>> own. >>>>> >>>>> And our development practices, amazingly above many many many groups >>>>> and standards out there, is woefully inaccurate for what we are doing. >>>>> >>>>> "Test pass, I'm not sure about all this but I'm going to commit" >>>>> (Tests never pass, must be a lie anyway) >>>>> "Leaving on vacation, going to fire this in" >>>>> "No one has looked at this huge thing, it's been a while, going to >>>>> commit" >>>>> *commit* >>>>> >>>>> And comments to that affect pretty much wrap up our careful and >>>>> thoughtful attitude. >>>>> >>>>> And then of course we come and clean up after, careful gardeners that >>>>> we are ... no, we don't. We are not setup to be gardeners, we are not >>>>> trying, even if we do, I only like grass and screw the other plants. >>>>> >>>>> Without SolrCloud, Solr wold be in trouble as well. Brute that it is, >>>>> it could go a few more rounds. SolrCloud is a ballerina. Doesn't look it, >>>>> cause we dont take care of it. But it is, and it cannot take the beating >>>>> that the brute does. >>>>> >>>>> - Mark >>>>> >>>>> On Tue, Nov 5, 2019 at 5:19 AM Mark Miller <markrmil...@gmail.com> >>>>> wrote: >>>>> >>>>>> Basically I can fix 99% of this without you guys - with simple care >>>>>> and effort and time that non of you are likely in the circumstances of >>>>>> being able to duplicate.. Been there done that, made it 100x-1000x faster >>>>>> to boot and added all kinds of fun. >>>>>> >>>>>> But I can't build the rest of Solr. I don't care about facets. So >>>>>> let's meet half way. >>>>>> >>>>>> On Tue, Nov 5, 2019 at 5:14 AM Mark Miller <markrmil...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> There are 10,000 problems here. >>>>>>> >>>>>>> So if you eventually land on one possible solution you agree on, we >>>>>>> a little closer. >>>>>>> >>>>>>> There is no problem with the current design. Design's can always be >>>>>>> improved, sure. I've made this one fast. You won't believe me fast. The >>>>>>> low >>>>>>> hanging fruit is astronomical, there is more fruit above that. >>>>>>> >>>>>>> We never focused on performance. Or at least didn't. That's after we >>>>>>> harden. >>>>>>> >>>>>>> Except performance is the key to everything. >>>>>>> >>>>>>> SolrCloud is not the only problem. The design of Solr, of SolrCloud, >>>>>>> they are fine. Change them, I don't care. Later. They are not a problem. >>>>>>> >>>>>>> But Solr has as many problems as SolrCloud at this point. This just >>>>>>> mater a whole hell of lot less unless they are messing with SolrCloud. >>>>>>> Standalone is more of a brute. >>>>>>> >>>>>>> We have 60 modules that are interconnected. We have a huge code >>>>>>> base. That is also fine. >>>>>>> >>>>>>> We don't tend our garden. That's not fine. I've tended the garden >>>>>>> before without one - more than once before. It's a great damn garden. >>>>>>> You >>>>>>> guys only get to see it grown over and full of weeds. >>>>>>> >>>>>>> Anyway, no redesign, no library, no nothing like that gonna save >>>>>>> this. >>>>>>> >>>>>>> This is hardly concrete awareness of a problem here. The awareness >>>>>>> to figure out what actually are the problems and what must be done - >>>>>>> that's >>>>>>> expensive shit these days if you ask me. I've been wrong lots tough. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Nov 4, 2019 at 2:26 PM Jörn Franke <jornfra...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> I guess this is also a bit normal with software that grows over the >>>>>>>> years. >>>>>>>> One could also say that one writes the current use cases and >>>>>>>> interesting future use cases for Solr in a document and designs from >>>>>>>> scratch new - taking only the good pieces out of the existing software. >>>>>>>> Of course there is a certain amount of time where you need to >>>>>>>> maintain both - but this will be also the case for a major rewrite. >>>>>>>> >>>>>>>> > Am 04.11.2019 um 20:58 schrieb Erick Erickson < >>>>>>>> erickerick...@gmail.com>: >>>>>>>> > >>>>>>>> > If Curator would make that easier and we’re doing major surgery >>>>>>>> anyway…. >>>>>>>> > >>>>>>>> > But yeah, a nifty, new, more modern tool isn’t going to magically >>>>>>>> help if the design is flawed. >>>>>>>> > >>>>>>>> > Or, if I’m putting my philosophical hat on, code doesn’t get >>>>>>>> gnarly intentionally. It gets gnarly because there are a bunch of >>>>>>>> problems >>>>>>>> to be solved and you don’t know what they are until you run into them. >>>>>>>> And >>>>>>>> it’s always a tension between fixing it enough to get by and fixing it >>>>>>>> by >>>>>>>> refactoring/redesign. >>>>>>>> > >>>>>>>> > But eventually “fixing it enough to get by” totters under it’s >>>>>>>> own weight and becomes increasingly fragile and you must take the hit >>>>>>>> and >>>>>>>> redo major portions of it. The questions now are: >>>>>>>> > 1> are we at that point? >>>>>>>> > 2> are we going to put the effort into rewriting some of the >>>>>>>> worst offenders? >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> >> On Nov 4, 2019, at 1:28 PM, Scott Blum <dragonsi...@gmail.com> >>>>>>>> wrote: >>>>>>>> >> >>>>>>>> >> Figuring out a better overall algorithmic & data structure >>>>>>>> design that's an order of magnitude improvement seems far more >>>>>>>> important >>>>>>>> than swapping out libraries. And I say this as a Curator fan and >>>>>>>> committer. ;) >>>>>>>> >> >>>>>>>> >> On Mon, Nov 4, 2019 at 11:44 AM Erick Erickson < >>>>>>>> erickerick...@gmail.com> wrote: >>>>>>>> >> Bram: >>>>>>>> >> >>>>>>>> >> Using Curator has been proposed before. It would require >>>>>>>> significant refactoring b/c of how deeply entwined raw ZK is in the >>>>>>>> code. >>>>>>>> That said, if we’re going to do major surgery it may be the right time >>>>>>>> to >>>>>>>> consider it. >>>>>>>> >> >>>>>>>> >> Erick >>>>>>>> >> >>>>>>>> >>>> On Nov 4, 2019, at 9:24 AM, Bram Van Dam <bram.van...@intix.eu> >>>>>>>> wrote: >>>>>>>> >>> >>>>>>>> >>>> SolrCloud is sick right now. The way low level Zookeeper is >>>>>>>> handeled >>>>>>>> >>> >>>>>>>> >>> On an unrelated project, I've stopped using "raw" ZK client >>>>>>>> access and >>>>>>>> >>> have switched to Curator. The API is a fair bit easier to work >>>>>>>> with, and >>>>>>>> >>> it results in less ugly code. I realize that this won't go very >>>>>>>> far in >>>>>>>> >>> resolving more fundamental issues, but it might be something >>>>>>>> that can >>>>>>>> >>> help improve the shape of the code. >>>>>>>> >>> >>>>>>>> >>> - Bram >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>>>>> >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>>>>> >>> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>>>>> >> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>>>>> >> >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> --------------------------------------------------------------------- >>>>>>>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>>>>> > For additional commands, e-mail: dev-h...@lucene.apache.org >>>>>>>> > >>>>>>>> >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> - Mark >>>>>>> >>>>>>> http://about.me/markrmiller >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> - Mark >>>>>> >>>>>> http://about.me/markrmiller >>>>>> >>>>> >>>>> >>>>> -- >>>>> - Mark >>>>> >>>>> http://about.me/markrmiller >>>>> >>>> >>>> >>>> -- >>>> - Mark >>>> >>>> http://about.me/markrmiller >>>> >>> >>> >>> -- >>> - Mark >>> >>> http://about.me/markrmiller >>> >> -- >> - Mark >> >> http://about.me/markrmiller >> > -- > - Mark > > http://about.me/markrmiller > -- - Mark http://about.me/markrmiller
