Dear Lucene/Solr Community, I have been searching for an answer for Solr and SolrCloud for a long time. I feel like I landed in a tornado and I don’t know where the time went. I forget even why I’m here. Because I didn’t come here to work for silicon valley companies, or make a lot of money, or impress people I don’t know. I came here for Lucene. I love Lucene. I love developing. I love Lucene tests. I don’t do much Lucene anymore. I was needed more in Solr, and someone started acting like a dictator.
I still love Lucene. I’ve tried to love to Solr. But I don’t. And so I’ve been searching for an answer, when not being depressed about it, and as often happens, it was right in front of me. So yeah, a couple times when I got sick of you guys - which is no one and everyone - I went off on my own and started chasing one of my own itches, which leads to things, which leads to things, which leads things. I love that I have no idea at the start. Anyway, after time and some learning I kind of got to the point where I knew enough about the stupid technologies and the whole system - it’s like a lot of code, a lot of debt, blah blah. But I’m banging my head against this - intuition guy - like, just bang bang bang, starts to make sense and I don’t even do any work. So starts to makes sense. I start to address this. And that. I make some progress. I find some things. I say screw working on making this work anymore, it’s impossible, I’m sick of it, I’m finally gonna do the thing I love. Make it fast. So I start making it fast here and there, sometimes. Most efforts are in like 3-4-5 different huge sprints or something - but always efforts around that. You know the lost work story. Lot of lost work. I usually don’t duplicate all the work when I make another attempt. I have enough memories that that is not the important part. The importance is that I learned that none of you you know anything about this system or the components that make it up. I didn’t either. I knew more than a lot of you, but not early enough. And you guys have worked on the very edges on some great necessary stuff and tools - and I take heavy goddamn advantage of those things. Thank you. And I add things. And I track things. And I turn on enforcers. And pluck away. And I strip out all our darn randomization or craziness test hierarchy (or start to try and control it), and I start adding logging that's useful, and debug logging, and I use a good profiler, and I start limiting resources and minimizing shit, until I have a system that I can start to understand and work through. And I spend almost just as much on making myself efficient, cause it’s big. But. All basic stuff. Maybe I’m smart somewhere, maybe I’m not. I’m lazy. I don’t think. I’m a math minor and most can probably attest I will not do a 1 dollar tip in my head. So I’m just learning about the system, the components, plucking away, cleaning up, finding bugs, adding stuff that will allow me to understand. Starting with basic tests, and like shooting for high goals. I want to be able to start 500 solrcores in 10-15 seconds in a single corecontainer. Thats what I want. So sometimes I work towards. Brings out a lot of great stuff. But the solution is neither fancy or some huge credit to me. We dont know anything, we have no good enforcement really, and we make it too crazy and wild when it's already crazy and wild and the it’s all way more than any human can realistically do anything with. Now I wrote a lot of this foundation. It’s not easy for people to take me seriously when I say its cause we are shit software developers. “Haha, you say cocreator, your software, please tell me how I am the one that sucks”. Even I had no confidence this could work so well compared to what was happening. I had to basically get there. Get there again cause then I didn't care, and then get close again. Like, I don’t trust myself or brain. So I didn’t need everything - god my knowledge and code is so spread around - but it’s not important. The design not important. I’d like you to have whatever design you want. But I know this one can work good enough to get you to the next one, and you need to conquer these demons before you can do anything on Solr. Mark
