So it’s been a while since I’ve brought up SolrCloud sickness. Plenty to navigate and figure out in the meantime. Given the constraints of life, there was a point I wanted to give and share some insight into what I could see. But it quickly became clear that was not a great plan - just what I was left with because I could not dedicate or plan beyond a small or very long time range. It turns out that telling someone, take a peak through these key holes, can you see what I see? 10-100x better!? The skys the limit? Don’t you see it?! What? The bathrooms? Yeah yeah, no the bathrooms are being demolished, look here through the key hole. More units tests and modules you say? Exaggeration confusion? Yeah yeah, never mind.
None the less, you do not tear down what you don’t have a plan to address. And you cannot start anew until you have acknowledged and accepted the past. Well, who knows, I have my own codes. What’s a man/woman without a code. And so, while maybe I’m always wrong or unintelligent, I’ve only ever expressed a truth I feel I could and can defend, if the chips started falling. And at this point, I’d like to say that SolrCloud might be sic. There is some ongoing work and plenty that will be happening for some time. But I have not tried to create a collection or many collections other than mostly around the small numbers that tests do (with the exception of 100 collections, created over time, sometimes with Ishans stress system). A couple to a few collections. A handful of shards. A handful of replicas. The other day I figured I’d modify a test just to see what I’d be dealing with when I get some fun times soon. I used a single 512mb test jvm, nightly settings and fired up a 12x12 collection on 4 jetty Solr runners. 144 SolrCores. It essentially just started up and returned. Shit, is that broken? The test is green? No way. Ok. I would have bet 0 dollars on that first run. Yesterday, okay, I’m feeling good, let’s roll the dice. I fire up 24 shards and 24 replicas for each. I’m not even optimistic, I mean this is cold and blind at these numbers. And after what looks like a nice start, things quickly deteriorate. Ok, ok, looks like maybe a limit adjust and maybe I’m asking a little much of 512mb of Ram. So I just bump it to 2 gig for some play headroom. Bam. 6 seconds. Fully green and active cluster. Sic. That’s two runs, so yeah, I’ve got a future play date scheduled for more. But, one, that’s indicative of a ridiculous amount of Solr code and behavior. It’s a load bearing beam of action. But also, I’ve got similar sicness buried all over the place. So yeah, the system can rock. Yonik’s design sense was not the monster after all, surprise ending. And interested parties could and will help take it forward so very much further. Don’t mortgage the house yet, an Apache release of my play time is not happening next week. But with the same shock and horror and surety that I said SolrCloud looks sick, I revise to SolrCloud looks sic. And it wants a second shot, and by god it will have it. -- - Mark http://about.me/markrmiller
