Well, my understanding for that specific point was that CouchDB was crashing while writing the docs. That can be because of an error appeared with high frequency or because the queue of documents to be written became too long. The first is with very low chances if there is no attack (remember, in that context, CouchDB crash was registered while inserting docs in the database, not during compaction or building a view or anything else), while the second can easily appear when the traffic is higher than what the harddisk can digest. That made me think pretty much of the conditions I spoke about.
If you want to reproduce the failure, just try to add single-doc requests at a very high rate (standard behavior of a monitored gen_server with listener in Erlang when the queued messages go beyond the predefined length). But, that is something you cannot avoid if you work directly on the hdd in whatever language you may develop the database (generally, queued messages are RAM usage). This is the price CouchDB has to pay for securing the data. So, if someone uses CouchDB, he/she should be aware of that risk, unless either that someone doesn't read the CouchDB documentation or that someone doesn't have any knowledge about hardware. In either case, that someone shouldn't start speaking about such a so called failure, don't you think? I would be embarrassed to report such a "problem" unless I am sure it's coming from other sources then the ones I spoke about. As far as I could notice, those guys didn't provide any detail about what would cause that problem. Even worse, they admitted that by increasing the speed of the harddisk, they noticed a slight improvement of the CouchDB capabilities (of course, if you improve the hdd speed, you increase the rate of document insertion which keeps the queue shorter unless you start again to increase the speed of insertion). So, it's either me, or those guys didn't understand CouchDB usage (considering they know something about hardware). And that leads to the unfortunate conclusion that those guys chose a product without thinking if that product fitted their project design or not by conception. I do not say CouchDB cannot be used at high rate document insertion, but one needs to pay attention how this can be achieved. Also, the fact that they reported some bugs they discovered and they got fast solutions from the devs proves that the devs are serious about this product and for that they should deserve congratulations. I do not want to offend those guys there because I know nothing about them, but they should try to understand that coming with such "advertisement", they need to be sure they have no blame there. I would write that kind of article after I stopped being pissed off. But that is me and, unfortunately, they acted too soon in my opinion. CGS On Fri, May 11, 2012 at 5:32 PM, Eli Stevens (Gmail) <wickedg...@gmail.com>wrote: > On Fri, May 11, 2012 at 7:57 AM, CGS <cgsmcml...@gmail.com> wrote: > > What I don't understand is the followings: > > 1. Those guys wanted a single front-end server which should keep up with > > the incoming requests, correct? As far as I understood, CouchDB > philosophy > > is based on safety of the data, which was implemented as direct writes on > > harddisk. So, having only one front-end server on which you force its hdd > > to to keep up with the high speed internet connection is just like you > want > > to force a river to flow only through a mouse hole. > > From my understanding of the post, the core issue wasn't a mismatch in > scale between desired throughput (the river) and available throughput > (the mousehole), it was that under high enough load CouchDB stopped > honoring certain classes of requests entirely. That's not a "too > slow" problem, it's a "fell over and can't get up" problem. > > I think it's very important that effort is made to reproduce and > address these issues, since without being able to put more definite > bounds on them, *everyone* is going to wonder if their load is high > enough to trigger the problems. Heck, I'm wondering it, and I don't > typically have more than a couple hundred docs per DB (but a lot of > DBs, and hundreds of megs of attachments per DB). > > Eli >
