The first thing is to ensure you have disabled delayed commits; curl -XPUT -d '"false" localhost:5984/_config/couchdb/delayed_commits
This is the production setting anyway (though not the default because of complaints from incompetent benchmarkers). This will ensure an fsync for each write and, as a consequence, will greatly smooth your insert performance. Since you said you were inserting concurrently, you should not experience a slowdown either. B. On 11 May 2012 02:42, Marco Monteiro <[email protected]> wrote: > Hello! > > I'm running a load test on CouchDB. I have a cluster of 8 node.js servers > writing to > CouchDB. They write about 30000 documents per minute (500 per second). > There are > multiple concurrent requests form each server. There are no updates: > documents are > created and not modified. > > I first tried CouchDB 1.1.1 from Debian 6.4 apt repo. After a few minutes, > CouchDB > starts freezing for a period of 1 to 3 seconds about every 10 seconds. It > keeps this > behaviour for some time and eventually it starts freezing more frequently > and for longer > periods. When the database has about 1.5 million documents, couchdb is > freezing for > more than 5 seconds each time. > > I then tried CouchDB 1.2, from build-couch. The freezes happen with it > also, but the > behavior is much worse: in less than one minute it's freezing for 5 seconds > or more, > and it spends more time not doing anything than working. > > When testing with 1.1.1 I was writing only to one database. With 1.2 I > tried with one database > and then with multiple databases but the problem was exactly the same. > > The documents have about 10 properties, only numbers or string and the > strings are small > (about 20 chars each). The document IDs are generated in the app and have > the format > > <milliseconds since epoch>-<random 16 chars string> > > When CouchDB freezes, it's processor use (from top) goes to zero. It does > not reply to read or write > requests. The disk does not seem to be the problem as iostat reports near > 0% utilization. > CPU is mostly idle, and from the 16 GB of RAM, some of it is free and is > not even used to > cache disk. > > There are no error message in Couchdb log. > > I tried this in two different machines and the problem is the same in both. > > I did not change anything in the configuration files expect changing the > database dir to use > a RAID partition. > > Anyone has any idea of what the problem could be? > > Any help solving this issue is greatly appreciated. > > Thanks, > Marco
