On Thu, Aug 11, 2011 at 12:29 PM, John Cheng <johnlich...@gmail.com> wrote:
> On Thu, Aug 11, 2011 at 11:36 AM, John Cheng <johnlich...@gmail.com>wrote: > >> On Thu, Aug 11, 2011 at 11:02 AM, Adam Kocoloski <kocol...@apache.org>wrote: >> >>> Hi John, first thing I'd point out is that Filipe and Paul have done >>> quite a lot of work on the JSON processor since 1.0.2 was released. Trunk >>> contains a JSON processor that calls out to YAJL instead of relying on pure >>> Erlang code. The improvement for large document bodies is pretty >>> substantial. Also, processing MBs of JSON is going to be CPU-bound in >>> 1.0.2. CouchDB makes good use of multicore servers to distribute some of >>> the load if you throw concurrent requests at it. Hope that helps, >>> >>> Adam >>> >> >> It does help here. I'll see if I can get compile the latest version of >> CouchDB on Ubuntu 10 LTS. It's been an adventure trying to compile CouchDB >> so far! >> > > > Just compiled CouchDB 1.1.0 > > Testing with doc target size = "100 kB", # of properties = "10", updates = > "500", numDocs = "100" > each document is approximately 100.1 kB > Using increment handler function to perform update > 500 updates in 42.1960351467 seconds, 11.8494545343 ops/second > > Now the updates dropped from 23 updates/sec to 11.84 updates/second > > Even inserts dropped to 24 inserts/second. Accroding to iostat, CPU was > mostly idle (~85 to 92%) and I'm only getting 300 kB writes per second. > > I guess this could be a problem with httplib2. Is there any configuration > or compile time options I may have missed that could cause such a big drop > in performance? > > -- > --- > John L Cheng > Just to follow up on this. I've pushed a test case written in Scala to Github (https://github.com/jlcheng/couchdb-test). With this test case, I am using Apache HttpClient to perform testing. Since I am more familiar with Java, I was able to optimize the test case more. With Scala and CouchDB 0.10.0 (default install from Ubuntu 10.04 LTS), I am seeing 94 inserts per second, document size of 100k (using batch=ok). I think this pretty close to being optimized. However, going to a custom compiled CouchDB 1.1.0, I am seeing a mere 20 inserts per second. I also see disk activity drop from ~7mb writes per second to ~2mb/sec. There must be something wrong with the way I have it compiled or configured. Any ideas where I should start? -- --- John L Cheng
