Thanks Jason. I removed the EBS volume and things seemed a little faster. CouchDB is actually saturating the CPU now, so it looks like I'm getting "accurate" benchmarks now. i.e. I'm actually exercising CouchDB and I'm not limited by some hardware/OS latency.
So I had 10 concurrent ruby processes performing batch inserts into two test databases. After 2 hours, I have 1.1M documents (1.5 GB) in each database, for a total of 2GB and 2.2 million documents. Does this still seem slow? Thanks. -Tom On Mon, May 4, 2009 at 5:29 PM, Jason Smith <[email protected]> wrote: > Tom Nichols wrote: >> >> Hi, I have some questions about insert performance. >> >> I have a single CouchDB 0.9.0 node running on small EC2 instance. I >> attached a huge EBS volume to it and mounted it where CouchDB's data >> files are stored. I fired up about ruby scripts running inserts and >> after a weekend I only have about 30GB/ 12M rows of data... Which >> seems small. 'top' tells me that my CPU is only about 30% utilized. >> >> Any idea what I might be doing wrong? I pretty much just followed >> these instructions: >> http://wiki.apache.org/couchdb/Getting_started_with_Amazon_EC2 > > Hi, Tom. I believe I read somewhere before that the smallest EC2 instances > have a slower and/or higher-latency connection to EBS, so you might want to > consider a large instance, or maybe even a high-memory small instance and > see whether you get better "hardware" performance. > > Although strangely, when googling it, the first article I found says that > their benchmarks found no difference between EBS or even the ephemeral > filesystem. > > http://www.paessler.com/blog/2009/04/07/prtg-7/monitoring-cloud-performance-with-prtg-comparing-disk-speed-for-instance-stores-and-ebs-volumes-on-amazon-ec2/ > > On the other hand, here is a forum posting and a random benchmark indicating > that more expensive instances get better throughput: > > http://developer.amazonwebservices.com/connect/message.jspa?messageID=125197 > http://blog.getasysadmin.com/2009/02/mysql-benchmarks-using-amazon-ec2.html > > -- > Jason Smith > Proven Corporation > Bangkok, Thailand > http://www.proven-corporation.com >
