Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-29 Thread Mike Bresnahan
In an attempt to determine whether top(1) is lying about the CPU utilization, I did an experiment. I fired up a EC2 c1.xlarge instance and ran pgbench and a tight loop in parallel. -bash-4.0$ uname -a Linux domu-12-31-39-00-8d-71.compute-1.internal 2.6.31-302-ec2 #7-Ubuntu SMP Tue Oct 13 19:55:22

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-28 Thread Mike Bresnahan
Greg Smith 2ndquadrant.com> writes: > Looks to me like you're running into a general memory bandwidth issue > here, possibly one that's made a bit worse by how pgbench works. It's a > somewhat funky workload Linux systems aren't always happy with, although > one of your tests had the right co

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-28 Thread Mike Bresnahan
Jim Mlodgenski gmail.com> writes: > Let's start from the beginning. Have you tuned your postgresql.conf file? What do you have shared_buffers set to? That would have the biggest effect on a test like this.  shared_buffers = 128MB maintenance_work_mem = 256MB checkpoint_segments = 20 -- Sent v

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread Mike Bresnahan
Greg Smith 2ndquadrant.com> writes: > Could you try this again with "top -c", which will label these > postmaster processes usefully, and include the pgbench client itself in > what you post? It's hard to sort out what's going on in these > situations without that style of breakdown. As a fur

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread Mike Bresnahan
> Could you try this again with "top -c", which will label these > postmaster processes usefully, and include the pgbench client itself in > what you post? It's hard to sort out what's going on in these > situations without that style of breakdown. I had run pgbench on a separate instance last

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread Mike Bresnahan
John R Pierce hogranch.com> writes: > more likely, he's disk IO bound, but hard to say as that iostat output > only showed a couple 2 second slices of work. the first output, which > shows average since system startup, seems to show the system has had > relatively high average wait times of 1

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread Mike Bresnahan
Jim Mlodgenski gmail.com> writes: > I have seen behavior like this in the past on EC2. I believe your bottleneck may be pulling the data out of cache. I benchmarked this a while back and found that memory speeds are not much faster than disk speeds on EC2. I am not sure if that is true of Xen in g

[GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread Mike Bresnahan
I have deployed PostgresSQL 8.4.1 on a Fedora 9 c1.xlarge (8x1 cores) instance in the Amazon E2 Cloud. When I run pgbench in read-only mode (-S) on a small database, I am unable to peg the CPUs no matter how many clients I throw at it. In fact, the CPU utilization never drops below 60% idle. I also

Re: [GENERAL] SMP Read-only Performance

2010-01-26 Thread Mike Bresnahan
Greg Smith 2ndquadrant.com> writes: > You're probably running into this problem: > http://notemagnet.blogspot.com/2008/05/pgbench-suffering-with-linux-2623-2626.html You are so right. The last thing I would have suspected is a kernel bug. I am definitely going to try to be more aware of kernel

[GENERAL] SMP Read-only Performance

2010-01-26 Thread Mike Bresnahan
I have a read-only database that I am testing the performance of to get a sense of how many concurrent users I can support. The database fits entirely in RAM so I expect there to be little to no disk activity. Because of this, I expect throughput to scale almost linearly with the number of CPUs I h