Re: [PERFORM] pgbench results on a new server
Craig James wrote: synchronous_commit = off full_page_writes = off I don't have any numbers handy on how much turning synchronous_commit and full_page_writes off improves performance on a system with a battery-backed write cache. Your numbers are therefore a bit inflated against similar ones that are doing a regular sync commit. Just something to keep in mind when comparing against other people's results. Also, just as a general comment, increase in work_mem and effective_cache_size don't actually do anything to the built-in pgbench test results. General numbers are OK, the major drop going from 30 to 40 clients is larger than it should be. I'd suggest running the 40 client count one again to see if that's consistent. It is consistent. When I run pgbench from a different server, I get this: pgbench -c40 -t 2500 -U test tps = 7999 pgbench -c100 -t 1000 -U test tps = 6693 Looks like you're just running into the limitations of the old pgbench code failing to keep up with high client count loads when run on the same system as the server. Nothing to be concerned about--that the drop is only small with the pgbench client remote says there's not actually a server problem here. With that sorted out, your system looks in the normal range for the sort of hardware you're using. I'm always concerned about the potential reliability issues that come with async commit and turning off full page writes though, so you might want to re-test with those turned on and see if you can live with the results. -- Greg Smith 2ndQuadrant US Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] pgbench results on a new server
On Mon, Jun 28, 2010 at 1:12 PM, Craig James wrote: > On 6/25/10 12:03 PM, Greg Smith wrote: >> >> Craig James wrote: >>> >>> I've got a new server and want to make sure it's running well. >> >> Any changes to the postgresql.conf file? Generally you need at least a >> moderate shared_buffers (1GB or so at a minimum) and checkpoint_segments >> (32 or higher) in order for the standard pgbench test to give good >> results. > > max_connections = 500 > shared_buffers = 1000MB > work_mem = 128MB > synchronous_commit = off > full_page_writes = off > wal_buffers = 256kB > checkpoint_segments = 30 > effective_cache_size = 4GB > > For fun I ran it with the installation defaults, and it never got above 1475 > TPS. > >>> pgbench -c20 -t 5000 -U test >>> tps = 5789 >>> pgbench -c30 -t -U test >>> tps = 6961 >>> pgbench -c40 -t 2500 -U test >>> tps = 2945 >> >> General numbers are OK, the major drop going from 30 to 40 clients is >> larger than it should be. I'd suggest running the 40 client count one >> again to see if that's consistent. > > It is consistent. When I run pgbench from a different server, I get this: > > pgbench -c40 -t 2500 -U test > tps = 7999 > > pgbench -c100 -t 1000 -U test > tps = 6693 6k tps over 8 7200 rpm disks is quite good imo. synchronous_commit setting is making that possible. building a server that could handle that much was insanely expensive just a few years ago, on relatively cheap sorage. that's 21m transactions an hour or ~ half a billion transactions a day (!). running this kind of load 24x7 on postgres 7.x would have been an enormous headache. how quickly we forget! :-) your 'real' tps write, 1475 tps, spread over 4 disks doing the actual writing, is giving you ~ 370 tps/device. not bad at all -- the raid controller is doing a good job (the raw drive might get 200 or so). I bet performance will be somewhat worse with a higher scaling factor (say, 500) because there is less locality of writes -- something to consider if you expect your database to get really big. merlin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] pgbench results on a new server
On 6/25/10 12:03 PM, Greg Smith wrote: Craig James wrote: I've got a new server and want to make sure it's running well. Any changes to the postgresql.conf file? Generally you need at least a moderate shared_buffers (1GB or so at a minimum) and checkpoint_segments (32 or higher) in order for the standard pgbench test to give good results. max_connections = 500 shared_buffers = 1000MB work_mem = 128MB synchronous_commit = off full_page_writes = off wal_buffers = 256kB checkpoint_segments = 30 effective_cache_size = 4GB For fun I ran it with the installation defaults, and it never got above 1475 TPS. pgbench -c20 -t 5000 -U test tps = 5789 pgbench -c30 -t -U test tps = 6961 pgbench -c40 -t 2500 -U test tps = 2945 General numbers are OK, the major drop going from 30 to 40 clients is larger than it should be. I'd suggest running the 40 client count one again to see if that's consistent. It is consistent. When I run pgbench from a different server, I get this: pgbench -c40 -t 2500 -U test tps = 7999 pgbench -c100 -t 1000 -U test tps = 6693 Craig -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] pgbench results on a new server
Craig James wrote: I've got a new server and want to make sure it's running well. Any changes to the postgresql.conf file? Generally you need at least a moderate shared_buffers (1GB or so at a minimum) and checkpoint_segments (32 or higher) in order for the standard pgbench test to give good results. pgbench -c20 -t 5000 -U test tps = 5789 pgbench -c30 -t -U test tps = 6961 pgbench -c40 -t 2500 -U test tps = 2945 General numbers are OK, the major drop going from 30 to 40 clients is larger than it should be. I'd suggest running the 40 client count one again to see if that's consistent. If it is, that may just be pgbench itself running into a problem. It doesn't handle high client counts very well unless you use the 9.0 version that supports multiple pgbench workers with the "-j" option. -- Greg Smith 2ndQuadrant US Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] pgbench results on a new server
On Fri, Jun 25, 2010 at 2:53 PM, Craig James wrote: > I've got a new server and want to make sure it's running well. Are these > pretty decent numbers? > > 8 cores (2x4 Intel Nehalem 2 GHz) > 12 GB memory > 12 x 7200 SATA 500 GB disks > 3WARE 9650SE-12ML RAID controller with BBU > WAL on ext2, 2 disks: RAID1 500GB, blocksize=4096 > Database on ext4, 8 disks: RAID10 2TB, stripe size 64K, blocksize=4096 > Ubuntu 10.04 LTS (Lucid) > Postgres 8.4.4 > > pgbench -i -s 100 -U test > pgbench -c 5 -t 2 -U test > tps = 4903 > pgbench -c 10 -t 1 -U test > tps = 4070 > pgbench -c20 -t 5000 -U test > tps = 5789 > pgbench -c30 -t -U test > tps = 6961 > pgbench -c40 -t 2500 -U test > tps = 2945 Numbers are okay, but you likely need much longer tests to see how they average out with the bgwriter / checkpoints happening, and keep track of your IO numbers to see where your dips are. I usually run pgbench runs, once they seem to get decent numbers, for several hours non-stop. Sometimes days during burn in. Note that running pgbench on a machine other than the actual db is often a good idea so you're not measuring how fast pgbench can run in contention with your own database. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
[PERFORM] pgbench results on a new server
I've got a new server and want to make sure it's running well. Are these pretty decent numbers? 8 cores (2x4 Intel Nehalem 2 GHz) 12 GB memory 12 x 7200 SATA 500 GB disks 3WARE 9650SE-12ML RAID controller with BBU WAL on ext2, 2 disks: RAID1 500GB, blocksize=4096 Database on ext4, 8 disks: RAID10 2TB, stripe size 64K, blocksize=4096 Ubuntu 10.04 LTS (Lucid) Postgres 8.4.4 pgbench -i -s 100 -U test pgbench -c 5 -t 2 -U test tps = 4903 pgbench -c 10 -t 1 -U test tps = 4070 pgbench -c20 -t 5000 -U test tps = 5789 pgbench -c30 -t -U test tps = 6961 pgbench -c40 -t 2500 -U test tps = 2945 Thanks, Craig -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
