[PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-14 Thread Carlo Stonebanks
My client just informed me that new hardware is available for our DB server. . Intel Core 2 Quads Quad . 48 GB RAM . 4 Disk RAID drive (RAID level TBD) I have put the ugly details of what we do with our DB below, as well as the postgres.conf settings. But, to summarize: we have a PostgreSQL 8.3

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-14 Thread Joshua D. Drake
On Thu, 14 Jan 2010 14:17:13 -0500, "Carlo Stonebanks" wrote: > My client just informed me that new hardware is available for our DB > server. > > . Intel Core 2 Quads Quad > . 48 GB RAM > . 4 Disk RAID drive (RAID level TBD) > > I have put the ugly details of what we do with our DB below, as we

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-14 Thread Scott Marlowe
On Thu, Jan 14, 2010 at 12:17 PM, Carlo Stonebanks wrote: > My client just informed me that new hardware is available for our DB server. > > . Intel Core 2 Quads Quad > . 48 GB RAM > . 4 Disk RAID drive (RAID level TBD) > > I have put the ugly details of what we do with our DB below, as well as th

[PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-14 Thread Carlo Stonebanks
My client just informed me that new hardware is available for our DB server. . Intel Core 2 Quads Quad . 48 GB RAM . 4 Disk RAID drive (RAID level TBD) I have put the ugly details of what we do with our DB below, as well as the postgres.conf settings. But, to summarize: we have a PostgreSQL 8.3.

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-14 Thread Greg Smith
Carlo Stonebanks wrote: 1) Which RAID level would you recommend It looks like you stepped over a critical step, which is "will the server have a good performing RAID card?". Your whole upgrade could underperform if you make a bad mistake on that part. It's really important to nail that dow

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-14 Thread Carlo Stonebanks
Pretty much everyone thinks their requirements are exceptional. It's funny how infrequently that's actually true. The techniques that favor index-use aren't that unique: collect better stats, set basic parameters correctly, adjust random_page_cost, investigate plans that don't do what you wa

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-15 Thread marcin mank
On Thu, Jan 14, 2010 at 8:17 PM, Carlo Stonebanks wrote: > . 48 GB RAM > 2) Which Windows OS would you recommend? (currently 2008 x64 Server) There is not a 64-bit windows build now - You would be limited to shared_buffers at about a gigabyte. Choose Linux Greetings Marcin Mańk -- Sent via pgs

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-19 Thread Carlo Stonebanks
Hi Scott, Sorry for the very late reply on this post, but I'd like to follow up. The reason that I took so long to reply was due to this suggestion: < My first thought was, does he mean against the entire DB? That would take a week! But, since it was recommended, I decided to see what woul

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-19 Thread Scott Marlowe
On Tue, Jan 19, 2010 at 2:09 PM, Carlo Stonebanks wrote: > Hi Scott, > > Sorry for the very late reply on this post, but I'd like to follow up. The > reason that I took so long to reply was due to this suggestion: > > < overrunning the max_fsm_pages settings or the max_fsm_relations. >>> > > My fi

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-20 Thread Carlo Stonebanks
yeah, the values are at the end. Sounds like your vacuum settings are too non-aggresive. Generally this is the vacuum cost delay being too high. Of course, I have to ask: what's the down side? Yes! You can run vacuum verbose against the regular old postgres database (or just create one for

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-20 Thread Kevin Grittner
"Carlo Stonebanks" wrote: >> yeah, the values are at the end. Sounds like your vacuum >> settings are too non-aggresive. Generally this is the vacuum >> cost delay being too high. > > Of course, I have to ask: what's the down side? If you make it too aggressive, it could impact throughput

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-20 Thread Robert Haas
On Wed, Jan 20, 2010 at 3:03 PM, Carlo Stonebanks wrote: >> Yes!  You can run vacuum verbose against the regular old postgres >> database (or just create one for testing with nothing in it) and >> you'll still get the fsm usage numbers from that!  So, no need to run >> it against the big db.  Howe

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-21 Thread Alvaro Herrera
Scott Marlowe escribió: > On Thu, Jan 14, 2010 at 12:17 PM, Carlo Stonebanks > wrote: > > 4) Is this the right PG version for our needs? > > 8.3 is very stable. Update to the latest. 8.4 seems good, but I've > had, and still am having, problems with it crashing in production. > Not often, mayb

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-21 Thread Scott Marlowe
On Thu, Jan 21, 2010 at 8:51 AM, Alvaro Herrera wrote: > Scott Marlowe escribió: >> On Thu, Jan 14, 2010 at 12:17 PM, Carlo Stonebanks >> wrote: > >> > 4) Is this the right PG version for our needs? >> >> 8.3 is very stable.  Update to the latest.  8.4 seems good, but I've >> had, and still am ha

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-21 Thread Alvaro Herrera
Scott Marlowe escribió: > On Thu, Jan 21, 2010 at 8:51 AM, Alvaro Herrera > wrote: > > Scott Marlowe escribió: > >> On Thu, Jan 14, 2010 at 12:17 PM, Carlo Stonebanks > >> wrote: > > > >> > 4) Is this the right PG version for our needs? > >> > >> 8.3 is very stable.  Update to the latest.  8.4 se

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-21 Thread Scott Marlowe
On Thu, Jan 21, 2010 at 9:44 AM, Alvaro Herrera wrote: > Scott Marlowe escribió: >> On Thu, Jan 21, 2010 at 8:51 AM, Alvaro Herrera >> wrote: >> > Scott Marlowe escribió: >> >> On Thu, Jan 14, 2010 at 12:17 PM, Carlo Stonebanks >> >> wrote: >> > >> >> > 4) Is this the right PG version for our ne

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-21 Thread Devrim GÜNDÜZ
On Thu, 2010-01-21 at 13:44 -0300, Alvaro Herrera wrote: > I think Devrim publishes debuginfo packages which you need to install > separately. Right. -- Devrim GÜNDÜZ, RHCE Command Prompt - http://www.CommandPrompt.com devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-22 Thread Carlo Stonebanks
Hi Greg, As a follow up to this suggestion: I don't see effective_cache_size listed there. If that's at the default, I wouldn't be surprised that you're seeing sequential scans instead of indexed ones far too often. I found an article written by you http://www.westnet.com/~gsmith/content/p

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-26 Thread Greg Smith
Carlo Stonebanks wrote:

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-14 Thread Dave Crooke
I'll bite 1. In general, RAID-10 is the only suitable RAID configuration for a database. The decision making comes in how many drives, and splitting stuff up into LUNs (like putting pg_xlog on its own LUN). 2. None of the above - you're asking the wrong question really. PostgreSQL is open

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-14 Thread Craig Ringer
On 15/01/2010 6:35 AM, Dave Crooke wrote: I'll bite 1. In general, RAID-10 is the only suitable RAID configuration for a database. The decision making comes in how many drives, and splitting stuff up into LUNs (like putting pg_xlog on its own LUN). The biggest problem with Postgres on

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Tony McC
On Thu, 14 Jan 2010 16:35:53 -0600 Dave Crooke wrote: > For any given database engine, regardless of the marketing and support > stance, there is only one true "primary" enterprise OS platform that > most big mission critical sites use, and is the best supported and > most stable platform for tha

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Richard Broersma
On Fri, Jan 15, 2010 at 8:10 AM, Tony McC wrote: >> most stable platform for that RDBMS. For Oracle, that's HP-UX (but 10 >> years ago, it was Solaris). For PostgreSQL, it's Linux. > > I am interested in this response and am wondering if this is just > Dave's opinion or some sort of official Post

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Robert Haas
On Fri, Jan 15, 2010 at 11:10 AM, Tony McC wrote: > what is it about Linux specifically (as > contrasted with other Unix-like OSes, especially Open Source ones) that > makes it particularly suitable for running PostgreSQL? Nothing that I know of. ...Robert -- Sent via pgsql-performance mailing

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Tom Lane
Richard Broersma writes: > On Fri, Jan 15, 2010 at 8:10 AM, Tony McC wrote: >>> most stable platform for that RDBMS. For Oracle, that's HP-UX (but 10 >>> years ago, it was Solaris). For PostgreSQL, it's Linux. >> I am interested in this response and am wondering if this is just >> Dave's opinion

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Greg Smith
Tom Lane wrote: Given the Linux kernel hackers' apparent disinterest in fixing their OOM kill policy or making write barriers work well (or at all, with LVM), I think arguing that Linux is the best database platform requires a certain amount of suspension of disbelief. Don't forget the gener

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Scott Marlowe
On Fri, Jan 15, 2010 at 11:28 AM, Greg Smith wrote: > Tom Lane wrote: >> >> Given the Linux kernel hackers' apparent disinterest in fixing their >> OOM kill policy or making write barriers work well (or at all, with >> LVM), I think arguing that Linux is the best database platform requires >> a ce

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Kevin Grittner
Scott Marlowe wrote: > I will say that XFS seems to be a very stable file system, and we > use it for some of our databases with no problems at all. But > most of our stuff sits on ext3 because it's stable and reliable > and fast enough. Our PostgreSQL data directories are all on xfs, with ev

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Dave Crooke
Just opinion, and like Greg, I was suggesting it along the lines of "it's the platform most production PG instances run on, so you're following a well trodden path, and any issue you encounter is likely to have been found and fixed by someone else". It's not about the general suitability of the OS

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Dave Crooke
This is the second time I've heard that "PG shared buffer on Windows doesn't matter" ... I'd like to understand the reasoning behind that claim, and why it differs from other DB servers. though that's much less important for Pg than for most other things, as > Pg uses a one-process-per-connec

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Greg Smith
Dave Crooke wrote: My reasoning goes like this: a. there is a significant performance benefit to using a large proportion of memory as in-process DB server cache instead of OS level block / filesystem cache b. the only way to do so on modern hardware (i.e. >>4GB) is with a 64-bit binary c. t

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Tom Lane
Dave Crooke writes: > This is the second time I've heard that "PG shared buffer on Windows doesn't > matter" ... I'd like to understand the reasoning behind that claim, and why > it differs from other DB servers. AFAIK we don't really understand why, but the experimental evidence is that increasi