Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-23 Thread Ben Chobot
On Feb 23, 2010, at 2:23 PM, Kevin Grittner wrote: > > Here are the values from our two largest and busiest systems (where > we found the pg_xlog placement to matter so much). It looks to me > like a more aggressive bgwriter would help, yes? > > cir=> select * from pg_stat_bgwriter ; > -[ RECOR

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-23 Thread Kevin Grittner
Greg Smith wrote: > You can easily quantify if the BGW is aggressive enough. Buffers > leave the cache three ways, and they each show up as separate > counts in pg_stat_bgwriter: buffers_checkpoint, buffers_clean > (the BGW), and buffers_backend (the queries). Cranking it up > further tends t

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-12 Thread Greg Smith
Kevin Grittner wrote: I wonder if it might also pay to make the background writer even more aggressive than we have, so that SELECT-only queries don't spend so much time writing pages. You can easily quantify if the BGW is aggressive enough. Buffers leave the cache three ways, and they each sho

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-12 Thread Alvaro Herrera
Kevin Grittner wrote: > Alvaro Herrera wrote: > > Actually, a transaction that performed no writes doesn't get a > > commit WAL record written, so it shouldn't make any difference at > > all. > > Well, concurrent to the web application is the replication. Would > asynchronous commit of that po

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-12 Thread Kevin Grittner
Alvaro Herrera wrote: > Alvaro Herrera wrote: >> Kevin Grittner wrote: > >> > Anyway, given that these are replication targets, and aren't >> > the "database of origin" for any data of their own, I guess >> > there's no reason not to try asynchronous commit. >> >> Yeah; since the transactions o

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-12 Thread Alvaro Herrera
Alvaro Herrera wrote: > Kevin Grittner wrote: > > Anyway, given that these are replication > > targets, and aren't the "database of origin" for any data of their > > own, I guess there's no reason not to try asynchronous commit. > > Yeah; since the transactions only ever write commit records to

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-12 Thread Alvaro Herrera
Kevin Grittner wrote: > Hannu Krosing wrote: > > > Can it be, that each request does at least 1 write op (update > > session or log something) ? > > Well, the web application connects through a login which only has > SELECT rights; but, in discussing a previous issue we've pretty well > establ

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-12 Thread Kevin Grittner
Hannu Krosing wrote: > Can it be, that each request does at least 1 write op (update > session or log something) ? Well, the web application connects through a login which only has SELECT rights; but, in discussing a previous issue we've pretty well established that it's not unusual for a read

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-11 Thread Kevin Grittner
Aidan Van Dyk wrote: > Alvaro Herrera wrote: >> Hmm, so maybe the performance benefit is not from it being on a >> separate array, but from it being RAID1 instead of RAID5? > > Or the cumulative effects of: > 1) Dedicated spindles/Raid1 > 2) More BBU cache available (I can't imagine the OS pair

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-11 Thread Aidan Van Dyk
* Alvaro Herrera [100211 12:58]: > Hmm, so maybe the performance benefit is not from it being on a separate > array, but from it being RAID1 instead of RAID5? Or the cumulative effects of: 1) Dedicated spindles/Raid1 2) More BBU cache available (I can't imagine the OS pair writing much) 3) not be

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-11 Thread Alvaro Herrera
Kevin Grittner wrote: > Another example of why I shouldn't trust my memory. Per the > hardware tech: > > > OS: /dev/sda is RAID1 - 2 x 2.5" 15k SAS disk > pg_xlog: /dev/sdb is RAID1 - 2 x 2.5" 15k SAS disk > > These reside on a ServeRAID-MR10k controller with 256MB BB cache

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-11 Thread Kevin Grittner
"Kevin Grittner" wrote: > Jesper Krogh wrote: > >> Sorry if it is obvious.. but what filesystem/OS are you using and >> do you have BBU-writeback on the main data catalog also? > File system is xfs noatime,nobarrier for all data; OS is on ext3. > I *think* the pg_xlog mirrored pair is hangin

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-11 Thread Scott Marlowe
On Thu, Feb 11, 2010 at 4:29 AM, Bruce Momjian wrote: > Kevin Grittner wrote: >> Jesper Krogh wrote: >> >> > Sorry if it is obvious.. but what filesystem/OS are you using and >> > do you have BBU-writeback on the main data catalog also? >> >> Sorry for not providing more context. >> >> ATHENA:/va

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-11 Thread Bruce Momjian
Kevin Grittner wrote: > Jesper Krogh wrote: > > > Sorry if it is obvious.. but what filesystem/OS are you using and > > do you have BBU-writeback on the main data catalog also? > > Sorry for not providing more context. > > ATHENA:/var/pgsql/data # uname -a > Linux ATHENA 2.6.16.60-0.39.3-smp

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-09 Thread Kevin Grittner
Amitabh Kant wrote: > Just curious if you have a 16 physical CPU's or 16 cores on 4 > CPU/8 cores over 2 CPU with HT. Four quad core CPUs: vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Xeon(R) CPU X7350 @ 2.93GHz stepping

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-09 Thread Amitabh Kant
On Tue, Feb 9, 2010 at 10:03 PM, Kevin Grittner wrote: > Jesper Krogh wrote: > File system is xfs noatime,nobarrier for all data; OS is on ext3. I > *think* the pg_xlog mirrored pair is hanging off the same > BBU-writeback controller as the big RAID, but I'd have to track down > the hardware te

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-09 Thread Kevin Grittner
Jesper Krogh wrote: > Sorry if it is obvious.. but what filesystem/OS are you using and > do you have BBU-writeback on the main data catalog also? Sorry for not providing more context. ATHENA:/var/pgsql/data # uname -a Linux ATHENA 2.6.16.60-0.39.3-smp #1 SMP Mon May 11 11:46:34 UTC 2009 x86

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-09 Thread Jesper Krogh
> > Frankly, I was quite surprised by this, since some of the benchmarks > people have published on the effects of using a separate RAID for > the WAL files have only shown a one or two percent difference when > using a hardware RAID controller with BBU cache configured for > write-back. Hi Kevin.

[PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-09 Thread Kevin Grittner
Due to an error during an update to the system kernel on a database server backing a web application, we ran for a month (mid-December to mid-January) with the WAL files in the pg_xlog subdirectory on the same 40-spindle array as the data -- only the OS was on a separate mirrored pair of drives. W

Re: [PERFORM] Moving pg_xlog

2005-06-02 Thread Himanshu Baweja
Tom Lane <[EMAIL PROTECTED]> wrote: >It seems highly unlikely that putting more stuff on the xlog disk will>improve performance --- at least not if your bottleneck is update speed.   Tom you are right.. i did some testing... 1) default config--- xlog on disk1 and data on disk2=>     27 mins and 22

Re: [PERFORM] Moving pg_xlog

2005-06-02 Thread Tom Lane
Himanshu Baweja <[EMAIL PROTECTED]> writes: > My database has two scsi disks > my current configuration has pg_xlog on disk1 and data on disk2 > the machine is used for database only > now did some logging and came to a conclusion that my disk2(data disk) is > getting used around 3 tim

Re: [PERFORM] Moving pg_xlog

2005-06-02 Thread Himanshu Baweja
My database has two scsi disks my current configuration has pg_xlog on disk1 and data on disk2 the machine is used for database only now did some logging and came to a conclusion that my disk2(data disk) is getting used around 3 times more than disk1(pg_xlog)   so wht is recommended

Re: [PERFORM] Moving pg_xlog

2005-06-01 Thread Tom Lane
John A Meinel <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Now that I think about it, you were (if I understood your layout >> correctly) proposing to put the xlog on your system's root disk. >> This is probably a bad idea for performance, ... > I certainly agree with what you wrote. But my un

Re: [PERFORM] Moving pg_xlog

2005-06-01 Thread John A Meinel
Tom Lane wrote: ... Now that I think about it, you were (if I understood your layout correctly) proposing to put the xlog on your system's root disk. This is probably a bad idea for performance, because there will always be other traffic to the root disk. What you are really trying to accomplis

Re: [PERFORM] Moving pg_xlog

2005-06-01 Thread Tom Lane
"Keith Worthington" <[EMAIL PROTECTED]> writes: > On Wed, 01 Jun 2005 12:19:40 -0400, Tom Lane wrote >> Put the xlog anywhere BUT there! > Is there a convention that most people follow. It would seem that > anywhere in the installation directory is a bad idea. From what I > have read on

Re: [PERFORM] Moving pg_xlog

2005-06-01 Thread Brad Nicholson
Keith Worthington wrote: On Wed, 01 Jun 2005 12:19:40 -0400, Tom Lane wrote "Keith Worthington" <[EMAIL PROTECTED]> writes: I have been reading about increasing PostgreSQL performance by relocating the pg_xlog to a disk other than the one where the database resides. I have the followi

Re: [PERFORM] Moving pg_xlog

2005-06-01 Thread Keith Worthington
On Wed, 01 Jun 2005 12:19:40 -0400, Tom Lane wrote > "Keith Worthington" <[EMAIL PROTECTED]> writes: > > I have been reading about increasing PostgreSQL performance > > by relocating the pg_xlog to a disk other than the one > > where the database resides. I have the following pg_xlogs > > on my sy

Re: [PERFORM] Moving pg_xlog

2005-06-01 Thread Tom Lane
"Keith Worthington" <[EMAIL PROTECTED]> writes: > I have been reading about increasing PostgreSQL performance by relocating the > pg_xlog to a disk other than the one where the database resides. I have the > following pg_xlogs on my system. > /raid02/databases/pg_xlog > /raid02/rhdb_databases/pg_

[PERFORM] Moving pg_xlog

2005-06-01 Thread Keith Worthington
Hi All, I have been reading about increasing PostgreSQL performance by relocating the pg_xlog to a disk other than the one where the database resides. I have the following pg_xlogs on my system. /raid02/databases/pg_xlog /raid02/rhdb_databases/pg_xlog /raid02/databases-8.0.0/pg_xlog /var/lib/pgs