Re: [HACKERS] tackling full page writes

2011-05-27 Thread Ross J. Reedstrom
On Wed, May 25, 2011 at 01:29:05PM -0400, Robert Haas wrote: > On Wed, May 25, 2011 at 1:06 PM, Greg Smith wrote: > > On 05/24/2011 04:34 PM, Robert Haas wrote: > > > I've been looking into a similar refactoring of the names here, where we > > bundle all of these speed over safety things (fsync,

Re: [HACKERS] tackling full page writes

2011-05-27 Thread Robert Haas
On Thu, May 26, 2011 at 12:38 AM, Fujii Masao wrote: > On Thu, May 26, 2011 at 1:18 PM, Robert Haas wrote: >>> The replay of the WAL record for A doesn't rely on the content of chunk 1 >>> which B modified. So I don't think that "partial page writes" has such >>> a problem. >>> No? >> >> Sorry.  

Re: [HACKERS] tackling full page writes

2011-05-25 Thread Fujii Masao
On Thu, May 26, 2011 at 1:18 PM, Robert Haas wrote: >> The replay of the WAL record for A doesn't rely on the content of chunk 1 >> which B modified. So I don't think that "partial page writes" has such >> a problem. >> No? > > Sorry.  WAL records today DO rely on the prior state of the page.  If

Re: [HACKERS] tackling full page writes

2011-05-25 Thread Robert Haas
On Wed, May 25, 2011 at 10:09 PM, Fujii Masao wrote: > On Wed, May 25, 2011 at 9:34 PM, Robert Haas wrote: >> On Tue, May 24, 2011 at 10:52 PM, Jeff Davis wrote: >>> On Tue, 2011-05-24 at 16:34 -0400, Robert Haas wrote: As I think about it a bit more, we'd need to XLOG not only the par

Re: [HACKERS] tackling full page writes

2011-05-25 Thread Fujii Masao
On Wed, May 25, 2011 at 9:34 PM, Robert Haas wrote: > On Tue, May 24, 2011 at 10:52 PM, Jeff Davis wrote: >> On Tue, 2011-05-24 at 16:34 -0400, Robert Haas wrote: >>> As I think about it a bit more, we'd >>> need to XLOG not only the parts of the page we actually modifying, but >>> any that the W

Re: [HACKERS] tackling full page writes

2011-05-25 Thread Greg Stark
On Tue, May 24, 2011 at 1:34 PM, Robert Haas wrote: > 1. Heikki suggested that instead of doing full page writes, we might > try to write only the parts of the page that have changed.  For > example, if we had 16 bits to play with in the page header (which we > don't), then we could imagine the pa

Re: [HACKERS] tackling full page writes

2011-05-25 Thread Robert Haas
On Wed, May 25, 2011 at 1:06 PM, Greg Smith wrote: > On 05/24/2011 04:34 PM, Robert Haas wrote: >> >> we could name this feature "partial full page writes" and enable it >> either with a setting of full_page_writes=partial > > +1 to overloading the initial name, but only if the parameter is named

Re: [HACKERS] tackling full page writes

2011-05-25 Thread Greg Smith
On 05/24/2011 04:34 PM, Robert Haas wrote: we could name this feature "partial full page writes" and enable it either with a setting of full_page_writes=partial +1 to overloading the initial name, but only if the parameter is named 'maybe', 'sometimes', or 'perhaps'. I've been looking into a

Re: [HACKERS] tackling full page writes

2011-05-25 Thread Robert Haas
On Wed, May 25, 2011 at 10:13 AM, Bruce Momjian wrote: >> > Idempotent does seem like the most promising idea. >> >> I tend to agree with you, but I'm worried it won't actually work out >> to a win.  By the time we augment the records with enough additional >> information we may have eaten up a lo

Re: [HACKERS] tackling full page writes

2011-05-25 Thread Bruce Momjian
Robert Haas wrote: > That idea has the additional advantage of confusing the level of > detail of our WAL logging (minimal vs. archive vs. hot standby) with > the mechanism used to protect against torn pages (full page writes vs. > idempotent WAL records vs. prayer). When they set it wrong and > d

Re: [HACKERS] tackling full page writes

2011-05-25 Thread Simon Riggs
On Tue, May 24, 2011 at 9:34 PM, Robert Haas wrote: > I'm not sure that these ideas are much good, but > for the sake of posterity: Both (1) and (2) seem promising to me. Heikki mentioned (2) would only be effective if we managed to change *all* WAL records. ISTM likely that we would find that

Re: [HACKERS] tackling full page writes

2011-05-25 Thread Robert Haas
On Tue, May 24, 2011 at 11:52 PM, Bruce Momjian wrote: > Robert Haas wrote: >> 2. The other fairly obvious alternative is to adjust our existing WAL >> record types to be idempotent - i.e. to not rely on the existing page >> contents.  For XLOG_HEAP_INSERT, we currently store the target tid and >>

Re: [HACKERS] tackling full page writes

2011-05-25 Thread Robert Haas
On Tue, May 24, 2011 at 10:52 PM, Jeff Davis wrote: > On Tue, 2011-05-24 at 16:34 -0400, Robert Haas wrote: >> As I think about it a bit more, we'd >> need to XLOG not only the parts of the page we actually modifying, but >> any that the WAL record would need to be correct on replay. > > I don't u

Re: [HACKERS] tackling full page writes

2011-05-24 Thread Bruce Momjian
Robert Haas wrote: > 2. The other fairly obvious alternative is to adjust our existing WAL > record types to be idempotent - i.e. to not rely on the existing page > contents. For XLOG_HEAP_INSERT, we currently store the target tid and > the tuple contents. I'm not sure if there's anything else, b

Re: [HACKERS] tackling full page writes

2011-05-24 Thread Jeff Davis
On Tue, 2011-05-24 at 16:34 -0400, Robert Haas wrote: > As I think about it a bit more, we'd > need to XLOG not only the parts of the page we actually modifying, but > any that the WAL record would need to be correct on replay. I don't understand that statement. Can you clarify? Regards,

[HACKERS] tackling full page writes

2011-05-24 Thread Robert Haas
While eating good Indian food and talking about aviation accidents on the last night of PGCon, Greg Stark, Heikki Linnakangas, and I found some time to brainstorm about possible ways to reduce the impact of full_page_writes. I'm not sure that these ideas are much good, but for the sake of posterit