Re: [HACKERS] 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-26 Thread Merlin Moncure
On Wed, Apr 25, 2012 at 9:28 AM, Merlin Moncure mmonc...@gmail.com wrote: That patch actually has more than one optimization in it, I think, but the basic idea is that if we could figure out a way to set HEAP_XMIN_COMMITTED when loading data into a table created or truncated within the same

Re: [HACKERS] 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-25 Thread Merlin Moncure
On Tue, Apr 24, 2012 at 3:40 PM, Robert Haas robertmh...@gmail.com wrote: On Sat, Apr 21, 2012 at 7:08 PM, Greg Stark st...@mit.edu wrote: The earlier consensus was to move all the hint bits to a dedicated area and exclude them from the checksum. I think double-write buffers seem to have

Re: [HACKERS] 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-25 Thread Robert Haas
On Tue, Apr 24, 2012 at 8:52 PM, Greg Stark st...@mit.edu wrote: On Tue, Apr 24, 2012 at 9:40 PM, Robert Haas robertmh...@gmail.com wrote:  For three things, index pages have hint-type changes that are not single-bit changes. ? Just how big are these? Part of the reason hint bit updates are

Re: [HACKERS] 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-24 Thread Robert Haas
On Sat, Apr 21, 2012 at 5:40 PM, Jeff Davis pg...@j-davis.com wrote: * In addition to detecting random garbage, we also need to be able to detect zeroing of pages. Right now, a zero page is not considered corrupt, so that's a problem. We'll need to WAL table extension operations, and we'll

Re: [HACKERS] 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-24 Thread Robert Haas
On Sat, Apr 21, 2012 at 7:08 PM, Greg Stark st...@mit.edu wrote: The earlier consensus was to move all the hint bits to a dedicated area and exclude them from the checksum. I think double-write buffers seem to have become more fashionable but a summary that doesn't describe the former is

Re: [HACKERS] 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-24 Thread Greg Stark
On Tue, Apr 24, 2012 at 9:40 PM, Robert Haas robertmh...@gmail.com wrote:  For three things, index pages have hint-type changes that are not single-bit changes. ? Just how big are these? Part of the reason hint bit updates are safe is because one bit definitely absolutely has to be entirely in

Re: [HACKERS] 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-24 Thread Josh Berkus
On 4/21/12 2:40 PM, Jeff Davis wrote: If we do use WAL for hint bit updates, that has an impact on Hot Standby, because HS can't write WAL. So, it would seem that HS could not set hint bits. If we're WAL-logging hint bits, then the standby would be receiving them, so it doesn't *need* to write

[HACKERS] 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-21 Thread Jeff Davis
A lot of discussion took place regarding corruption detection, and I am attempting to summarize it in a useful way. Please excuse the lack of references; I'm hoping to agree on the basic problem space and the nature of the solutions offered, and then turn it into a wiki where we can get into the

Re: [HACKERS] 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-21 Thread Greg Stark
On Sat, Apr 21, 2012 at 10:40 PM, Jeff Davis pg...@j-davis.com wrote: * In addition to detecting random garbage, we also need to be able to detect zeroing of pages. Right now, a zero page is not considered corrupt, so that's a problem. We'll need to WAL table extension operations, and we'll