Re: [HACKERS] WIP checksums patch

2012-11-14 Thread Merlin Moncure
On Tue, Oct 9, 2012 at 1:51 AM, Amit Kapila wrote: > On Monday, October 01, 2012 11:11 PM Jeff Davis wrote: >> On Mon, 2012-10-01 at 18:14 +0100, Simon Riggs wrote: >> > You are missing large parts of the previous thread, giving various >> > opinions on what the UI should look like for enabling ch

Re: [HACKERS] WIP checksums patch

2012-11-14 Thread Bruce Momjian
On Mon, Nov 12, 2012 at 12:39:17AM -0500, Greg Smith wrote: > On 11/9/12 6:14 PM, Jeff Davis wrote: > >On Mon, 2012-11-05 at 12:19 -0500, Robert Haas wrote: > >>Yeah. I definitely think that we could shed an enormous amount of > >>complexity by deciding that this is, for now, an option that can on

Re: [HACKERS] WIP checksums patch

2012-11-11 Thread Greg Smith
On 11/9/12 6:14 PM, Jeff Davis wrote: On Mon, 2012-11-05 at 12:19 -0500, Robert Haas wrote: Yeah. I definitely think that we could shed an enormous amount of complexity by deciding that this is, for now, an option that can only be selected at initdb time. That would remove approximately 85% of

Re: [HACKERS] WIP checksums patch

2012-11-09 Thread Jeff Davis
On Mon, 2012-10-01 at 10:22 -0700, Josh Berkus wrote: > > I think that's OK, because it's still protected by the WAL CRC, and > > there's no expectation that the checksum is correct in shared buffers, > > and the correct checksum should be set on the next checkpoint. Just an > > observation. > > W

Re: [HACKERS] WIP checksums patch

2012-11-09 Thread Jeff Davis
On Fri, 2012-11-09 at 10:18 -0500, Robert Haas wrote: > Sure, I agree. I don't think it should stay that way forever, but > removing the burden of dealing with this issue from the initial commit > would likely allow that commit to happen this release cycle, perhaps > even in the next CommitFest.

Re: [HACKERS] WIP checksums patch

2012-11-09 Thread Jeff Davis
On Mon, 2012-11-05 at 12:19 -0500, Robert Haas wrote: > Yeah. I definitely think that we could shed an enormous amount of > complexity by deciding that this is, for now, an option that can only > be selected at initdb time. That would remove approximately 85% of > everything I've ever disliked ab

Re: [HACKERS] WIP checksums patch

2012-11-09 Thread Robert Haas
On Thu, Nov 8, 2012 at 9:17 PM, Christopher Browne wrote: > I see one thing to be concerned about, there... > > I imagine it would not be a totally happy thing if the only way to switch it > on/off was to use Slony or Londiste to replicate into a database with the > opposite setting. (e.g. - This

Re: [HACKERS] WIP checksums patch

2012-11-08 Thread Christopher Browne
On Mon, Nov 5, 2012 at 12:19 PM, Robert Haas wrote: > On Mon, Oct 29, 2012 at 4:31 PM, Jim Nasby wrote: > > For whatever it's worth... we (and presumably others) still use londiste > (or > > Slony) as our upgrade path, so we could tolerate a cluster-wide setting. > > We'd just set it when buildi

Re: [HACKERS] WIP checksums patch

2012-11-05 Thread Robert Haas
On Mon, Oct 29, 2012 at 4:31 PM, Jim Nasby wrote: > For whatever it's worth... we (and presumably others) still use londiste (or > Slony) as our upgrade path, so we could tolerate a cluster-wide setting. > We'd just set it when building new clusters via londiste and forget about > it. > > So I'd r

Re: [HACKERS] WIP checksums patch

2012-10-30 Thread jesper
> On 10/1/12 12:22 PM, Josh Berkus wrote: >>> >Perhaps we don't allow this to be turned per page, but rather per >>> >cluster, and per-cluster would require the entire cluster to be >>> >rewritten. >> We dicussed this last year, and options which require a total rewrite of >> the database in order

Re: [HACKERS] WIP checksums patch

2012-10-29 Thread Jim Nasby
On 10/1/12 12:22 PM, Josh Berkus wrote: >Perhaps we don't allow this to be turned per page, but rather per >cluster, and per-cluster would require the entire cluster to be >rewritten. We dicussed this last year, and options which require a total rewrite of the database in order to turn on the op

Re: [HACKERS] WIP checksums patch

2012-10-08 Thread Amit Kapila
On Monday, October 01, 2012 11:11 PM Jeff Davis wrote: > On Mon, 2012-10-01 at 18:14 +0100, Simon Riggs wrote: > > You are missing large parts of the previous thread, giving various > > opinions on what the UI should look like for enabling checksums. > > I read through all of the discussion that I

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Jeff Davis
On Mon, 2012-10-01 at 18:14 +0100, Simon Riggs wrote: > You are missing large parts of the previous thread, giving various > opinions on what the UI should look like for enabling checksums. I read through all of the discussion that I could find. There was quite a lot, so perhaps I have forgotten p

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Simon Riggs
On 1 October 2012 18:04, Jeff Davis wrote: > On Mon, 2012-10-01 at 12:35 -0400, Bruce Momjian wrote: >> The heap/index files are copied unmodified from the old cluster, so >> there are no checksums on the pages. > > That's fine though, the patch still reads the old format the same way, > and the p

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Josh Berkus
> I think that's OK, because it's still protected by the WAL CRC, and > there's no expectation that the checksum is correct in shared buffers, > and the correct checksum should be set on the next checkpoint. Just an > observation. We'd need to document that emphatically. Otherwise folks running

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Bruce Momjian
On Mon, Oct 1, 2012 at 10:04:09AM -0700, Jeff Davis wrote: > On Mon, 2012-10-01 at 12:35 -0400, Bruce Momjian wrote: > > The heap/index files are copied unmodified from the old cluster, so > > there are no checksums on the pages. > > That's fine though, the patch still reads the old format the sa

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Jeff Davis
On Fri, 2012-09-14 at 17:58 -0700, Jeff Davis wrote: > This is just a rebased version of the patch by Simon here: > > http://archives.postgresql.org/message-id/CA > +u5nmkw_gbs6qq_y8-rjgl1v7mvw2hwbhartb8lojhnpfx...@mail.gmail.com Another thing I noticed about the design of this patch: It looks l

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Jeff Davis
On Mon, 2012-10-01 at 12:35 -0400, Bruce Momjian wrote: > The heap/index files are copied unmodified from the old cluster, so > there are no checksums on the pages. That's fine though, the patch still reads the old format the same way, and the pages are treated as though they have no checksum. How

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Bruce Momjian
On Mon, Oct 1, 2012 at 09:25:43AM -0700, Jeff Davis wrote: > On Mon, 2012-10-01 at 10:43 -0400, Bruce Momjian wrote: > > > The default is off for backwards compatibility and > > > to allow upgrade. The recommended setting is on though > > > this should not be enabled until upgrade is success

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Jeff Davis
On Mon, 2012-10-01 at 10:43 -0400, Bruce Momjian wrote: > > The default is off for backwards compatibility and > > to allow upgrade. The recommended setting is on though > > this should not be enabled until upgrade is successfully complete > > with full set of new backups. > > > > I don't

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Bruce Momjian
On Sun, Sep 30, 2012 at 07:09:20PM -0700, Jeff Davis wrote: > On Fri, 2012-09-14 at 17:58 -0700, Jeff Davis wrote: > > This is just a rebased version of the patch by Simon here: > > I just noticed the following note in the docs for this patch: > > The default is off for backwards compatibility

Re: [HACKERS] WIP checksums patch

2012-09-30 Thread Jeff Davis
On Fri, 2012-09-14 at 17:58 -0700, Jeff Davis wrote: > * we might want to make it slightly easier for external utilities, like > for backup/replication, to verify the pages Ideally, PageVerificationInfoOK should be available to external utilities, so that someone might script a background job to v

Re: [HACKERS] WIP checksums patch

2012-09-30 Thread Jeff Davis
On Fri, 2012-09-14 at 17:58 -0700, Jeff Davis wrote: > This is just a rebased version of the patch by Simon here: I just noticed the following note in the docs for this patch: The default is off for backwards compatibility and to allow upgrade. The recommended setting is on though this shou

[HACKERS] WIP checksums patch

2012-09-14 Thread Jeff Davis
This is just a rebased version of the patch by Simon here: http://archives.postgresql.org/message-id/CA +u5nmkw_gbs6qq_y8-rjgl1v7mvw2hwbhartb8lojhnpfx...@mail.gmail.com There are other things I'd like to do, like: * include page# in checksum, and perhaps relfilenode or object OID (but those two