Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-02 Thread Robert Haas
On Wed, Dec 2, 2009 at 3:48 PM, Tom Lane wrote: > Greg Stark writes: >> This whole discussion is based on assumptions which do not match my >> recollection of the old discussion. I would suggest people go back and >> read the emails but it's clear at least some people have so it seems >> people g

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-02 Thread Tom Lane
Greg Stark writes: > This whole discussion is based on assumptions which do not match my > recollection of the old discussion. I would suggest people go back and > read the emails but it's clear at least some people have so it seems > people get different things out of those old emails. My recolle

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-02 Thread Greg Stark
On Wed, Dec 2, 2009 at 6:34 PM, Robert Haas wrote: >> Also, your logic seems to presume that no backports are possible to the old >> server. > > The problem on the table at the moment is that the proposed CRC > feature will expand every page by a uniform amount - so in this case a > fixed-space-pe

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-02 Thread Robert Haas
On Wed, Dec 2, 2009 at 2:27 PM, Greg Smith wrote: > Robert Haas wrote: > > On Wed, Dec 2, 2009 at 1:56 PM, Greg Smith wrote: > > > There's no reason the associated catalog support had to ship with the old > version.  You can always modify the catalog after initdb, but before running > the pre-upg

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-02 Thread Greg Smith
Robert Haas wrote: On Wed, Dec 2, 2009 at 1:56 PM, Greg Smith wrote: There's no reason the associated catalog support had to ship with the old version. You can always modify the catalog after initdb, but before running the pre-upgrade utility. pg_migrator might make that change for you.

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-02 Thread Robert Haas
On Wed, Dec 2, 2009 at 1:56 PM, Greg Smith wrote: > Robert Haas wrote: >>> Some additional catalog support was suggested to mark what the >>> pre-upgrade >>> utility had processed.   I'm sure I could find the messages about again >>> if I >>> had to. >> And that's a perfectly sensible solution, ex

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-02 Thread Greg Smith
Robert Haas wrote: Some additional catalog support was suggested to mark what the pre-upgrade utility had processed. I'm sure I could find the messages about again if I had to. And that's a perfectly sensible solution, except that adding a catalog column to 8.4 at this point would force i

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-02 Thread Robert Haas
On Wed, Dec 2, 2009 at 1:08 PM, Greg Smith wrote: > Robert Haas wrote: >> >> The problem I'm referring to is that there is no guarantee that you >> would be able predict how much space to reserve.  In a case like CRCs, >> it may be as simple as "4 bytes".  But what if, say, we switch to a >> diffe

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-02 Thread Greg Smith
Robert Haas wrote: The problem I'm referring to is that there is no guarantee that you would be able predict how much space to reserve. In a case like CRCs, it may be as simple as "4 bytes". But what if, say, we switch to a different compression algorithm for inline toast? Upthread, you made a

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-02 Thread Robert Haas
On Wed, Dec 2, 2009 at 11:08 AM, Simon Riggs wrote: > On Wed, 2009-12-02 at 10:48 -0500, Robert Haas wrote: >> Well, that's sort of a circular argument.  If you're going to reserve >> space with a pre-upgrade utility, you're going to need to put the >> pre-upgrade utility into the version you want

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-02 Thread Simon Riggs
On Wed, 2009-12-02 at 10:48 -0500, Robert Haas wrote: > Well, that's sort of a circular argument. If you're going to reserve > space with a pre-upgrade utility, you're going to need to put the > pre-upgrade utility into the version you want to upgrade FROM. If we > wanted to be able to use a pre-

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-02 Thread Robert Haas
On Tue, Dec 1, 2009 at 11:45 PM, Bruce Momjian wrote: > Robert Haas wrote: >> >> The key issue, as I think Heikki identified at the time, is to figure >> >> out how you're eventually going to get rid of the old pages. ?He >> >> proposed running a pre-upgrade utility on each page to reserve the >>

Re: [HACKERS] Page-level version upgrade

2009-12-02 Thread Dimitri Fontaine
Greg Stark writes: > On Wed, Dec 2, 2009 at 11:26 AM, Dimitri Fontaine > wrote: >> We already have had demand for read only tables (some on-disk format >> optimisation would then be possible). What about having page level >> read-only restriction, thus allowing the newer server version to operate

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-02 Thread Bruce Momjian
David Fetter wrote: > > Right. There were two basic approaches to handling a patch that > > would expand when upgraded to the new version --- either allow the > > system to write the old format, or have a pre-upgrade script that > > moved tuples so there was guaranteed enough free space in every p

Re: [HACKERS] Page-level version upgrade

2009-12-02 Thread Greg Stark
On Wed, Dec 2, 2009 at 11:26 AM, Dimitri Fontaine wrote: > We already have had demand for read only tables (some on-disk format > optimisation would then be possible). What about having page level > read-only restriction, thus allowing the newer server version to operate > in read-only mode on the

Re: [HACKERS] Page-level version upgrade

2009-12-02 Thread Dimitri Fontaine
Hi, As we're talking about crazy ideas... Bruce Momjian writes: > Well, yea, the idea would be that the 8.5 server would either convert > the page to the new format on read (assuming there is enough free space, > perhaps requiring a pre-upgrade script), or have the server write the > page in the

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-01 Thread David Fetter
On Tue, Dec 01, 2009 at 10:34:11PM -0500, Bruce Momjian wrote: > Robert Haas wrote: > > The key issue, as I think Heikki identified at the time, is to > > figure out how you're eventually going to get rid of the old > > pages. He proposed running a pre-upgrade utility on each page to > > reserve t

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-01 Thread Bruce Momjian
Robert Haas wrote: > >> The key issue, as I think Heikki identified at the time, is to figure > >> out how you're eventually going to get rid of the old pages. ?He > >> proposed running a pre-upgrade utility on each page to reserve the > >> right amount of free space. > >> > >> http://archives.post

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-01 Thread Robert Haas
On Tue, Dec 1, 2009 at 10:45 PM, Greg Smith wrote: > Robert Haas wrote: >> >> If the pre-upgrade utility is something >> that has to be run while the database is off-line, then it defeats the >> point of an in-place upgrade.  If it can be run while the database is >> up, I fear it will need to be

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-01 Thread Robert Haas
On Tue, Dec 1, 2009 at 10:34 PM, Bruce Momjian wrote: > Robert Haas wrote: >> > Well, there were quite a number of open issues relating to page >> > conversion: >> > >> > ? ? ? ?o ?Do we write the old version or just convert on read? >> > ? ? ? ?o ?How do we write pages that get larger on conversi

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-01 Thread Greg Smith
Robert Haas wrote: If the pre-upgrade utility is something that has to be run while the database is off-line, then it defeats the point of an in-place upgrade. If it can be run while the database is up, I fear it will need to be deeply integrated into the server. And since we can't know the req

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-01 Thread Bruce Momjian
Robert Haas wrote: > > Well, there were quite a number of open issues relating to page > > conversion: > > > > ? ? ? ?o ?Do we write the old version or just convert on read? > > ? ? ? ?o ?How do we write pages that get larger on conversion to the > > ? ? ? ? ? new format? > > > > As I rember the pa

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-01 Thread Robert Haas
On Tue, Dec 1, 2009 at 9:31 PM, Bruce Momjian wrote: > Robert Haas wrote: >> On Tue, Dec 1, 2009 at 5:15 PM, Greg Stark wrote: >> > On Tue, Dec 1, 2009 at 9:58 PM, decibel wrote: >> >> What happened to the work that was being done to allow a page to be >> >> upgraded >> >> on the fly when it wa

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-01 Thread Bruce Momjian
Robert Haas wrote: > On Tue, Dec 1, 2009 at 5:15 PM, Greg Stark wrote: > > On Tue, Dec 1, 2009 at 9:58 PM, decibel wrote: > >> What happened to the work that was being done to allow a page to be > >> upgraded > >> on the fly when it was read in from disk? > > > > There were no page level changes

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-01 Thread Robert Haas
On Tue, Dec 1, 2009 at 5:15 PM, Greg Stark wrote: > On Tue, Dec 1, 2009 at 9:58 PM, decibel wrote: >> What happened to the work that was being done to allow a page to be upgraded >> on the fly when it was read in from disk? > > There were no page level changes between 8.3 and 8.4. That's true, b

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-01 Thread Bruce Momjian
Greg Stark wrote: > On Tue, Dec 1, 2009 at 9:58 PM, decibel wrote: > > What happened to the work that was being done to allow a page to be upgraded > > on the fly when it was read in from disk? > > There were no page level changes between 8.3 and 8.4. Yea, we have the idea of how to do it (in ca

Re: [HACKERS] Page-level version upgrade (was: Block-level CRC checks)

2009-12-01 Thread Greg Stark
On Tue, Dec 1, 2009 at 9:58 PM, decibel wrote: > What happened to the work that was being done to allow a page to be upgraded > on the fly when it was read in from disk? There were no page level changes between 8.3 and 8.4. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr