Re: [HACKERS] Logical to physical page mapping

2012-11-05 Thread Bruce Momjian
On Mon, Oct 29, 2012 at 07:05:39AM -0400, Robert Haas wrote: > Yet another idea we've tossed around is to make only vacuum records > include FPWs, and have the more common heap insert/update/delete > operations include enough information that they can still be applied > correctly even if the page h

Re: [HACKERS] Logical to physical page mapping

2012-10-30 Thread Markus Wanner
On 10/29/2012 12:05 PM, Robert Haas wrote: > OK, I'll stop babbling now... Not perceived as babbling here. Thanks for that nice round-up of options and ideas around the torn page problem. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chang

Re: [HACKERS] Logical to physical page mapping

2012-10-29 Thread Robert Haas
On Sat, Oct 27, 2012 at 1:01 AM, Jan Wieck wrote: > The reason why we need full_page_writes is that we need to guard against > torn pages or partial writes. So what if smgr would manage a mapping between > logical page numbers and their physical location in the relation? This sounds a lot like ht

Re: [HACKERS] Logical to physical page mapping

2012-10-28 Thread Jan Wieck
On 10/28/2012 10:50 PM, Peter Geoghegan wrote: On 28 October 2012 22:35, Jan Wieck wrote: The amount of WAL generated with full_page_writes=on is quite substantial. For pgbench for example the ratio 20:1. Meaning with full_page_writes you write 20x the amount you do without. Sure, but as alwa

Re: [HACKERS] Logical to physical page mapping

2012-10-28 Thread Peter Geoghegan
On 28 October 2012 22:35, Jan Wieck wrote: > The amount of WAL generated with full_page_writes=on is quite substantial. > For pgbench for example the ratio 20:1. Meaning with full_page_writes you > write 20x the amount you do without. Sure, but as always, pgbench pessimises everything by having w

Re: [HACKERS] Logical to physical page mapping

2012-10-28 Thread Jan Wieck
On 10/27/2012 2:41 PM, Heikki Linnakangas wrote: And it's not at all clear to me that it would perform better than full_page_writes. You're writing and flushing out roughly the same amount of data AFAICS. I think this assumption is wrong. full_page_writes=on means we write the full page conten

Re: [HACKERS] Logical to physical page mapping

2012-10-28 Thread Greg Stark
On Sat, Oct 27, 2012 at 8:41 PM, Heikki Linnakangas wrote: > If the pointers are stored as simple 4-byte integers, you probably could > assume that they're atomic, and won't be torn. Actually I think any fixed-size data structure would be fine. We're worried about storage on disk here, not in mem

Re: [HACKERS] Logical to physical page mapping

2012-10-27 Thread Gavin Flower
On 28/10/12 07:41, Heikki Linnakangas wrote: On 27.10.2012 16:43, Tom Lane wrote: Jan Wieck writes: The reason why we need full_page_writes is that we need to guard against torn pages or partial writes. So what if smgr would manage a mapping between logical page numbers and their physical loca

Re: [HACKERS] Logical to physical page mapping

2012-10-27 Thread Tom Lane
Claudio Freire writes: > On Sat, Oct 27, 2012 at 3:41 PM, Heikki Linnakangas > wrote: >>> I think you're just moving the atomic-write problem from the data pages >>> to wherever you keep these pointers. >> If the pointers are stored as simple 4-byte integers, you probably could >> assume that th

Re: [HACKERS] Logical to physical page mapping

2012-10-27 Thread Claudio Freire
On Sat, Oct 27, 2012 at 3:41 PM, Heikki Linnakangas wrote: > >> I think you're just moving the atomic-write problem from the data pages >> to wherever you keep these pointers. > > > If the pointers are stored as simple 4-byte integers, you probably could > assume that they're atomic, and won't be

Re: [HACKERS] Logical to physical page mapping

2012-10-27 Thread Heikki Linnakangas
On 27.10.2012 16:43, Tom Lane wrote: Jan Wieck writes: The reason why we need full_page_writes is that we need to guard against torn pages or partial writes. So what if smgr would manage a mapping between logical page numbers and their physical location in the relation? At the moment where w

Re: [HACKERS] Logical to physical page mapping

2012-10-27 Thread Tom Lane
Jan Wieck writes: > The reason why we need full_page_writes is that we need to guard against > torn pages or partial writes. So what if smgr would manage a mapping > between logical page numbers and their physical location in the relation? > At the moment where we today require a full page writ

[HACKERS] Logical to physical page mapping

2012-10-26 Thread Jan Wieck
I just had this thought a few minutes ago, discussed it briefly with RhodiumToad on #postgresql and wanted to put it out here for discussion. Feel free to rip it apart. It probably is a bit "al-dente" at this point and needs more cooking. The reason why we need full_page_writes is that we need

[HACKERS] Logical to physical page mapping

2012-10-26 Thread Jan Wieck
I just had this thought a few minutes ago, discussed it briefly with RhodiumToad on #postgresql and wanted to put it out here for discussion. Feel free to rip it apart. It probably is a bit "al-dente" at this point and needs more cooking. The reason why we need full_page_writes is that we need