Re: [HACKERS] Single pass vacuum - take 1

2011-07-21 Thread Robert Haas
On Tue, Jul 12, 2011 at 4:47 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: Comments ? I was going to spend some time reviewing this, but I see that (1) it has bit-rotted slightly - there is a failing hunk in pg_class.h and (2) some of the comments downthread seem to suggest that you're

Re: [HACKERS] Single pass vacuum - take 1

2011-07-21 Thread Robert Haas
On Thu, Jul 14, 2011 at 12:43 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: How does this interact with the visibility map? If you set the visibility map bit after vacuuming indexes, a subsequent vacuum will not visit the page. The second vacuum will update

Re: [HACKERS] Single pass vacuum - take 1

2011-07-21 Thread Pavan Deolasee
On Thu, Jul 21, 2011 at 11:51 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jul 12, 2011 at 4:47 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: Comments ? I was going to spend some time reviewing this, but I see that (1) it has bit-rotted slightly - there is a failing hunk in

Re: [HACKERS] Single pass vacuum - take 1

2011-07-21 Thread Pavan Deolasee
On Thu, Jul 21, 2011 at 12:17 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jul 14, 2011 at 12:43 PM, Heikki Linnakangas I think you can sidestep that if you check that the page's vacuum LSN = vacuum LSN in pg_class, instead of equality. I don't think that works, because the

Re: [HACKERS] Single pass vacuum - take 1

2011-07-21 Thread Robert Haas
On Thu, Jul 21, 2011 at 12:51 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: The way we force the subsequent vacuum to also look at the pages scanned and pruned by previous failed vacuum, all the pages that have dead-vacuum line pointers would have a new stamp once the vacuum finishes

Re: [HACKERS] Single pass vacuum - take 1

2011-07-21 Thread Pavan Deolasee
On Thu, Jul 21, 2011 at 4:01 PM, Robert Haas robertmh...@gmail.com wrote: I think we are better off doing only equality comparisons and dodging this problem altogether. Fair enough. Just-plain-dead line pointers would have lp_off = 0. Dead-vacuumed line pointers would have lp_off !=

Re: [HACKERS] Single pass vacuum - take 1

2011-07-19 Thread Alvaro Herrera
Excerpts from Pavan Deolasee's message of lun jul 18 14:50:03 -0400 2011: On Mon, Jul 18, 2011 at 3:14 AM, Simon Riggs si...@2ndquadrant.com wrote: I will be happy to remove it again when we have shown there are no bugs getting this wrong is a data loss issue. Though I understand the

Re: [HACKERS] Single pass vacuum - take 1

2011-07-18 Thread Simon Riggs
On Mon, Jul 18, 2011 at 2:20 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Jul 15, 2011 at 7:23 AM, Simon Riggs si...@2ndquadrant.com wrote: My additional requests would be that we can easily tell which blocks have been modified like this, that we have a way to turn this off if we get

Re: [HACKERS] Single pass vacuum - take 1

2011-07-18 Thread Pavan Deolasee
On Mon, Jul 18, 2011 at 3:14 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, Jul 18, 2011 at 2:20 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Jul 15, 2011 at 7:23 AM, Simon Riggs si...@2ndquadrant.com wrote: My additional requests would be that we can easily tell which blocks

Re: [HACKERS] Single pass vacuum - take 1

2011-07-17 Thread Robert Haas
On Fri, Jul 15, 2011 at 7:23 AM, Simon Riggs si...@2ndquadrant.com wrote: My additional requests would be that we can easily tell which blocks have been modified like this, that we have a way to turn this off if we get bugs for next few releases, that we check it all works with Hot Standby

Re: [HACKERS] Single pass vacuum - take 1

2011-07-15 Thread Simon Riggs
On Fri, Jul 15, 2011 at 12:56 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: On Thu, Jul 14, 2011 at 6:22 PM, Simon Riggs si...@2ndquadrant.com wrote: This is a very rare issue, because of all the work yourself and Heikki have put in. I don't think its rare case since vacuum on any

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Pavan Deolasee
On Thu, Jul 14, 2011 at 11:46 AM, Simon Riggs si...@2ndquadrant.com wrote: Hi Pavan, I'd say that seems way too complex for such a small use case and we've only just fixed the bugs from 8.4 vacuum map complexity. The code's looking very robust now and I'm uneasy that such changes are

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Simon Riggs
On Tue, Jul 12, 2011 at 9:47 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: http://archives.postgresql.org/pgsql-hackers/2011-05/msg01119.php PFA a patch which implements the idea with some variation. At the start of the first pass, we remember the current LSN. Every page that needs some

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Heikki Linnakangas
On 14.07.2011 18:57, Pavan Deolasee wrote: On Thu, Jul 14, 2011 at 11:46 AM, Simon Riggssi...@2ndquadrant.com wrote: I'd say that seems way too complex for such a small use case and we've only just fixed the bugs from 8.4 vacuum map complexity. The code's looking very robust now and I'm uneasy

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Pavan Deolasee
On Thu, Jul 14, 2011 at 12:43 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 14.07.2011 18:57, Pavan Deolasee wrote: On Thu, Jul 14, 2011 at 11:46 AM, Simon Riggssi...@2ndquadrant.com wrote: I'd say that seems way too complex for such a small use case and we've

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Alvaro Herrera
Excerpts from Pavan Deolasee's message of jue jul 14 13:54:36 -0400 2011: On Thu, Jul 14, 2011 at 12:43 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Seems odd to store relindxvacxlogid/off as two int32 columns. Store it in one uint64 column, or invent a new datatype

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Simon Riggs
On Thu, Jul 14, 2011 at 4:57 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: Thanks Simon for looking at the patch. Sorry, I didn't notice there was a patch attached. Not reviewed it. I thought we were still just talking. I am not sure if the use case is really narrow. This is a very

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Pavan Deolasee
On Thu, Jul 14, 2011 at 6:22 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, Jul 14, 2011 at 4:57 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: Thanks Simon for looking at the patch. Sorry, I didn't notice there was a patch attached. Not reviewed it. I thought we were still just

[HACKERS] Single pass vacuum - take 1

2011-07-12 Thread Pavan Deolasee
Hi All, As per discussion here http://archives.postgresql.org/pgsql-hackers/2011-05/msg01119.php PFA a patch which implements the idea with some variation. At the start of the first pass, we remember the current LSN. Every page that needs some work is HOT-pruned so that dead tuples are