Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-21 Thread David Steele
On 3/7/17 9:42 PM, Pavan Deolasee wrote: Fair point. I'm not going to "persist" with the idea too long. It seemed like a good, low-risk feature to me which can benefit certain use cases quite reasonably. It's not uncommon to create indexes (or reindex existing indexes to remove index bloats) on

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-08 Thread Stephen Frost
* Andres Freund (and...@anarazel.de) wrote: > On 2017-03-07 21:38:40 -0500, Robert Haas wrote: > > > I wonder however, if careful snapshot managment couldn't solve this as > > > well. I have *not* thought a lot about this, but afaics we can easily > > > prevent all-visible from being set in cases

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-08 Thread Stephen Frost
* Pavan Deolasee (pavan.deola...@gmail.com) wrote: > On Wed, Mar 8, 2017 at 7:33 AM, Robert Haas wrote: > > > On Tue, Mar 7, 2017 at 4:26 PM, Stephen Frost wrote: > > > Right, that's what I thought he was getting at and my general thinking > > > was

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Andres Freund
On 2017-03-07 21:38:40 -0500, Robert Haas wrote: > > I wonder however, if careful snapshot managment couldn't solve this as > > well. I have *not* thought a lot about this, but afaics we can easily > > prevent all-visible from being set in cases it'd bother us by having an > > "appropriate" xmin

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Pavan Deolasee
On Wed, Mar 8, 2017 at 8:08 AM, Robert Haas wrote: > On Tue, Mar 7, 2017 at 9:12 PM, Andres Freund wrote: > > > > > I wonder however, if careful snapshot managment couldn't solve this as > > well. I have *not* thought a lot about this, but afaics we

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Pavan Deolasee
On Wed, Mar 8, 2017 at 7:33 AM, Robert Haas wrote: > On Tue, Mar 7, 2017 at 4:26 PM, Stephen Frost wrote: > > Right, that's what I thought he was getting at and my general thinking > > was that we would need a way to discover if a CIC is ongoing on the

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Robert Haas
On Tue, Mar 7, 2017 at 9:12 PM, Andres Freund wrote: > On 2017-03-07 21:03:53 -0500, Robert Haas wrote: >> On Tue, Mar 7, 2017 at 4:26 PM, Stephen Frost wrote: >> > Right, that's what I thought he was getting at and my general thinking >> > was that we

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Andres Freund
On 2017-03-07 21:03:53 -0500, Robert Haas wrote: > On Tue, Mar 7, 2017 at 4:26 PM, Stephen Frost wrote: > > Right, that's what I thought he was getting at and my general thinking > > was that we would need a way to discover if a CIC is ongoing on the > > relation and therefore

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Robert Haas
On Tue, Mar 7, 2017 at 4:26 PM, Stephen Frost wrote: > Right, that's what I thought he was getting at and my general thinking > was that we would need a way to discover if a CIC is ongoing on the > relation and therefore heap_page_prune(), or anything else, would know > that

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Mar 3, 2017 at 6:06 PM, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > >> On 2017-02-28 19:12:03 +0530, Pavan Deolasee wrote: > >> > Since VM bits are only set during VACUUM which conflicts

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Robert Haas
On Fri, Mar 3, 2017 at 6:06 PM, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: >> On 2017-02-28 19:12:03 +0530, Pavan Deolasee wrote: >> > Since VM bits are only set during VACUUM which conflicts with CIC on the >> > relation lock, I don't see any risk of

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-03 Thread Andres Freund
On 2017-03-03 15:12:04 -0800, Peter Geoghegan wrote: > On Tue, Feb 28, 2017 at 5:42 AM, Pavan Deolasee > wrote: > > During the second heap scan of CREATE INDEX CONCURRENTLY, we're only > > interested in the tuples which were inserted after the first scan was > > started.

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-03 Thread Peter Geoghegan
On Tue, Feb 28, 2017 at 5:42 AM, Pavan Deolasee wrote: > During the second heap scan of CREATE INDEX CONCURRENTLY, we're only > interested in the tuples which were inserted after the first scan was > started. All such tuples can only exists in pages which have their VM

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-03 Thread Stephen Frost
Andres, * Andres Freund (and...@anarazel.de) wrote: > On 2017-02-28 19:12:03 +0530, Pavan Deolasee wrote: > > Since VM bits are only set during VACUUM which conflicts with CIC on the > > relation lock, I don't see any risk of incorrectly skipping pages that the > > second scan should have

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-03 Thread Peter Geoghegan
On Fri, Mar 3, 2017 at 2:54 PM, Andres Freund wrote: > On 2017-02-28 19:12:03 +0530, Pavan Deolasee wrote: >> Since VM bits are only set during VACUUM which conflicts with CIC on the >> relation lock, I don't see any risk of incorrectly skipping pages that the >> second scan

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-03 Thread Andres Freund
Hi, On 2017-02-28 19:12:03 +0530, Pavan Deolasee wrote: > Since VM bits are only set during VACUUM which conflicts with CIC on the > relation lock, I don't see any risk of incorrectly skipping pages that the > second scan should have scanned. I think that's true currently, but it'd also prevent

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-01 Thread Masahiko Sawada
On Tue, Feb 28, 2017 at 10:42 PM, Pavan Deolasee wrote: > Hello All, > > During the second heap scan of CREATE INDEX CONCURRENTLY, we're only > interested in the tuples which were inserted after the first scan was > started. All such tuples can only exists in pages which

[HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-02-28 Thread Pavan Deolasee
Hello All, During the second heap scan of CREATE INDEX CONCURRENTLY, we're only interested in the tuples which were inserted after the first scan was started. All such tuples can only exists in pages which have their VM bit unset. So I propose the attached patch which consults VM during second