Re: New vacuum option to do only freezing

2019-06-24 Thread Michael Paquier
On Sun, Jun 23, 2019 at 10:29:25PM +0900, Michael Paquier wrote: > So, are there any objections with this patch? Or do people think that > it's too late for v12 and that it is better to wait until v13 opens > for business? Committed, and open item closed. -- Michael signature.asc Description:

Re: New vacuum option to do only freezing

2019-06-23 Thread Michael Paquier
On Thu, Jun 20, 2019 at 03:50:32PM +0900, Michael Paquier wrote: > And I finish with the attached. Thoughts? So, are there any objections with this patch? Or do people think that it's too late for v12 and that it is better to wait until v13 opens for business? -- Michael signature.asc

Re: New vacuum option to do only freezing

2019-06-20 Thread Michael Paquier
On Wed, Jun 19, 2019 at 12:51:41PM -0700, Peter Geoghegan wrote: > On Tue, Jun 18, 2019 at 10:39 PM Michael Paquier wrote: >> +INSERT INTO no_index_cleanup(i, t) VALUES(1, repeat('1234567890',3)); >> Do we really need a string as long as that? > > Specifying EXTERNAL storage might make

Re: New vacuum option to do only freezing

2019-06-19 Thread Peter Geoghegan
On Tue, Jun 18, 2019 at 10:39 PM Michael Paquier wrote: > +INSERT INTO no_index_cleanup(i, t) VALUES(1, repeat('1234567890',3)); > Do we really need a string as long as that? Specifying EXTERNAL storage might make things easier. I have used PLAIN storage to test the 1/3 of a page restriction

Re: New vacuum option to do only freezing

2019-05-20 Thread Robert Haas
On Fri, May 17, 2019 at 4:55 PM Andres Freund wrote: > Robert, this indeed looks near trivial. What do you think? Hmm, yeah, I guess that'd be OK. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: New vacuum option to do only freezing

2019-05-17 Thread Tom Lane
Andres Freund writes: > Robert, this indeed looks near trivial. What do you think? > On 2019-05-09 14:14:20 +0900, Masahiko Sawada wrote: >> +"toast.vacuum_index_clenaup", Not trivial enough to not have typos, apparently. regards, tom lane

Re: New vacuum option to do only freezing

2019-05-17 Thread Andres Freund
Hi, On 2019-05-09 14:14:20 +0900, Masahiko Sawada wrote: > I agree that this item is neither critical and bug. But this is an > (my) oversight and is a small patch and I think there is no specific > reason why we don't dare to include this in 12. So if this patch could > get reviewed enough I

Re: New vacuum option to do only freezing

2019-05-08 Thread Masahiko Sawada
On Thu, May 2, 2019 at 1:39 AM Robert Haas wrote: > > On Wed, May 1, 2019 at 12:14 PM Andres Freund wrote: > > On 2019-04-06 16:13:53 +0900, Michael Paquier wrote: > > > On Sat, Apr 06, 2019 at 11:31:31AM +0900, Masahiko Sawada wrote: > > > > Yes, but Fujii-san pointed out that this option

Re: New vacuum option to do only freezing

2019-05-07 Thread Masahiko Sawada
On Fri, May 3, 2019 at 12:09 AM Robert Haas wrote: > > On Tue, Apr 16, 2019 at 4:00 PM Tom Lane wrote: > > I wrote: > > > I'm thinking that we really need to upgrade vacuum's reporting totals > > > so that it accounts in some more-honest way for pre-existing dead > > > line pointers. The patch

Re: New vacuum option to do only freezing

2019-05-02 Thread Alvaro Herrera
Pushed. I added one comment to explain xmin_frozen also, which otherwise seemed a bit mysterious. I did not backpatch, though, so 9.6-11 are a bit different, but I'm not sure it's a good idea at this point, though it should be pretty innocuous. -- Álvaro Herrera

Re: New vacuum option to do only freezing

2019-05-02 Thread Alvaro Herrera
On 2019-May-01, Andres Freund wrote: > Alvaro, could we perhaps clean this up a bit? This is pretty confusing > looking. I think this probably could just be changed to > > boolxmin_frozen = false; > > xid = HeapTupleHeaderGetXmin(tuple); > > if (xid ==

Re: New vacuum option to do only freezing

2019-05-02 Thread Robert Haas
On Thu, May 2, 2019 at 10:28 AM Andres Freund wrote: > It'd be good if somebody could make a pass over the safety mechanisms in > heap_prepare_freeze_tuple(). I added them at some point, after a data > corrupting bug related to freezing, but they really were more intended > as a secondary layer

Re: New vacuum option to do only freezing

2019-05-02 Thread Andres Freund
On 2019-05-02 11:09:10 -0400, Robert Haas wrote: > If I understand correctly, your first proposal amounts to redefining > tups_vacuumed to count #2 rather than #1, and your second proposal > amounts to making tups_vacuumed count #1 + #2 rather than #1. I > suggest a third option: have two

Re: New vacuum option to do only freezing

2019-05-02 Thread Robert Haas
On Tue, Apr 16, 2019 at 4:00 PM Tom Lane wrote: > I wrote: > > I'm thinking that we really need to upgrade vacuum's reporting totals > > so that it accounts in some more-honest way for pre-existing dead > > line pointers. The patch as it stands has made the reporting even more > > confusing,

Re: New vacuum option to do only freezing

2019-05-02 Thread Andres Freund
Hi, On 2019-05-02 10:20:25 -0400, Robert Haas wrote: > On Tue, Apr 16, 2019 at 12:44 PM Tom Lane wrote: > > So after thinking about this a bit more ... > > 2. Act as if the tuple were still live, just as would've happened if the > > state didn't change till just after we looked instead of just

Re: New vacuum option to do only freezing

2019-05-02 Thread Robert Haas
On Tue, Apr 16, 2019 at 12:44 PM Tom Lane wrote: > So after thinking about this a bit more ... > > ISTM that what we have here is a race condition (ie, tuple changed state > since heap_page_prune), and that ideally we want the code to resolve it > as if no race had happened. That is, either of

Re: New vacuum option to do only freezing

2019-05-01 Thread Robert Haas
On Wed, May 1, 2019 at 12:14 PM Andres Freund wrote: > On 2019-04-06 16:13:53 +0900, Michael Paquier wrote: > > On Sat, Apr 06, 2019 at 11:31:31AM +0900, Masahiko Sawada wrote: > > > Yes, but Fujii-san pointed out that this option doesn't support toast > > > tables and I think there is not

Re: New vacuum option to do only freezing

2019-05-01 Thread Andres Freund
Hi, On 2019-04-06 16:13:53 +0900, Michael Paquier wrote: > On Sat, Apr 06, 2019 at 11:31:31AM +0900, Masahiko Sawada wrote: > > Yes, but Fujii-san pointed out that this option doesn't support toast > > tables and I think there is not specific reason why not supporting > > them. So it might be

Re: New vacuum option to do only freezing

2019-05-01 Thread Andres Freund
Hi, This thread is referenced an open item, and we ought to make some progress on it. On a more cosmetic note: On 2019-04-16 09:10:19 -0700, Andres Freund wrote: > On 2019-04-16 12:01:36 -0400, Tom Lane wrote: > > (BTW, I don't understand why that code will throw "found xmin %u from > > before

Re: New vacuum option to do only freezing

2019-04-26 Thread Masahiko Sawada
On Fri, Apr 26, 2019 at 12:11 AM Robert Haas wrote: > > On Tue, Apr 23, 2019 at 7:09 AM Masahiko Sawada wrote: > > For the second issue, I've changed lazy vacuum so that it reports both > > the number of kilobytes we freed and the number of kilobytes can be > > freed after index cleanup. > > I

Re: New vacuum option to do only freezing

2019-04-25 Thread Robert Haas
On Tue, Apr 23, 2019 at 7:09 AM Masahiko Sawada wrote: > For the second issue, I've changed lazy vacuum so that it reports both > the number of kilobytes we freed and the number of kilobytes can be > freed after index cleanup. I am not very convinced that this reporting is in any useful to

Re: New vacuum option to do only freezing

2019-04-23 Thread Masahiko Sawada
On Thu, Apr 18, 2019 at 3:12 PM Masahiko Sawada wrote: > > On Thu, Apr 18, 2019 at 4:20 AM Peter Geoghegan wrote: > > > > On Wed, Apr 17, 2019 at 10:46 AM Tom Lane wrote: > > > Yeah, if we wanted to expose these complications more directly, we > > > could think about adding or changing the main

Re: New vacuum option to do only freezing

2019-04-18 Thread Masahiko Sawada
On Thu, Apr 18, 2019 at 4:20 AM Peter Geoghegan wrote: > > On Wed, Apr 17, 2019 at 10:46 AM Tom Lane wrote: > > Yeah, if we wanted to expose these complications more directly, we > > could think about adding or changing the main counters. I was wondering > > about whether we should have it

Re: New vacuum option to do only freezing

2019-04-17 Thread Peter Geoghegan
On Wed, Apr 17, 2019 at 10:46 AM Tom Lane wrote: > Yeah, if we wanted to expose these complications more directly, we > could think about adding or changing the main counters. I was wondering > about whether we should have it report "x bytes and y line pointers > freed", rather than counting

Re: New vacuum option to do only freezing

2019-04-17 Thread Tom Lane
Masahiko Sawada writes: > On Wed, Apr 17, 2019 at 5:00 AM Tom Lane wrote: >>> I'm thinking that we really need to upgrade vacuum's reporting totals >>> so that it accounts in some more-honest way for pre-existing dead >>> line pointers. The patch as it stands has made the reporting even more

Re: New vacuum option to do only freezing

2019-04-17 Thread Masahiko Sawada
On Wed, Apr 17, 2019 at 5:00 AM Tom Lane wrote: > > I wrote: > > I'm thinking that we really need to upgrade vacuum's reporting totals > > so that it accounts in some more-honest way for pre-existing dead > > line pointers. The patch as it stands has made the reporting even more > > confusing,

Re: New vacuum option to do only freezing

2019-04-16 Thread Tom Lane
I wrote: > I'm thinking that we really need to upgrade vacuum's reporting totals > so that it accounts in some more-honest way for pre-existing dead > line pointers. The patch as it stands has made the reporting even more > confusing, rather than less so. Here's a couple of ideas about that: 1.

Re: New vacuum option to do only freezing

2019-04-16 Thread Tom Lane
So after thinking about this a bit more ... ISTM that what we have here is a race condition (ie, tuple changed state since heap_page_prune), and that ideally we want the code to resolve it as if no race had happened. That is, either of these behaviors would be acceptable: 1. Delete the tuple,

Re: New vacuum option to do only freezing

2019-04-16 Thread Andres Freund
Hi, On 2019-04-16 12:01:36 -0400, Tom Lane wrote: > (BTW, I don't understand why that code will throw "found xmin %u from > before relfrozenxid %u" if HeapTupleHeaderXminFrozen is true? Shouldn't > the whole if-branch at lines 6113ff be skipped if xmin_frozen?) I *think* that just looks odd,

Re: New vacuum option to do only freezing

2019-04-16 Thread Tom Lane
Robert Haas writes: > On Mon, Apr 15, 2019 at 9:07 PM Tom Lane wrote: >> If we're failing to remove it, and it's below the desired freeze >> horizon, then we'd darn well better freeze it instead, no? > I don't know that that's safe. IIRC, the freeze code doesn't cope > nicely with being given

Re: New vacuum option to do only freezing

2019-04-16 Thread Andres Freund
Hi, On 2019-04-16 11:38:01 -0400, Tom Lane wrote: > Alvaro Herrera writes: > > On 2019-Apr-16, Robert Haas wrote: > >> On Mon, Apr 15, 2019 at 9:07 PM Tom Lane wrote: > >>> If we're failing to remove it, and it's below the desired freeze > >>> horizon, then we'd darn well better freeze it

Re: New vacuum option to do only freezing

2019-04-16 Thread Andres Freund
Hi, On 2019-04-16 10:54:34 -0400, Alvaro Herrera wrote: > On 2019-Apr-16, Robert Haas wrote: > > On Mon, Apr 15, 2019 at 9:07 PM Tom Lane wrote: > > > > I'm not sure that's correct. If you do that, it'll end up in the > > > > non-tupgone case, which might try to freeze a tuple that should've >

Re: New vacuum option to do only freezing

2019-04-16 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Apr-16, Robert Haas wrote: >> On Mon, Apr 15, 2019 at 9:07 PM Tom Lane wrote: >>> If we're failing to remove it, and it's below the desired freeze >>> horizon, then we'd darn well better freeze it instead, no? >> I don't know that that's safe. IIRC, the freeze

Re: New vacuum option to do only freezing

2019-04-16 Thread Masahiko Sawada
On Tue, Apr 16, 2019 at 11:26 PM Robert Haas wrote: > > On Mon, Apr 15, 2019 at 9:07 PM Tom Lane wrote: > > > I'm not sure that's correct. If you do that, it'll end up in the > > > non-tupgone case, which might try to freeze a tuple that should've > > > been removed. Or am I confused? > > > >

Re: New vacuum option to do only freezing

2019-04-16 Thread Alvaro Herrera
On 2019-Apr-16, Robert Haas wrote: > On Mon, Apr 15, 2019 at 9:07 PM Tom Lane wrote: > > > I'm not sure that's correct. If you do that, it'll end up in the > > > non-tupgone case, which might try to freeze a tuple that should've > > > been removed. Or am I confused? > > > > If we're failing to

Re: New vacuum option to do only freezing

2019-04-16 Thread Robert Haas
On Mon, Apr 15, 2019 at 9:07 PM Tom Lane wrote: > > I'm not sure that's correct. If you do that, it'll end up in the > > non-tupgone case, which might try to freeze a tuple that should've > > been removed. Or am I confused? > > If we're failing to remove it, and it's below the desired freeze >

Re: New vacuum option to do only freezing

2019-04-16 Thread Masahiko Sawada
On Tue, Apr 16, 2019 at 4:47 AM Tom Lane wrote: > > Robert Haas writes: > > On Mon, Apr 15, 2019 at 1:13 PM Tom Lane wrote: > >> I have a very strong feeling that this patch was not fully baked. > > > I think you're right, but I don't understand the comment in the > > preceding paragraph. How

Re: New vacuum option to do only freezing

2019-04-15 Thread Michael Paquier
On Mon, Apr 15, 2019 at 09:07:16PM -0400, Tom Lane wrote: > If we're failing to remove it, and it's below the desired freeze > horizon, then we'd darn well better freeze it instead, no? > > Since we know that the tuple only just became dead, I suspect > that the case would be unreachable in

Re: New vacuum option to do only freezing

2019-04-15 Thread Tom Lane
Robert Haas writes: > On Mon, Apr 15, 2019 at 3:47 PM Tom Lane wrote: >> No. I'm thinking there should be exactly one test of index_cleanup >> in this logic, and what it would be is along the lines of ... > I'm not sure that's correct. If you do that, it'll end up in the > non-tupgone case,

Re: New vacuum option to do only freezing

2019-04-15 Thread Robert Haas
On Mon, Apr 15, 2019 at 3:47 PM Tom Lane wrote: > No. I'm thinking there should be exactly one test of index_cleanup > in this logic, and what it would be is along the lines of > > if (HeapTupleIsHotUpdated() || > HeapTupleIsHeapOnly() || > +

Re: New vacuum option to do only freezing

2019-04-15 Thread Tom Lane
Robert Haas writes: > On Mon, Apr 15, 2019 at 1:13 PM Tom Lane wrote: >> I have a very strong feeling that this patch was not fully baked. > I think you're right, but I don't understand the comment in the > preceding paragraph. How does this problem prevent tupgone from > getting set? My

Re: New vacuum option to do only freezing

2019-04-15 Thread Robert Haas
On Mon, Apr 15, 2019 at 1:13 PM Tom Lane wrote: > Masahiko Sawada writes: > >> Ugh, I think the assertion is right but the above condition is > >> completely wrong. We should increment nleft_dead_tuples when index > >> cleanup is *not* enabled. > > > Here is a draft patch to fix this issue. > >

Re: New vacuum option to do only freezing

2019-04-15 Thread Tom Lane
Masahiko Sawada writes: >> Ugh, I think the assertion is right but the above condition is >> completely wrong. We should increment nleft_dead_tuples when index >> cleanup is *not* enabled. > Here is a draft patch to fix this issue. So the real issue here, I fear, is that we've got no consistent

Re: New vacuum option to do only freezing

2019-04-15 Thread Masahiko Sawada
On Mon, Apr 15, 2019 at 9:28 AM Masahiko Sawada wrote: > > On Mon, Apr 15, 2019 at 12:47 AM Tom Lane wrote: > > > > Robert Haas writes: > > > On Wed, Apr 3, 2019 at 10:32 PM Masahiko Sawada > > > wrote: > > >> Attached the updated version patch. > > > > > Committed with a little bit of

Re: New vacuum option to do only freezing

2019-04-14 Thread Masahiko Sawada
On Mon, Apr 15, 2019 at 12:47 AM Tom Lane wrote: > > Robert Haas writes: > > On Wed, Apr 3, 2019 at 10:32 PM Masahiko Sawada > > wrote: > >> Attached the updated version patch. > > > Committed with a little bit of documentation tweaking. > > topminnow just failed an assertion from this patch:

Re: New vacuum option to do only freezing

2019-04-14 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 3, 2019 at 10:32 PM Masahiko Sawada wrote: >> Attached the updated version patch. > Committed with a little bit of documentation tweaking. topminnow just failed an assertion from this patch:

Re: New vacuum option to do only freezing

2019-04-06 Thread Michael Paquier
On Sat, Apr 06, 2019 at 11:31:31AM +0900, Masahiko Sawada wrote: > Yes, but Fujii-san pointed out that this option doesn't support toast > tables and I think there is not specific reason why not supporting > them. So it might be good to add toast.vacuum_index_cleanup. Attached > patch. Being able

Re: New vacuum option to do only freezing

2019-04-05 Thread Masahiko Sawada
On Sat, Apr 6, 2019 at 10:23 AM Andres Freund wrote: > > Hi, > > On 2019-04-04 15:05:55 -0400, Robert Haas wrote: > > On Wed, Apr 3, 2019 at 10:32 PM Masahiko Sawada > > wrote: > > > Attached the updated version patch. > > > > Committed with a little bit of documentation tweaking. > > I've

Re: New vacuum option to do only freezing

2019-04-05 Thread Andres Freund
Hi, On 2019-04-04 15:05:55 -0400, Robert Haas wrote: > On Wed, Apr 3, 2019 at 10:32 PM Masahiko Sawada wrote: > > Attached the updated version patch. > > Committed with a little bit of documentation tweaking. I've closed the commitfest entry. I hope that's accurate? Greetings, Andres Freund

Re: New vacuum option to do only freezing

2019-04-05 Thread Masahiko Sawada
On Fri, Apr 5, 2019 at 10:17 AM Masahiko Sawada wrote: > > On Fri, Apr 5, 2019 at 4:06 AM Robert Haas wrote: > > > > On Wed, Apr 3, 2019 at 10:32 PM Masahiko Sawada > > wrote: > > > Attached the updated version patch. > > > > Committed with a little bit of documentation tweaking. > > > > Thank

Re: New vacuum option to do only freezing

2019-04-05 Thread Robert Haas
On Thu, Apr 4, 2019 at 5:37 PM Bossart, Nathan wrote: > Thanks! I noticed a very small typo in the new documentation. Committed, thanks. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: New vacuum option to do only freezing

2019-04-04 Thread Masahiko Sawada
On Fri, Apr 5, 2019 at 4:06 AM Robert Haas wrote: > > On Wed, Apr 3, 2019 at 10:32 PM Masahiko Sawada wrote: > > Attached the updated version patch. > > Committed with a little bit of documentation tweaking. > Thank you for committing them! Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND

Re: New vacuum option to do only freezing

2019-04-04 Thread Bossart, Nathan
On 4/4/19, 12:06 PM, "Robert Haas" wrote: > Committed with a little bit of documentation tweaking. Thanks! I noticed a very small typo in the new documentation. diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index fdd8151220..c652f8b0bc 100644 ---

Re: New vacuum option to do only freezing

2019-04-04 Thread Robert Haas
On Wed, Apr 3, 2019 at 10:32 PM Masahiko Sawada wrote: > Attached the updated version patch. Committed with a little bit of documentation tweaking. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: New vacuum option to do only freezing

2019-04-03 Thread Masahiko Sawada
On Thu, Apr 4, 2019 at 9:18 AM Kyotaro HORIGUCHI wrote: > > Hello. > > At Wed, 3 Apr 2019 11:55:00 -0400, Robert Haas wrote > in > > On Wed, Apr 3, 2019 at 1:32 AM Masahiko Sawada > > wrote: > > > Attached the updated version patches including the > > > DISABLE_PAGE_SKIPPING part (0003). > >

Re: New vacuum option to do only freezing

2019-04-03 Thread Kyotaro HORIGUCHI
Hello. At Wed, 3 Apr 2019 11:55:00 -0400, Robert Haas wrote in > On Wed, Apr 3, 2019 at 1:32 AM Masahiko Sawada wrote: > > Attached the updated version patches including the > > DISABLE_PAGE_SKIPPING part (0003). > > I am confused about nleft_dead_tuples. It looks like it gets > incremented

Re: New vacuum option to do only freezing

2019-04-03 Thread Robert Haas
On Wed, Apr 3, 2019 at 1:32 AM Masahiko Sawada wrote: > Attached the updated version patches including the > DISABLE_PAGE_SKIPPING part (0003). I am confused about nleft_dead_tuples. It looks like it gets incremented whenever we set tupgone = true, regardless of whether we are doing index

Re: New vacuum option to do only freezing

2019-04-02 Thread Masahiko Sawada
On Wed, Apr 3, 2019 at 1:17 PM Kyotaro HORIGUCHI wrote: > > At Wed, 3 Apr 2019 12:10:03 +0900, Masahiko Sawada > wrote in > > > And in the following part: > > > > > > + /* Set index cleanup option based on reloptions */ > > > + if (params->index_cleanup == VACUUM_OPTION_DEFAULT) >

Re: New vacuum option to do only freezing

2019-04-02 Thread Kyotaro HORIGUCHI
At Wed, 3 Apr 2019 12:10:03 +0900, Masahiko Sawada wrote in > > And in the following part: > > > > + /* Set index cleanup option based on reloptions */ > > + if (params->index_cleanup == VACUUM_OPTION_DEFAULT) > > + { > > + if (onerel->rd_options == NULL || > > +

Re: New vacuum option to do only freezing

2019-04-02 Thread Masahiko Sawada
On Wed, Apr 3, 2019 at 10:56 AM Kyotaro HORIGUCHI wrote: > > Hello. > > At Mon, 1 Apr 2019 14:26:15 +0900, Masahiko Sawada > wrote in > > On Sat, Mar 30, 2019 at 5:04 AM Robert Haas wrote: > > > > > > On Fri, Mar 29, 2019 at 12:27 PM Masahiko Sawada > > > wrote: > > > > Yeah, but since

Re: New vacuum option to do only freezing

2019-04-02 Thread Kyotaro HORIGUCHI
Hello. At Mon, 1 Apr 2019 14:26:15 +0900, Masahiko Sawada wrote in > On Sat, Mar 30, 2019 at 5:04 AM Robert Haas wrote: > > > > On Fri, Mar 29, 2019 at 12:27 PM Masahiko Sawada > > wrote: > > > Yeah, but since multiple relations might be specified in VACUUM > > > command we need to process

Re: New vacuum option to do only freezing

2019-04-02 Thread Darafei Praliaskouski
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested I have read this patch. I like the concept and would like it to get

Re: New vacuum option to do only freezing

2019-03-31 Thread Masahiko Sawada
On Sat, Mar 30, 2019 at 5:04 AM Robert Haas wrote: > > On Fri, Mar 29, 2019 at 12:27 PM Masahiko Sawada > wrote: > > Yeah, but since multiple relations might be specified in VACUUM > > command we need to process index_cleanup option after opened each > > relations. Maybe we need to process all

Re: New vacuum option to do only freezing

2019-03-29 Thread Robert Haas
On Fri, Mar 29, 2019 at 12:27 PM Masahiko Sawada wrote: > Yeah, but since multiple relations might be specified in VACUUM > command we need to process index_cleanup option after opened each > relations. Maybe we need to process all options except for > INDEX_CLEANUP in ExecVacuum() and pass

Re: New vacuum option to do only freezing

2019-03-29 Thread Masahiko Sawada
On Fri, Mar 29, 2019 at 10:46 PM Robert Haas wrote: > > On Fri, Mar 29, 2019 at 2:16 AM Masahiko Sawada wrote: > > Attached updated patches. These patches are applied on top of 0001 > > patch on parallel vacuum thread[1]. > > +bool index_cleanup = true; /* by default */ > > I think we

Re: New vacuum option to do only freezing

2019-03-29 Thread Robert Haas
On Fri, Mar 29, 2019 at 2:16 AM Masahiko Sawada wrote: > Attached updated patches. These patches are applied on top of 0001 > patch on parallel vacuum thread[1]. +bool index_cleanup = true; /* by default */ I think we should instead initialize index_cleanup to the reloption value, if there

Re: New vacuum option to do only freezing

2019-03-29 Thread Masahiko Sawada
On Fri, Mar 29, 2019 at 4:39 AM Robert Haas wrote: > > On Thu, Mar 28, 2019 at 2:00 AM Masahiko Sawada wrote: > > The patch adds the basic functionality to disable index cleanup but > > one possible argument could be whether we should always disable it > > when anti-wraparound vacuum. As

Re: New vacuum option to do only freezing

2019-03-28 Thread Robert Haas
On Thu, Mar 28, 2019 at 2:00 AM Masahiko Sawada wrote: > The patch adds the basic functionality to disable index cleanup but > one possible argument could be whether we should always disable it > when anti-wraparound vacuum. As discussed on another thread[1] > anti-wraparound vacuum still could

Re: New vacuum option to do only freezing

2019-03-28 Thread Masahiko Sawada
On Wed, Mar 27, 2019 at 12:12 AM Masahiko Sawada wrote: > > On Sat, Mar 23, 2019 at 3:25 AM Robert Haas wrote: > > > > On Fri, Mar 8, 2019 at 12:14 AM Masahiko Sawada > > wrote: > > > IIUC we've discussed the field-and-value style vacuum option. I > > > suggested that since we have already the

Re: New vacuum option to do only freezing

2019-03-26 Thread Masahiko Sawada
On Sat, Mar 23, 2019 at 3:25 AM Robert Haas wrote: > > On Fri, Mar 8, 2019 at 12:14 AM Masahiko Sawada wrote: > > IIUC we've discussed the field-and-value style vacuum option. I > > suggested that since we have already the disable_page_skipping option > > the disable_page_skipping option would

Re: New vacuum option to do only freezing

2019-03-22 Thread Robert Haas
On Fri, Mar 8, 2019 at 12:14 AM Masahiko Sawada wrote: > IIUC we've discussed the field-and-value style vacuum option. I > suggested that since we have already the disable_page_skipping option > the disable_page_skipping option would be more natural style and > consistent. I think "VACUUM

Re: New vacuum option to do only freezing

2019-03-07 Thread Masahiko Sawada
On Fri, Mar 8, 2019 at 12:04 AM Robert Haas wrote: > > On Thu, Mar 7, 2019 at 1:03 AM Masahiko Sawada wrote: > > Hmm, the patch already has new reloption vacuum_index_cleanup and > > default value is true but you meant I should change its name to > > index_cleanup? > > No, I mean that you should

Re: New vacuum option to do only freezing

2019-03-07 Thread Robert Haas
On Thu, Mar 7, 2019 at 1:03 AM Masahiko Sawada wrote: > Hmm, the patch already has new reloption vacuum_index_cleanup and > default value is true but you meant I should change its name to > index_cleanup? No, I mean that you should make it so that someone writes VACUUM (INDEX_CLEANUP false)

Re: New vacuum option to do only freezing

2019-03-06 Thread Masahiko Sawada
On Thu, Mar 7, 2019 at 3:55 AM Robert Haas wrote: > > On Tue, Mar 5, 2019 at 11:29 PM Masahiko Sawada wrote: > > Attached updated patch incorporated all of comments. Also I've added > > new reloption vacuum_index_cleanup as per discussion on the "reloption > > to prevent VACUUM from truncating

Re: New vacuum option to do only freezing

2019-03-06 Thread Robert Haas
On Tue, Mar 5, 2019 at 11:29 PM Masahiko Sawada wrote: > Attached updated patch incorporated all of comments. Also I've added > new reloption vacuum_index_cleanup as per discussion on the "reloption > to prevent VACUUM from truncating empty pages at the end of relation" > thread. Autovacuums also

Re: New vacuum option to do only freezing

2019-03-05 Thread Masahiko Sawada
On Tue, Mar 5, 2019 at 8:01 PM Kyotaro HORIGUCHI wrote: > > Hello, I have some other comments. > Thank you for the comment! On Tue, Mar 5, 2019 at 8:01 PM Kyotaro HORIGUCHI wrote: > > > + nleft;/* item pointers we left */ > > The name seems to be something other,

Re: New vacuum option to do only freezing

2019-03-05 Thread Bossart, Nathan
On 3/5/19, 1:22 AM, "Masahiko Sawada" wrote: > On Tue, Mar 5, 2019 at 8:27 AM Bossart, Nathan wrote: >> + VACUUM removes dead tuples and prunes HOT-updated >> + tuples chain for live tuples on table. If the table has any dead tuple >> + it removes them from both table and indexes

Re: New vacuum option to do only freezing

2019-03-05 Thread Kyotaro HORIGUCHI
Hello, I have some other comments. At Mon, 4 Mar 2019 23:27:10 +, "Bossart, Nathan" wrote in <48410154-e6c5-4c07-8122-8d04e3bcd...@amazon.com> > On 2/28/19, 12:13 AM, "Masahiko Sawada" wrote: > > Attached the updated version patch. I've incorporated all review > > comments I got and have

Re: New vacuum option to do only freezing

2019-03-05 Thread Masahiko Sawada
On Tue, Mar 5, 2019 at 8:27 AM Bossart, Nathan wrote: > > On 2/28/19, 12:13 AM, "Masahiko Sawada" wrote: > > Attached the updated version patch. I've incorporated all review > > comments I got and have changed the number of tuples being reported as > > 'removed tuples'. With this option, tuples

Re: New vacuum option to do only freezing

2019-03-04 Thread Bossart, Nathan
On 2/28/19, 12:13 AM, "Masahiko Sawada" wrote: > Attached the updated version patch. I've incorporated all review > comments I got and have changed the number of tuples being reported as > 'removed tuples'. With this option, tuples completely being removed is > only tuples marked as unused during

Re: New vacuum option to do only freezing

2019-03-01 Thread Robert Haas
On Thu, Feb 28, 2019 at 3:12 AM Masahiko Sawada wrote: > Attached the updated version patch. Regarding the user interface for this patch, please have a look at the concerns I mention in https://www.postgresql.org/message-id/ca+tgmozorx_uuv67rjasx_aswkdzwv8kwfkfrwxyldcqfqj...@mail.gmail.com I

Re: New vacuum option to do only freezing

2019-02-28 Thread Masahiko Sawada
On Thu, Feb 28, 2019 at 2:46 AM Bossart, Nathan wrote: > > On 2/27/19, 2:08 AM, "Masahiko Sawada" wrote: > >> + if (skip_index_vacuum) > >> + appendStringInfo(, ngettext("%.0f tuple is left as > >> dead.\n", > >> +

Re: New vacuum option to do only freezing

2019-02-27 Thread Bossart, Nathan
On 2/27/19, 2:08 AM, "Masahiko Sawada" wrote: >> + if (skip_index_vacuum) >> + appendStringInfo(, ngettext("%.0f tuple is left as >> dead.\n", >> + >>"%.0f tuples are left as dead.\n", >> +

Re: New vacuum option to do only freezing

2019-02-27 Thread Masahiko Sawada
On Wed, Feb 27, 2019 at 10:02 AM Bossart, Nathan wrote: > > Sorry for the delay. I finally got a chance to look through the > latest patches. > > On 2/3/19, 1:48 PM, "Masahiko Sawada" wrote: > > On Fri, Feb 1, 2019 at 11:43 PM Bossart, Nathan wrote: > >> > >> + if

Re: New vacuum option to do only freezing

2019-02-26 Thread Bossart, Nathan
Sorry for the delay. I finally got a chance to look through the latest patches. On 2/3/19, 1:48 PM, "Masahiko Sawada" wrote: > On Fri, Feb 1, 2019 at 11:43 PM Bossart, Nathan wrote: >> >> + if (skip_index_vacuum) >> + ereport(elevel, >> +

Re: New vacuum option to do only freezing

2019-02-04 Thread Bossart, Nathan
On 2/3/19, 1:48 PM, "Masahiko Sawada" wrote: > On Sat, Feb 2, 2019 at 7:00 PM Alvaro Herrera > wrote: >> >> On 2019-Feb-01, Bossart, Nathan wrote: >> >> > IMHO we could document this feature at a slightly higher level without >> > leaving out any really important user-facing behavior. Here's a

Re: New vacuum option to do only freezing

2019-02-03 Thread Masahiko Sawada
On Sat, Feb 2, 2019 at 7:00 PM Alvaro Herrera wrote: > > On 2019-Feb-01, Bossart, Nathan wrote: > > > IMHO we could document this feature at a slightly higher level without > > leaving out any really important user-facing behavior. Here's a quick > > attempt to show what I am thinking: > > > >

Re: New vacuum option to do only freezing

2019-02-02 Thread Alvaro Herrera
On 2019-Feb-01, Bossart, Nathan wrote: > IMHO we could document this feature at a slightly higher level without > leaving out any really important user-facing behavior. Here's a quick > attempt to show what I am thinking: > > With this option, VACUUM skips all index cleanup behavior and

Re: New vacuum option to do only freezing

2019-02-01 Thread Michael Paquier
On Fri, Feb 01, 2019 at 10:43:37PM +, Bossart, Nathan wrote: > I would suggest adding this option to vacuumdb in this patch set as > well. Doing it would be simple enough, for now I have moved it to next CF. -- Michael signature.asc Description: PGP signature

Re: New vacuum option to do only freezing

2019-02-01 Thread Bossart, Nathan
On 1/21/19, 2:23 AM, "Masahiko Sawada" wrote: > Understood and Agreed. I've attached the new version patch > incorporated the review comments. I took a look at the latest version of the patch. + +DISABLE_INDEX_CLEANUP + + + VACUUM removes dead tuples and prunes HOT-updated

Re: New vacuum option to do only freezing

2019-01-21 Thread Masahiko Sawada
On Sat, Jan 19, 2019 at 5:08 AM Robert Haas wrote: > > On Thu, Jan 17, 2019 at 1:57 AM Masahiko Sawada wrote: > > The reason why I processed the tuples that became dead after the first > > heap pass is that I was not sure the reason why we ignore such tuples > > in the second heap pass despite

Re: New vacuum option to do only freezing

2019-01-18 Thread Robert Haas
On Thu, Jan 17, 2019 at 1:57 AM Masahiko Sawada wrote: > The reason why I processed the tuples that became dead after the first > heap pass is that I was not sure the reason why we ignore such tuples > in the second heap pass despite of there already have been the code > doing so which has been

Re: New vacuum option to do only freezing

2019-01-16 Thread Masahiko Sawada
On Thu, Jan 17, 2019 at 1:33 AM Robert Haas wrote: > > On Wed, Jan 16, 2019 at 3:30 AM Masahiko Sawada wrote: > > As the above comment says, it's possible that the state of an > > INSERT_IN_PROGRESS tuple could be changed to 'dead' after > > heap_page_prune(). Since such tuple is not truncated

Re: New vacuum option to do only freezing

2019-01-16 Thread Robert Haas
On Wed, Jan 16, 2019 at 3:30 AM Masahiko Sawada wrote: > As the above comment says, it's possible that the state of an > INSERT_IN_PROGRESS tuple could be changed to 'dead' after > heap_page_prune(). Since such tuple is not truncated at this point we > record it and set it as UNUSED in

Re: New vacuum option to do only freezing

2019-01-16 Thread Masahiko Sawada
On Wed, Jan 16, 2019 at 5:24 AM Robert Haas wrote: > > On Mon, Jan 14, 2019 at 9:24 PM Masahiko Sawada wrote: > > I think that because the tuples that got dead after heap_page_prune() > > looked are recorded but not removed without lazy_vacuum_page() we need > > to process them in

Re: New vacuum option to do only freezing

2019-01-15 Thread Robert Haas
On Mon, Jan 14, 2019 at 9:24 PM Masahiko Sawada wrote: > I think that because the tuples that got dead after heap_page_prune() > looked are recorded but not removed without lazy_vacuum_page() we need > to process them in lazy_vacuum_page(). For decision about whether to > truncate we should not

Re: New vacuum option to do only freezing

2019-01-14 Thread Masahiko Sawada
On Sat, Jan 12, 2019 at 3:27 AM Robert Haas wrote: > > On Fri, Jan 11, 2019 at 1:14 AM Masahiko Sawada wrote: > > Attached the updated patch. Please review it. > > I'm quite confused by this patch. It seems to me that the easiest way > to implement this patch would be to (1) make

Re: New vacuum option to do only freezing

2019-01-11 Thread Robert Haas
On Fri, Jan 11, 2019 at 1:14 AM Masahiko Sawada wrote: > Attached the updated patch. Please review it. I'm quite confused by this patch. It seems to me that the easiest way to implement this patch would be to (1) make lazy_space_alloc take the maxtuples = MaxHeapTuplesPerPage branch when the

Re: New vacuum option to do only freezing

2019-01-10 Thread Masahiko Sawada
On Thu, Jan 10, 2019 at 2:45 PM Masahiko Sawada wrote: > > On Thu, Jan 10, 2019 at 5:23 AM Bossart, Nathan wrote: > > > > Hi, > > > > On 1/8/19, 7:03 PM, "Masahiko Sawada" wrote: > > > Attached the updated version patch incorporated all comments I got. > > > > Thanks for the new patch! > > > >

Re: New vacuum option to do only freezing

2019-01-09 Thread Masahiko Sawada
On Thu, Jan 10, 2019 at 5:23 AM Bossart, Nathan wrote: > > Hi, > > On 1/8/19, 7:03 PM, "Masahiko Sawada" wrote: > > Attached the updated version patch incorporated all comments I got. > > Thanks for the new patch! > > > * Instead of freezing xmax I've changed the behaviour of the new > > option

  1   2   >