Re: Allow single table VACUUM in transaction block

2022-11-23 Thread Justin Pryzby
On Tue, Nov 22, 2022 at 05:16:59PM +, Simon Riggs wrote: > Justin, if you wanted to take up the patch from here, I would be more > than happy. You have the knowledge and insight to make this work > right. I have no particular use for this, so I wouldn't be a good person to finish or shepherd

Re: Allow single table VACUUM in transaction block

2022-11-22 Thread Simon Riggs
On Tue, 22 Nov 2022 at 16:43, Justin Pryzby wrote: > > On Mon, Nov 21, 2022 at 03:07:25PM +, Simon Riggs wrote: > > Attached patch implements VACUUM (BACKGROUND). > > > > There are quite a few small details to consider; please read the docs > > and comments. > > > > There is a noticeable

Re: Allow single table VACUUM in transaction block

2022-11-22 Thread Justin Pryzby
On Mon, Nov 21, 2022 at 03:07:25PM +, Simon Riggs wrote: > Attached patch implements VACUUM (BACKGROUND). > > There are quite a few small details to consider; please read the docs > and comments. > > There is a noticeable delay before the background vacuum starts. You disallowed some

Re: Allow single table VACUUM in transaction block

2022-11-21 Thread Simon Riggs
On Fri, 18 Nov 2022 at 11:54, Simon Riggs wrote: > > On Thu, 17 Nov 2022 at 20:00, Justin Pryzby wrote: > > > > On Wed, Nov 16, 2022 at 05:14:07PM -0500, Greg Stark wrote: > > > I think this requesting autovacuum worker should be a distinct > > > command. Or at least an explicit option to

Re: Allow single table VACUUM in transaction block

2022-11-21 Thread Simon Riggs
On Fri, 18 Nov 2022 at 18:26, Tom Lane wrote: > > Robert Haas writes: > > On Fri, Nov 18, 2022 at 7:04 AM Simon Riggs > > wrote: > >> So if consistency is also a strong requirement, then maybe we should > >> make that new command the default, i.e. make VACUUM always just a > >> request to

Re: Allow single table VACUUM in transaction block

2022-11-18 Thread Tom Lane
Robert Haas writes: > On Fri, Nov 18, 2022 at 7:04 AM Simon Riggs > wrote: >> So if consistency is also a strong requirement, then maybe we should >> make that new command the default, i.e. make VACUUM always just a >> request to vacuum in background. That way it will be consistent. > Since one

Re: Allow single table VACUUM in transaction block

2022-11-18 Thread Robert Haas
On Fri, Nov 18, 2022 at 7:04 AM Simon Riggs wrote: > Outside a transaction - works perfectly > In a transaction - throws ERROR, which prevents a whole script from > executing correctly Right, but your proposal would move that inconsistency to a different place. It wouldn't eliminate it. I don't

Re: Allow single table VACUUM in transaction block

2022-11-18 Thread Simon Riggs
On Thu, 17 Nov 2022 at 20:06, Robert Haas wrote: > > On Wed, Nov 16, 2022 at 5:14 PM Greg Stark wrote: > > However I'm not a fan of commands that sometimes do one thing and > > sometimes magically do something very different. I don't like the idea > > that the same vacuum command would sometimes

Re: Allow single table VACUUM in transaction block

2022-11-18 Thread Simon Riggs
On Thu, 17 Nov 2022 at 20:00, Justin Pryzby wrote: > > On Wed, Nov 16, 2022 at 05:14:07PM -0500, Greg Stark wrote: > > I think this requesting autovacuum worker should be a distinct > > command. Or at least an explicit option to vacuum. > > +1. I was going to suggest VACUUM (NOWAIT) .. Yes, I

Re: Allow single table VACUUM in transaction block

2022-11-17 Thread Robert Haas
On Wed, Nov 16, 2022 at 5:14 PM Greg Stark wrote: > However I'm not a fan of commands that sometimes do one thing and > sometimes magically do something very different. I don't like the idea > that the same vacuum command would sometimes run in-process and > sometimes do this out of process

Re: Allow single table VACUUM in transaction block

2022-11-17 Thread Justin Pryzby
On Wed, Nov 16, 2022 at 05:14:07PM -0500, Greg Stark wrote: > I think this requesting autovacuum worker should be a distinct > command. Or at least an explicit option to vacuum. +1. I was going to suggest VACUUM (NOWAIT) .. -- Justin

Re: Allow single table VACUUM in transaction block

2022-11-16 Thread Tom Lane
Greg Stark writes: > I think this requesting autovacuum worker should be a distinct > command. Or at least an explicit option to vacuum. +1. That'd reduce confusion, and perhaps we could remove some of the restrictions. regards, tom lane

Re: Allow single table VACUUM in transaction block

2022-11-16 Thread Greg Stark
I think the idea of being able to request an autovacuum worker for a specific table is actually very good. I think it's what most users actually want when they are running vacuum. In fact in previous jobs people have built infrastructure that basically duplicates autovacuum just so they could do

Re: Allow single table VACUUM in transaction block

2022-11-15 Thread Simon Riggs
On Mon, 14 Nov 2022 at 19:52, Simon Riggs wrote: > > On Tue, 8 Nov 2022 at 03:10, Simon Riggs wrote: > > > > On Mon, 7 Nov 2022 at 08:20, Simon Riggs > > wrote: > > > > > Temp tables are actually easier, since we don't need any of the > > > concurrency features we get with lazy vacuum. > > >

Re: Allow single table VACUUM in transaction block

2022-11-14 Thread Simon Riggs
On Tue, 8 Nov 2022 at 03:10, Simon Riggs wrote: > > On Mon, 7 Nov 2022 at 08:20, Simon Riggs wrote: > > > Temp tables are actually easier, since we don't need any of the > > concurrency features we get with lazy vacuum. > Thoughts? New patch, which does this, when in a xact block 1. For temp

Re: Allow single table VACUUM in transaction block

2022-11-07 Thread Simon Riggs
On Mon, 7 Nov 2022 at 08:20, Simon Riggs wrote: > Temp tables are actually easier, since we don't need any of the > concurrency features we get with lazy vacuum. So the answer is to > always run a VACUUM FULL on temp tables since this skips any issues > with indexes etc.. So I see 3 options for

Re: Allow single table VACUUM in transaction block

2022-11-07 Thread Peter Geoghegan
On Mon, Nov 7, 2022 at 12:20 AM Simon Riggs wrote: > > Another related idea: better behavior in the event of a manually > > issued VACUUM (now just an enqueued autovacuum) that cannot do useful > > work due to the presence of a long running snapshot. The VACUUM > > doesn't have to dutifully

Re: Allow single table VACUUM in transaction block

2022-11-07 Thread Simon Riggs
On Sun, 6 Nov 2022 at 20:40, Peter Geoghegan wrote: > > On Sun, Nov 6, 2022 at 11:14 AM Tom Lane wrote: > > In general, I do not believe in encouraging users to run VACUUM > > manually in the first place. We would be far better served by > > spending our effort to improve autovacuum's

Re: Allow single table VACUUM in transaction block

2022-11-06 Thread Simon Riggs
On Sun, 6 Nov 2022 at 18:50, Peter Geoghegan wrote: > > On Thu, Oct 27, 2022 at 2:31 AM Simon Riggs > wrote: > > Fix, so that this works without issue: > > > > BEGIN; > > > > VACUUM (ANALYZE) vactst; > > > > COMMIT; > > > > Allows both ANALYZE and vacuum of toast tables, but not VACUUM

Re: Allow single table VACUUM in transaction block

2022-11-06 Thread Peter Geoghegan
On Sun, Nov 6, 2022 at 11:14 AM Tom Lane wrote: > In general, I do not believe in encouraging users to run VACUUM > manually in the first place. We would be far better served by > spending our effort to improve autovacuum's shortcomings. I couldn't agree more. A lot of problems seem related to

Re: Allow single table VACUUM in transaction block

2022-11-06 Thread Tom Lane
Peter Geoghegan writes: > My guess is that there are more things like that. Possibly even things > that were never directly considered. VACUUM evolved in a world where > we absolutely took not running in a transaction for granted. Changing > that now is a pretty big deal. Maybe it would all be

Re: Allow single table VACUUM in transaction block

2022-11-06 Thread Peter Geoghegan
On Thu, Oct 27, 2022 at 2:31 AM Simon Riggs wrote: > Fix, so that this works without issue: > > BEGIN; > > VACUUM (ANALYZE) vactst; > > COMMIT; > > Allows both ANALYZE and vacuum of toast tables, but not VACUUM FULL. > > When in a xact block, we do not set PROC_IN_VACUUM, > nor update

Re: Allow single table VACUUM in transaction block

2022-11-04 Thread Rahila Syed
Hi, On Fri, Nov 4, 2022 at 2:39 PM Simon Riggs wrote: > Hi Rahila, > > Thanks for your review. > > On Fri, 4 Nov 2022 at 07:37, Rahila Syed wrote: > > >> I would like to bring up a few points that I came across while looking > into the vacuum code. > >> > >> 1. As a result of this change to

Re: Allow single table VACUUM in transaction block

2022-11-04 Thread Simon Riggs
Hi Rahila, Thanks for your review. On Fri, 4 Nov 2022 at 07:37, Rahila Syed wrote: >> I would like to bring up a few points that I came across while looking into >> the vacuum code. >> >> 1. As a result of this change to allow VACUUM inside a user transaction, I >> think there is some

Re: Allow single table VACUUM in transaction block

2022-11-04 Thread Rahila Syed
Hi Simon, On Fri, Nov 4, 2022 at 10:15 AM Rahila Syed wrote: > Hi Simon, > > On Thu, Nov 3, 2022 at 3:53 PM Simon Riggs > wrote: > >> On Tue, 1 Nov 2022 at 23:56, Simon Riggs >> wrote: >> >> > > I haven't checked the rest of the patch, but +1 for allowing VACUUM >> FULL >> > > within a user

Re: Allow single table VACUUM in transaction block

2022-11-03 Thread Rahila Syed
Hi Simon, On Thu, Nov 3, 2022 at 3:53 PM Simon Riggs wrote: > On Tue, 1 Nov 2022 at 23:56, Simon Riggs > wrote: > > > > I haven't checked the rest of the patch, but +1 for allowing VACUUM > FULL > > > within a user txn. > > > > My intention was to prevent that. I am certainly quite uneasy

Re: Allow single table VACUUM in transaction block

2022-11-03 Thread Simon Riggs
On Tue, 1 Nov 2022 at 23:56, Simon Riggs wrote: > > I haven't checked the rest of the patch, but +1 for allowing VACUUM FULL > > within a user txn. > > My intention was to prevent that. I am certainly quite uneasy about > changing anything related to CLUSTER/VF, since they are old, complex > and

Re: Allow single table VACUUM in transaction block

2022-11-01 Thread Simon Riggs
On Thu, 27 Oct 2022 at 21:07, Justin Pryzby wrote: > > On Thu, Oct 27, 2022 at 10:31:31AM +0100, Simon Riggs wrote: > > Allows both ANALYZE and vacuum of toast tables, but not VACUUM FULL. > > Maybe I misunderstood what you meant: you said "not VACUUM FULL", but > with your patch, that works: > >

Re: Allow single table VACUUM in transaction block

2022-10-27 Thread Justin Pryzby
On Thu, Oct 27, 2022 at 10:31:31AM +0100, Simon Riggs wrote: > Allows both ANALYZE and vacuum of toast tables, but not VACUUM FULL. Maybe I misunderstood what you meant: you said "not VACUUM FULL", but with your patch, that works: postgres=# begin; VACUUM FULL pg_class; commit; BEGIN VACUUM

Re: Allow single table VACUUM in transaction block

2022-10-27 Thread Bharath Rupireddy
On Thu, Oct 27, 2022 at 9:49 PM Simon Riggs wrote: > > On Thu, 27 Oct 2022 at 10:31, Simon Riggs > wrote: > > > Tests, docs. > > The patch tester says that a pg_upgrade test is failing on Windows, > but works for me. > > t/002_pg_upgrade.pl .. ok > > Anybody shed any light on that, much

Re: Allow single table VACUUM in transaction block

2022-10-27 Thread Simon Riggs
On Thu, 27 Oct 2022 at 10:31, Simon Riggs wrote: > Tests, docs. The patch tester says that a pg_upgrade test is failing on Windows, but works for me. t/002_pg_upgrade.pl .. ok Anybody shed any light on that, much appreciated. -- Simon Riggshttp://www.EnterpriseDB.com/

Allow single table VACUUM in transaction block

2022-10-27 Thread Simon Riggs
It is a common user annoyance to have a script fail because someone added a VACUUM, especially when using --single-transaction option. Fix, so that this works without issue: BEGIN; VACUUM (ANALYZE) vactst; COMMIT; Allows both ANALYZE and vacuum of toast tables, but not VACUUM FULL.