Re: Should vacuum process config file reload more often

2023-04-28 Thread Daniel Gustafsson
> On 27 Apr 2023, at 23:25, Daniel Gustafsson wrote: >> On 27 Apr 2023, at 16:53, Melanie Plageman wrote: >> Fix LGTM. > > Thanks for review. I plan to push this in the morning. Done, thanks. -- Daniel Gustafsson

Re: Should vacuum process config file reload more often

2023-04-27 Thread Daniel Gustafsson
> On 27 Apr 2023, at 16:53, Melanie Plageman wrote: > On Thu, Apr 27, 2023 at 8:55 AM Daniel Gustafsson wrote: >> >>> On 27 Apr 2023, at 14:10, Masahiko Sawada wrote: >>> Good catch. I think the problem is that vacuum_rel() is called >>> recursively and we don't reset VacuumFailsafeActive

Re: Should vacuum process config file reload more often

2023-04-27 Thread Melanie Plageman
On Thu, Apr 27, 2023 at 8:55 AM Daniel Gustafsson wrote: > > > On 27 Apr 2023, at 14:10, Masahiko Sawada wrote: > > > > On Thu, Apr 27, 2023 at 6:30 PM John Naylor > > wrote: > >> > >> > >> On Fri, Apr 7, 2023 at 6:08 AM Daniel Gustafsson wrote: > >>> > >>> I had another read-through and

Re: Should vacuum process config file reload more often

2023-04-27 Thread Daniel Gustafsson
> On 27 Apr 2023, at 14:10, Masahiko Sawada wrote: > > On Thu, Apr 27, 2023 at 6:30 PM John Naylor > wrote: >> >> >> On Fri, Apr 7, 2023 at 6:08 AM Daniel Gustafsson wrote: >>> >>> I had another read-through and test-through of this version, and have >>> applied >>> it with some minor

Re: Should vacuum process config file reload more often

2023-04-27 Thread Masahiko Sawada
On Thu, Apr 27, 2023 at 6:30 PM John Naylor wrote: > > > On Fri, Apr 7, 2023 at 6:08 AM Daniel Gustafsson wrote: > > > > I had another read-through and test-through of this version, and have > > applied > > it with some minor changes to comments and whitespace. Thanks for the quick > >

Re: Should vacuum process config file reload more often

2023-04-27 Thread Daniel Gustafsson
> On 27 Apr 2023, at 11:29, John Naylor wrote: > On Fri, Apr 7, 2023 at 6:08 AM Daniel Gustafsson wrote: > > I had another read-through and test-through of this version, and have > > applied > > it with some minor changes to comments and whitespace. Thanks for the quick > > turnaround times

Re: Should vacuum process config file reload more often

2023-04-27 Thread John Naylor
On Fri, Apr 7, 2023 at 6:08 AM Daniel Gustafsson wrote: > > I had another read-through and test-through of this version, and have applied > it with some minor changes to comments and whitespace. Thanks for the quick > turnaround times on reviews in this thread! - VacuumFailsafeActive =

Re: Should vacuum process config file reload more often

2023-04-25 Thread Masahiko Sawada
On Tue, Apr 25, 2023 at 10:35 PM Daniel Gustafsson wrote: > > > On 25 Apr 2023, at 15:31, Masahiko Sawada wrote: > > > > On Tue, Apr 25, 2023 at 9:39 PM Daniel Gustafsson wrote: > >> > >>> On 17 Apr 2023, at 04:04, Masahiko Sawada wrote: > >> > >>> I've attached an updated patch for fixing

Re: Should vacuum process config file reload more often

2023-04-25 Thread Daniel Gustafsson
> On 25 Apr 2023, at 15:31, Masahiko Sawada wrote: > > On Tue, Apr 25, 2023 at 9:39 PM Daniel Gustafsson wrote: >> >>> On 17 Apr 2023, at 04:04, Masahiko Sawada wrote: >> >>> I've attached an updated patch for fixing at_dobalance condition. >> >> I revisited this and pushed it to all

Re: Should vacuum process config file reload more often

2023-04-25 Thread Masahiko Sawada
On Tue, Apr 25, 2023 at 9:39 PM Daniel Gustafsson wrote: > > > On 17 Apr 2023, at 04:04, Masahiko Sawada wrote: > > > I've attached an updated patch for fixing at_dobalance condition. > > I revisited this and pushed it to all supported branches after another round > of > testing and reading.

Re: Should vacuum process config file reload more often

2023-04-25 Thread Daniel Gustafsson
> On 17 Apr 2023, at 04:04, Masahiko Sawada wrote: > I've attached an updated patch for fixing at_dobalance condition. I revisited this and pushed it to all supported branches after another round of testing and reading. -- Daniel Gustafsson

Re: Should vacuum process config file reload more often

2023-04-16 Thread Masahiko Sawada
On Wed, Apr 12, 2023 at 12:05 AM Masahiko Sawada wrote: > > On Fri, Apr 7, 2023 at 10:23 PM Daniel Gustafsson wrote: > > > > > On 7 Apr 2023, at 15:07, Melanie Plageman > > > wrote: > > > On Fri, Apr 7, 2023 at 2:53 AM Masahiko Sawada > > > wrote: > > > > > > Definitely seems worth fixing as

Re: Should vacuum process config file reload more often

2023-04-15 Thread Daniel Gustafsson
> On 11 Apr 2023, at 17:05, Masahiko Sawada wrote: > The comment of message_level_is_interesting() says: > > * This is useful to short-circuit any expensive preparatory work that > * might be needed for a logging message. > > Which can apply to taking a lwlock, I think. I agree that we can,

Re: Should vacuum process config file reload more often

2023-04-11 Thread Masahiko Sawada
On Fri, Apr 7, 2023 at 10:23 PM Daniel Gustafsson wrote: > > > On 7 Apr 2023, at 15:07, Melanie Plageman wrote: > > On Fri, Apr 7, 2023 at 2:53 AM Masahiko Sawada > > wrote: > > >> + /* Only log updates to cost-related variables */ > >> + if (vacuum_cost_delay ==

Re: Should vacuum process config file reload more often

2023-04-10 Thread Melanie Plageman
On Fri, Apr 7, 2023 at 9:07 AM Melanie Plageman wrote: > > On Fri, Apr 7, 2023 at 2:53 AM Masahiko Sawada wrote: > > > > On Fri, Apr 7, 2023 at 8:08 AM Daniel Gustafsson wrote: > > > > > > > On 7 Apr 2023, at 00:12, Melanie Plageman > > > > wrote: > > > > > > > > On Thu, Apr 6, 2023 at 5:45 

Re: Should vacuum process config file reload more often

2023-04-07 Thread Daniel Gustafsson
> On 7 Apr 2023, at 15:07, Melanie Plageman wrote: > On Fri, Apr 7, 2023 at 2:53 AM Masahiko Sawada wrote: >> + /* Only log updates to cost-related variables */ >> + if (vacuum_cost_delay == original_cost_delay && >> + vacuum_cost_limit == original_cost_limit) >> +

Re: Should vacuum process config file reload more often

2023-04-07 Thread Melanie Plageman
On Fri, Apr 7, 2023 at 2:53 AM Masahiko Sawada wrote: > > On Fri, Apr 7, 2023 at 8:08 AM Daniel Gustafsson wrote: > > > > > On 7 Apr 2023, at 00:12, Melanie Plageman > > > wrote: > > > > > > On Thu, Apr 6, 2023 at 5:45 PM Daniel Gustafsson wrote: > > >> > > >>> On 6 Apr 2023, at 23:06,

Re: Should vacuum process config file reload more often

2023-04-07 Thread Daniel Gustafsson
> On 7 Apr 2023, at 08:52, Masahiko Sawada wrote: > On Fri, Apr 7, 2023 at 8:08 AM Daniel Gustafsson wrote: >> I had another read-through and test-through of this version, and have applied >> it with some minor changes to comments and whitespace. Thanks for the quick >> turnaround times on

Re: Should vacuum process config file reload more often

2023-04-07 Thread Masahiko Sawada
On Fri, Apr 7, 2023 at 8:08 AM Daniel Gustafsson wrote: > > > On 7 Apr 2023, at 00:12, Melanie Plageman wrote: > > > > On Thu, Apr 6, 2023 at 5:45 PM Daniel Gustafsson wrote: > >> > >>> On 6 Apr 2023, at 23:06, Melanie Plageman > >>> wrote: > >> > >>> Autovacuum workers, at the end of

Re: Should vacuum process config file reload more often

2023-04-06 Thread Daniel Gustafsson
> On 7 Apr 2023, at 00:12, Melanie Plageman wrote: > > On Thu, Apr 6, 2023 at 5:45 PM Daniel Gustafsson wrote: >> >>> On 6 Apr 2023, at 23:06, Melanie Plageman wrote: >> >>> Autovacuum workers, at the end of VacuumUpdateCosts(), check if cost >>> limit or cost delay have been changed. If

Re: Should vacuum process config file reload more often

2023-04-06 Thread Melanie Plageman
On Thu, Apr 6, 2023 at 5:45 PM Daniel Gustafsson wrote: > > > On 6 Apr 2023, at 23:06, Melanie Plageman wrote: > > > Autovacuum workers, at the end of VacuumUpdateCosts(), check if cost > > limit or cost delay have been changed. If they have, they assert that > > they don't already hold the

Re: Should vacuum process config file reload more often

2023-04-06 Thread Daniel Gustafsson
> On 6 Apr 2023, at 23:06, Melanie Plageman wrote: > Autovacuum workers, at the end of VacuumUpdateCosts(), check if cost > limit or cost delay have been changed. If they have, they assert that > they don't already hold the AutovacuumLock, take it in shared mode, and > do the logging. Another

Re: Should vacuum process config file reload more often

2023-04-06 Thread Melanie Plageman
I think attached v18 addresses all outstanding issues except a run through the docs making sure all mentions of the balancing algorithm are still correct. On Wed, Apr 5, 2023 at 9:10 AM Daniel Gustafsson wrote: > > On 4 Apr 2023, at 22:04, Melanie Plageman wrote: > >> +extern int

Re: Should vacuum process config file reload more often

2023-04-06 Thread Melanie Plageman
v17 attached does not yet fix the logging problem or variable naming problem. I have not changed where AutoVacuumUpdateCostLimit() is called either. This is effectively just a round of cleanup. I hope I have managed to address all other code review feedback so far, though some may have slipped

Re: Should vacuum process config file reload more often

2023-04-06 Thread Daniel Gustafsson
> On 6 Apr 2023, at 19:18, Robert Haas wrote: > > On Thu, Apr 6, 2023 at 11:52 AM Melanie Plageman > wrote: >>> Gah, I think I misunderstood you. You are saying that only calling >>> AutoVacuumUpdateCostLimit() after napping while vacuuming a table may >>> not be enough. The frequency at which

Re: Should vacuum process config file reload more often

2023-04-06 Thread Robert Haas
On Wed, Apr 5, 2023 at 4:59 PM Peter Geoghegan wrote: > I think that I agree. I think that the difficulty of tuning autovacuum > is the actual real problem. (Or maybe it's just very closely related > to the real problem -- the precise definition doesn't seem important.) I agree, and I think that

Re: Should vacuum process config file reload more often

2023-04-06 Thread Robert Haas
On Thu, Apr 6, 2023 at 11:52 AM Melanie Plageman wrote: > > Gah, I think I misunderstood you. You are saying that only calling > > AutoVacuumUpdateCostLimit() after napping while vacuuming a table may > > not be enough. The frequency at which the number of workers changes will > > likely be

Re: Should vacuum process config file reload more often

2023-04-06 Thread Melanie Plageman
On Wed, Apr 5, 2023 at 11:10 PM Melanie Plageman wrote: > On Wed, Apr 5, 2023 at 3:43 PM Melanie Plageman > wrote: > > On Wed, Apr 5, 2023 at 2:56 PM Robert Haas wrote: > > > > > > + /* > > > + * Balance and update limit values for autovacuum workers. We must > > > + * always do this in case

Re: Should vacuum process config file reload more often

2023-04-06 Thread Daniel Gustafsson
> On 6 Apr 2023, at 08:39, Masahiko Sawada wrote: > Also I agree with > where to put the log but I think the log message should start with > lower cases: > > +elog(DEBUG2, > + "Autovacuum VacuumUpdateCosts(db=%u, rel=%u, In principle I agree, but in this

Re: Should vacuum process config file reload more often

2023-04-06 Thread Masahiko Sawada
On Thu, Apr 6, 2023 at 12:29 AM Melanie Plageman wrote: > > Thanks all for the reviews. > > v16 attached. I put it together rather quickly, so there might be a few > spurious whitespaces or similar. There is one rather annoying pgindent > outlier that I have to figure out what to do about as

Re: Should vacuum process config file reload more often

2023-04-05 Thread Melanie Plageman
On Wed, Apr 5, 2023 at 3:43 PM Melanie Plageman wrote: > > On Wed, Apr 5, 2023 at 2:56 PM Robert Haas wrote: > > > > + /* > > + * Balance and update limit values for autovacuum workers. We must > > + * always do this in case the autovacuum launcher or another > > + * autovacuum worker has

Re: Should vacuum process config file reload more often

2023-04-05 Thread Peter Geoghegan
On Wed, Apr 5, 2023 at 1:38 PM Daniel Gustafsson wrote: > Not to derail this thread, and pre-empt a thread where this can be discussed > in > its own context, but isn't that kind of the main problem? Tuning autovacuum > is > really complicated and one of the parameters that I think universally

Re: Should vacuum process config file reload more often

2023-04-05 Thread Daniel Gustafsson
> On 5 Apr 2023, at 22:19, Peter Geoghegan wrote: > The bigger problem seems to > be everything else -- the way that tuning autovacuum_max_workers kinda > makes sense (it shouldn't be an interesting tunable) Not to derail this thread, and pre-empt a thread where this can be discussed in its own

Re: Should vacuum process config file reload more often

2023-04-05 Thread Peter Geoghegan
On Wed, Apr 5, 2023 at 11:56 AM Robert Haas wrote: > To be honest, I think that the whole system where we divide the cost > limit across the workers is the wrong idea. Does anyone actually like > that behavior? This patch probably shouldn't touch that, just in the > interest of getting something

Re: Should vacuum process config file reload more often

2023-04-05 Thread Robert Haas
On Wed, Apr 5, 2023 at 3:44 PM Melanie Plageman wrote: > VacuumUpdateCosts() also calls AutoVacuumUpdateCostLimit(), so this will > happen if a config reload is pending the next time vacuum_delay_point() > is called (which is pretty often -- roughly once per block vacuumed but > definitely more

Re: Should vacuum process config file reload more often

2023-04-05 Thread Melanie Plageman
On Wed, Apr 5, 2023 at 2:56 PM Robert Haas wrote: > > + /* > + * Balance and update limit values for autovacuum workers. We must > + * always do this in case the autovacuum launcher or another > + * autovacuum worker has recalculated the number of workers across > + * which we must balance the

Re: Should vacuum process config file reload more often

2023-04-05 Thread Robert Haas
On Wed, Apr 5, 2023 at 3:04 PM Daniel Gustafsson wrote: > > Daniel: Are you intending to commit this? > > Yes, my plan is to get it in before feature freeze. All right, let's make it happen! I think this is pretty close to ready to ship, and it would solve a problem that is real, annoying, and

Re: Should vacuum process config file reload more often

2023-04-05 Thread Daniel Gustafsson
> On 5 Apr 2023, at 20:55, Robert Haas wrote: > Again, I don't think this is something we should try to > address right now under time pressure, but in the future, I think we > should consider ripping this behavior out. I would not be opposed to that, but I wholeheartedly agree that it's not

Re: Should vacuum process config file reload more often

2023-04-05 Thread Robert Haas
On Wed, Apr 5, 2023 at 11:29 AM Melanie Plageman wrote: > Thanks all for the reviews. > > v16 attached. I put it together rather quickly, so there might be a few > spurious whitespaces or similar. There is one rather annoying pgindent > outlier that I have to figure out what to do about as well.

Re: Should vacuum process config file reload more often

2023-04-05 Thread Daniel Gustafsson
> On 5 Apr 2023, at 17:29, Melanie Plageman wrote: >> >> I think I wasn't clear in my comment, sorry. I don't have a problem with >> introducing a new variable to split the balanced value from the GUC value. >> What I don't think we should do is repurpose an exported symbol into doing a >> new

Re: Should vacuum process config file reload more often

2023-04-05 Thread Melanie Plageman
Thanks all for the reviews. v16 attached. I put it together rather quickly, so there might be a few spurious whitespaces or similar. There is one rather annoying pgindent outlier that I have to figure out what to do about as well. The remaining functional TODOs that I know of are: - Resolve

Re: Should vacuum process config file reload more often

2023-04-05 Thread Daniel Gustafsson
> On 4 Apr 2023, at 22:04, Melanie Plageman wrote: > > On Tue, Apr 4, 2023 at 4:27 AM Masahiko Sawada wrote: >> Also, I don't think there is any reason why we want to exclude only >> the autovacuum launcher. > > My rationale is that the launcher is the only other process type which > might

Re: Should vacuum process config file reload more often

2023-04-05 Thread Kyotaro Horiguchi
Hi. About 0001: + * VacuumFailsafeActive is a defined as a global so that we can determine + * whether or not to re-enable cost-based vacuum delay when vacuuming a table. + * If failsafe mode has been engaged, we will not re-enable cost-based delay + * for the table until after vacuuming has

Re: Should vacuum process config file reload more often

2023-04-04 Thread Masahiko Sawada
On Wed, Apr 5, 2023 at 5:05 AM Melanie Plageman wrote: > > On Tue, Apr 4, 2023 at 4:27 AM Masahiko Sawada wrote: > > --- > > -if (worker->wi_proc != NULL) > > -elog(DEBUG2, "autovac_balance_cost(pid=%d > > db=%u, rel=%u, dobalance=%s cost_limit=%d,

Re: Should vacuum process config file reload more often

2023-04-04 Thread Melanie Plageman
On Tue, Apr 4, 2023 at 4:27 AM Masahiko Sawada wrote: > The 0001 patch mostly looks good to me except for one > point: > > @@ -391,7 +389,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params, > Assert(params->index_cleanup != VACOPTVALUE_UNSPECIFIED); >

Re: Should vacuum process config file reload more often

2023-04-04 Thread Daniel Gustafsson
> On 4 Apr 2023, at 00:35, Melanie Plageman wrote: > > On Mon, Apr 3, 2023 at 3:08 PM Andres Freund wrote: >> On 2023-04-03 14:43:14 -0400, Tom Lane wrote: >>> Melanie Plageman writes: v13 attached with requested updates. >>> >>> I'm afraid I'd not been paying any attention to this

Re: Should vacuum process config file reload more often

2023-04-04 Thread Masahiko Sawada
On Tue, Apr 4, 2023 at 1:41 AM Melanie Plageman wrote: > > On Sun, Apr 2, 2023 at 10:28 PM Masahiko Sawada wrote: > > Thank you for updating the patches. Here are comments for 0001, 0002, > > and 0003 patches: > > Thanks for the review! > > v13 attached with requested updates. > > > 0001: > > >

Re: Should vacuum process config file reload more often

2023-04-03 Thread Melanie Plageman
On Mon, Apr 3, 2023 at 3:08 PM Andres Freund wrote: > On 2023-04-03 14:43:14 -0400, Tom Lane wrote: > > Melanie Plageman writes: > > > v13 attached with requested updates. > > > > I'm afraid I'd not been paying any attention to this discussion, > > but better late than never. I'm okay with

Re: Should vacuum process config file reload more often

2023-04-03 Thread Andres Freund
Hi, On 2023-04-03 14:43:14 -0400, Tom Lane wrote: > Melanie Plageman writes: > > v13 attached with requested updates. > > I'm afraid I'd not been paying any attention to this discussion, > but better late than never. I'm okay with letting autovacuum > processes reload config files more often

Re: Should vacuum process config file reload more often

2023-04-03 Thread Tom Lane
Melanie Plageman writes: > v13 attached with requested updates. I'm afraid I'd not been paying any attention to this discussion, but better late than never. I'm okay with letting autovacuum processes reload config files more often than now. However, I object to allowing ProcessConfigFile to be

Re: Should vacuum process config file reload more often

2023-04-03 Thread Melanie Plageman
On Sun, Apr 2, 2023 at 10:28 PM Masahiko Sawada wrote: > Thank you for updating the patches. Here are comments for 0001, 0002, > and 0003 patches: Thanks for the review! v13 attached with requested updates. > 0001: > > @@ -391,7 +389,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params, >

Re: Should vacuum process config file reload more often

2023-04-02 Thread Masahiko Sawada
On Sat, Apr 1, 2023 at 4:09 AM Melanie Plageman wrote: > > On Fri, Mar 31, 2023 at 10:31 AM Melanie Plageman > wrote: > > > > On Thu, Mar 30, 2023 at 3:26 PM Daniel Gustafsson wrote: > > > > > > > On 30 Mar 2023, at 04:57, Masahiko Sawada wrote: > > > > > > > As another idea, why don't we use

Re: Should vacuum process config file reload more often

2023-03-31 Thread Melanie Plageman
On Fri, Mar 31, 2023 at 10:31 AM Melanie Plageman wrote: > > On Thu, Mar 30, 2023 at 3:26 PM Daniel Gustafsson wrote: > > > > > On 30 Mar 2023, at 04:57, Masahiko Sawada wrote: > > > > > As another idea, why don't we use macros for that? For example, > > > suppose VacuumCostStatus is like: > >

Re: Should vacuum process config file reload more often

2023-03-31 Thread Melanie Plageman
On Thu, Mar 30, 2023 at 3:26 PM Daniel Gustafsson wrote: > > > On 30 Mar 2023, at 04:57, Masahiko Sawada wrote: > > > As another idea, why don't we use macros for that? For example, > > suppose VacuumCostStatus is like: > > > > typedef enum VacuumCostStatus > > { > >

Re: Should vacuum process config file reload more often

2023-03-30 Thread Daniel Gustafsson
> On 30 Mar 2023, at 04:57, Masahiko Sawada wrote: > As another idea, why don't we use macros for that? For example, > suppose VacuumCostStatus is like: > > typedef enum VacuumCostStatus > { >VACUUM_COST_INACTIVE_LOCKED = 0, >VACUUM_COST_INACTIVE, >VACUUM_COST_ACTIVE, > }

Re: Should vacuum process config file reload more often

2023-03-29 Thread Masahiko Sawada
Hi, Thank you for updating the patches. On Thu, Mar 30, 2023 at 5:01 AM Melanie Plageman wrote: > > Thanks for the detailed review! > > On Tue, Mar 28, 2023 at 11:09 PM Kyotaro Horiguchi > wrote: > > > > At Tue, 28 Mar 2023 20:35:28 -0400, Melanie Plageman > > wrote in > > > On Tue, Mar 28,

Re: Should vacuum process config file reload more often

2023-03-29 Thread Melanie Plageman
Thanks for the detailed review! On Tue, Mar 28, 2023 at 11:09 PM Kyotaro Horiguchi wrote: > > At Tue, 28 Mar 2023 20:35:28 -0400, Melanie Plageman > wrote in > > On Tue, Mar 28, 2023 at 4:21 AM Kyotaro Horiguchi > > wrote: > > > > > > At Mon, 27 Mar 2023 14:12:03 -0400, Melanie Plageman > >

Re: Should vacuum process config file reload more often

2023-03-29 Thread Kyotaro Horiguchi
At Wed, 29 Mar 2023 13:21:55 +0900 (JST), Kyotaro Horiguchi wrote in > So, sorry for the noise. I'll review it while this into cnosideration. 0003: It's not this patche's fault, but I don't like the fact that the variables used for GUC, VacuumCostDelay and VacuumCostLimit, are updated outside

Re: Should vacuum process config file reload more often

2023-03-29 Thread Kyotaro Horiguchi
At Wed, 29 Mar 2023 13:21:55 +0900 (JST), Kyotaro Horiguchi wrote in > autovacuum.c:2893 > /* >* If any of the cost delay parameters has been set > individually for >* this table, disable the balancing algorithm. >*/ >

Re: Should vacuum process config file reload more often

2023-03-28 Thread Kyotaro Horiguchi
At Wed, 29 Mar 2023 12:09:08 +0900 (JST), Kyotaro Horiguchi wrote in > timeing perfectly. I think we might accidentally add a reload > timing. In that case, the assumption could break. In most cases, I > think we use snapshotting in various ways to avoid unintended variable > changes. (And I

Re: Should vacuum process config file reload more often

2023-03-28 Thread Kyotaro Horiguchi
At Tue, 28 Mar 2023 20:35:28 -0400, Melanie Plageman wrote in > On Tue, Mar 28, 2023 at 4:21 AM Kyotaro Horiguchi > wrote: > > > > At Mon, 27 Mar 2023 14:12:03 -0400, Melanie Plageman > > wrote in > > > So, I've attached an alternate version of the patchset which takes the > > > approach of

Re: Should vacuum process config file reload more often

2023-03-28 Thread Melanie Plageman
On Tue, Mar 28, 2023 at 4:21 AM Kyotaro Horiguchi wrote: > > At Mon, 27 Mar 2023 14:12:03 -0400, Melanie Plageman > wrote in > > So, I've attached an alternate version of the patchset which takes the > > approach of having one commit which only enables cost-based delay GUC > > refresh for

Re: Should vacuum process config file reload more often

2023-03-28 Thread Kyotaro Horiguchi
At Mon, 27 Mar 2023 14:12:03 -0400, Melanie Plageman wrote in > So, I've attached an alternate version of the patchset which takes the > approach of having one commit which only enables cost-based delay GUC > refresh for VACUUM and another commit which enables it for autovacuum > and makes the

Re: Should vacuum process config file reload more often

2023-03-27 Thread Melanie Plageman
On Sat, Mar 25, 2023 at 3:03 PM Melanie Plageman wrote: > > On Thu, Mar 23, 2023 at 8:27 PM Melanie Plageman > wrote: > > On Thu, Mar 23, 2023 at 2:09 AM Masahiko Sawada > > wrote: > > > > And, I was wondering if it was worth trying to split up the part that > > > > reloads the config file and

Re: Should vacuum process config file reload more often

2023-03-25 Thread Melanie Plageman
On Thu, Mar 23, 2023 at 8:27 PM Melanie Plageman wrote: > On Thu, Mar 23, 2023 at 2:09 AM Masahiko Sawada wrote: > > > And, I was wondering if it was worth trying to split up the part that > > > reloads the config file and all of the autovacuum stuff. The reloading > > > of the config file by

Re: Should vacuum process config file reload more often

2023-03-24 Thread Melanie Plageman
On Fri, Mar 24, 2023 at 1:21 AM Masahiko Sawada wrote: > > On Fri, Mar 24, 2023 at 9:27 AM Melanie Plageman > wrote: > > > > On Thu, Mar 23, 2023 at 2:09 AM Masahiko Sawada > > wrote: > > > On Sun, Mar 19, 2023 at 7:47 AM Melanie Plageman > > > wrote: > > > Do we need to calculate the number

Re: Should vacuum process config file reload more often

2023-03-23 Thread Masahiko Sawada
On Fri, Mar 24, 2023 at 9:27 AM Melanie Plageman wrote: > > On Thu, Mar 23, 2023 at 2:09 AM Masahiko Sawada wrote: > > On Sun, Mar 19, 2023 at 7:47 AM Melanie Plageman > > wrote: > > Do we need to calculate the number of workers running with > > nworkers_for_balance by iterating over the

Re: Should vacuum process config file reload more often

2023-03-23 Thread Melanie Plageman
On Thu, Mar 23, 2023 at 2:09 AM Masahiko Sawada wrote: > On Sun, Mar 19, 2023 at 7:47 AM Melanie Plageman > wrote: > Do we need to calculate the number of workers running with > nworkers_for_balance by iterating over the running worker list? I > guess autovacuum workers can increment/decrement

Re: Should vacuum process config file reload more often

2023-03-23 Thread Daniel Gustafsson
> On 23 Mar 2023, at 07:08, Masahiko Sawada wrote: > On Sun, Mar 19, 2023 at 7:47 AM Melanie Plageman > wrote: > It makes sense to me that we need to reload the config file even when > vacuum-delay is disabled. But I think it's not convenient for users > that we don't reload the configuration

Re: Should vacuum process config file reload more often

2023-03-23 Thread Masahiko Sawada
On Sun, Mar 19, 2023 at 7:47 AM Melanie Plageman wrote: > > On Wed, Mar 15, 2023 at 1:14 AM Masahiko Sawada wrote: > > On Sat, Mar 11, 2023 at 8:11 AM Melanie Plageman > > wrote: > > > I've implemented the atomic cost limit in the attached patch. Though, > > > I'm pretty unsure about how I

Re: Should vacuum process config file reload more often

2023-03-19 Thread Melanie Plageman
On Sat, Mar 18, 2023 at 6:47 PM Melanie Plageman wrote: > On Wed, Mar 15, 2023 at 1:14 AM Masahiko Sawada wrote: > > On Sat, Mar 11, 2023 at 8:11 AM Melanie Plageman > > wrote: > > > > > Also not sure how the patch interacts with failsafe autovac and > > > > > parallel > > > > > vacuum. > > >

Re: Should vacuum process config file reload more often

2023-03-18 Thread Melanie Plageman
On Wed, Mar 15, 2023 at 1:14 AM Masahiko Sawada wrote: > On Sat, Mar 11, 2023 at 8:11 AM Melanie Plageman > wrote: > > I've implemented the atomic cost limit in the attached patch. Though, > > I'm pretty unsure about how I initialized the atomics in > > AutoVacuumShmemInit()... > > + >

Re: Should vacuum process config file reload more often

2023-03-14 Thread Masahiko Sawada
On Sat, Mar 11, 2023 at 8:11 AM Melanie Plageman wrote: > > Quotes below are combined from two of Sawada-san's emails. > > I've also attached a patch with my suggested current version. > > On Thu, Mar 9, 2023 at 10:27 PM Masahiko Sawada wrote: > > > > On Fri, Mar 10, 2023 at 11:23 AM Melanie

Re: Should vacuum process config file reload more often

2023-03-10 Thread Melanie Plageman
On Fri, Mar 10, 2023 at 6:11 PM Melanie Plageman wrote: > > Quotes below are combined from two of Sawada-san's emails. > > I've also attached a patch with my suggested current version. > > On Thu, Mar 9, 2023 at 10:27 PM Masahiko Sawada wrote: > > > > On Fri, Mar 10, 2023 at 11:23 AM Melanie

Re: Should vacuum process config file reload more often

2023-03-10 Thread Melanie Plageman
Quotes below are combined from two of Sawada-san's emails. I've also attached a patch with my suggested current version. On Thu, Mar 9, 2023 at 10:27 PM Masahiko Sawada wrote: > > On Fri, Mar 10, 2023 at 11:23 AM Melanie Plageman > wrote: > > > > On Tue, Mar 7, 2023 at 12:10 AM Masahiko Sawada

Re: Should vacuum process config file reload more often

2023-03-09 Thread Masahiko Sawada
On Fri, Mar 10, 2023 at 11:23 AM Melanie Plageman wrote: > > On Tue, Mar 7, 2023 at 12:10 AM Masahiko Sawada wrote: > > > > On Mon, Mar 6, 2023 at 5:26 AM Melanie Plageman > > wrote: > > > > > > On Thu, Mar 2, 2023 at 6:37 PM Melanie Plageman > > > In this version I've removed wi_cost_delay

Re: Should vacuum process config file reload more often

2023-03-09 Thread Melanie Plageman
On Tue, Mar 7, 2023 at 12:10 AM Masahiko Sawada wrote: > > On Mon, Mar 6, 2023 at 5:26 AM Melanie Plageman > wrote: > > > > On Thu, Mar 2, 2023 at 6:37 PM Melanie Plageman > > In this version I've removed wi_cost_delay from WorkerInfoData. There is > > no synchronization of cost_delay amongst

Re: Should vacuum process config file reload more often

2023-03-09 Thread Masahiko Sawada
On Thu, Mar 9, 2023 at 4:47 PM John Naylor wrote: > > > On Thu, Mar 9, 2023 at 12:42 AM Jim Nasby wrote: > > > > Doesn't the dead tuple space grow as needed? Last I looked we don't > > allocate up to 1GB right off the bat. > > Incorrect. > > > Of course, if the patch that eliminates the 1GB

Re: Should vacuum process config file reload more often

2023-03-08 Thread John Naylor
On Thu, Mar 9, 2023 at 12:42 AM Jim Nasby wrote: > > Doesn't the dead tuple space grow as needed? Last I looked we don't allocate up to 1GB right off the bat. Incorrect. > Of course, if the patch that eliminates the 1GB vacuum limit gets committed the situation will be even worse. If you're

Re: Should vacuum process config file reload more often

2023-03-08 Thread Andres Freund
Hi, On 2023-03-08 11:42:31 -0600, Jim Nasby wrote: > On 3/2/23 1:36 AM, Masahiko Sawada wrote: > > > > > > For example, I guess we will need to take care of changes of > > > > > maintenance_work_mem. Currently we initialize the dead tuple space at > > > > > the beginning of lazy vacuum, but

Re: Should vacuum process config file reload more often

2023-03-08 Thread Jim Nasby
On 3/2/23 1:36 AM, Masahiko Sawada wrote: For example, I guess we will need to take care of changes of maintenance_work_mem. Currently we initialize the dead tuple space at the beginning of lazy vacuum, but perhaps we would need to enlarge/shrink it based on the new value? Doesn't the dead

Re: Should vacuum process config file reload more often

2023-03-06 Thread Masahiko Sawada
On Mon, Mar 6, 2023 at 5:26 AM Melanie Plageman wrote: > > On Thu, Mar 2, 2023 at 6:37 PM Melanie Plageman > wrote: > > > > On Thu, Mar 2, 2023 at 2:36 AM Masahiko Sawada > > wrote: > > > > > > On Thu, Mar 2, 2023 at 10:41 AM Melanie Plageman > > > wrote: > > > > On another topic, I've just

Re: Should vacuum process config file reload more often

2023-03-05 Thread Melanie Plageman
On Thu, Mar 2, 2023 at 6:37 PM Melanie Plageman wrote: > > On Thu, Mar 2, 2023 at 2:36 AM Masahiko Sawada wrote: > > > > On Thu, Mar 2, 2023 at 10:41 AM Melanie Plageman > > wrote: > > > On another topic, I've just realized that when autovacuuming we only > > > update

Re: Should vacuum process config file reload more often

2023-03-02 Thread Melanie Plageman
On Thu, Mar 2, 2023 at 2:36 AM Masahiko Sawada wrote: > > On Thu, Mar 2, 2023 at 10:41 AM Melanie Plageman > wrote: > > On another topic, I've just realized that when autovacuuming we only > > update tab->at_vacuum_cost_delay/limit from > > autovacuum_vacuum_cost_delay/limit for each table (in >

Re: Should vacuum process config file reload more often

2023-03-01 Thread Masahiko Sawada
On Thu, Mar 2, 2023 at 10:41 AM Melanie Plageman wrote: > > On Mon, Feb 27, 2023 at 9:12 AM Masahiko Sawada wrote: > > On Fri, Feb 24, 2023 at 7:08 AM Melanie Plageman > > wrote: > > > Users may wish to speed up long-running vacuum of a large table by > > > decreasing

Re: Should vacuum process config file reload more often

2023-03-01 Thread Amit Kapila
On Thu, Mar 2, 2023 at 5:45 AM Andres Freund wrote: > > On 2023-02-28 11:16:45 +0900, Masahiko Sawada wrote: > > On Tue, Feb 28, 2023 at 10:21 AM Andres Freund wrote: > > > On 2023-02-27 23:11:53 +0900, Masahiko Sawada wrote: > > > > As far as I know there are not such GUC parameters in the core

Re: Should vacuum process config file reload more often

2023-03-01 Thread Melanie Plageman
On Mon, Feb 27, 2023 at 9:12 AM Masahiko Sawada wrote: > On Fri, Feb 24, 2023 at 7:08 AM Melanie Plageman > wrote: > > Users may wish to speed up long-running vacuum of a large table by > > decreasing autovacuum_vacuum_cost_delay/vacuum_cost_delay, however the > > config file is only reloaded

Re: Should vacuum process config file reload more often

2023-03-01 Thread Andres Freund
Hi, On 2023-02-28 11:16:45 +0900, Masahiko Sawada wrote: > On Tue, Feb 28, 2023 at 10:21 AM Andres Freund wrote: > > On 2023-02-27 23:11:53 +0900, Masahiko Sawada wrote: > > > As far as I know there are not such GUC parameters in the core but > > > there might be in third-party table AM and

Re: Should vacuum process config file reload more often

2023-03-01 Thread Melanie Plageman
Thanks for the feedback and questions, Pavel! On Fri, Feb 24, 2023 at 3:43 AM Pavel Borisov wrote: > I have a couple of small questions: > Can this patch also read the current GUC value if it's modified by the > SET command, without editing config file? If a user sets a guc like

Re: Should vacuum process config file reload more often

2023-02-27 Thread Masahiko Sawada
On Tue, Feb 28, 2023 at 10:21 AM Andres Freund wrote: > > Hi, > > On 2023-02-27 23:11:53 +0900, Masahiko Sawada wrote: > > As far as I know there are not such GUC parameters in the core but > > there might be in third-party table AM and index AM extensions. > > We already reload in a pretty broad

Re: Should vacuum process config file reload more often

2023-02-27 Thread Andres Freund
Hi, On 2023-02-27 23:11:53 +0900, Masahiko Sawada wrote: > As far as I know there are not such GUC parameters in the core but > there might be in third-party table AM and index AM extensions. We already reload in a pretty broad range of situations, so I'm not sure there's a lot that could be

Re: Should vacuum process config file reload more often

2023-02-27 Thread Masahiko Sawada
Hi, On Fri, Feb 24, 2023 at 7:08 AM Melanie Plageman wrote: > > Hi, > > Users may wish to speed up long-running vacuum of a large table by > decreasing autovacuum_vacuum_cost_delay/vacuum_cost_delay, however the > config file is only reloaded between tables (for autovacuum) or after > the

Re: Should vacuum process config file reload more often

2023-02-24 Thread Pavel Borisov
Hi, Melanie! On Fri, 24 Feb 2023 at 02:08, Melanie Plageman wrote: > > Hi, > > Users may wish to speed up long-running vacuum of a large table by > decreasing autovacuum_vacuum_cost_delay/vacuum_cost_delay, however the > config file is only reloaded between tables (for autovacuum) or after > the

Should vacuum process config file reload more often

2023-02-23 Thread Melanie Plageman
Hi, Users may wish to speed up long-running vacuum of a large table by decreasing autovacuum_vacuum_cost_delay/vacuum_cost_delay, however the config file is only reloaded between tables (for autovacuum) or after the statement (for explicit vacuum). This has been brought up for autovacuum in [1].