Re: a verbose option for autovacuum

2021-03-22 Thread Masahiko Sawada
On Tue, Mar 23, 2021 at 1:31 PM Michael Paquier wrote: > > On Mon, Mar 22, 2021 at 12:17:37PM +0900, Masahiko Sawada wrote: > > I've updated the patch. I saved the index names at the beginning of > > heap_vacuum_rel() for autovacuum logging, and add indstats and > > nindexes to LVRelStats. Some

Re: a verbose option for autovacuum

2021-03-22 Thread Michael Paquier
On Mon, Mar 22, 2021 at 12:17:37PM +0900, Masahiko Sawada wrote: > I've updated the patch. I saved the index names at the beginning of > heap_vacuum_rel() for autovacuum logging, and add indstats and > nindexes to LVRelStats. Some functions still have nindexes as a > function argument but it seems

Re: a verbose option for autovacuum

2021-03-21 Thread Masahiko Sawada
On Sat, Mar 20, 2021 at 3:40 PM Michael Paquier wrote: > > On Sat, Mar 20, 2021 at 01:06:51PM +0900, Masahiko Sawada wrote: > > It's not bad but it seems redundant a bit to me. We pass the idx in > > spite of passing also Irel[idx] and &(vacrelstats->indstats[idx]). I > > think your first idea

Re: a verbose option for autovacuum

2021-03-20 Thread Michael Paquier
On Sat, Mar 20, 2021 at 01:06:51PM +0900, Masahiko Sawada wrote: > It's not bad but it seems redundant a bit to me. We pass the idx in > spite of passing also Irel[idx] and &(vacrelstats->indstats[idx]). I > think your first idea that is done in v4 patch (saving index names at > the beginning of

Re: a verbose option for autovacuum

2021-03-19 Thread Masahiko Sawada
On Fri, Mar 19, 2021 at 3:14 PM Michael Paquier wrote: > > On Thu, Mar 18, 2021 at 11:30:46PM +0900, Masahiko Sawada wrote: > > Sorry, I attached the wrong version patch. So attached the right one. > > Thanks. I have been hacking aon that, and I think that we could do > more in terms of

Re: a verbose option for autovacuum

2021-03-19 Thread Michael Paquier
On Thu, Mar 18, 2021 at 11:30:46PM +0900, Masahiko Sawada wrote: > Sorry, I attached the wrong version patch. So attached the right one. Thanks. I have been hacking aon that, and I think that we could do more in terms of integration of the index stats into LVRelStats to help with debugging

Re: a verbose option for autovacuum

2021-03-18 Thread Peter Geoghegan
On Thu, Mar 18, 2021 at 2:08 PM Michael Paquier wrote: > Yes, I was waiting for Sawada-san to send an updated version, which he > just did. Great. This really seems worth having. I was hoping that somebody else could pick this one up. -- Peter Geoghegan

Re: a verbose option for autovacuum

2021-03-18 Thread Michael Paquier
On Thu, Mar 18, 2021 at 09:38:05AM -0700, Peter Geoghegan wrote: > Were you going to take care of this, Michael? Yes, I was waiting for Sawada-san to send an updated version, which he just did. -- Michael signature.asc Description: PGP signature

Re: a verbose option for autovacuum

2021-03-18 Thread Peter Geoghegan
On Thu, Mar 18, 2021 at 5:14 AM Masahiko Sawada wrote: > Okay, I've updated the patch accordingly. If we add > IndexBulkDeleteResult to LVRelStats I think we can remove > IndexBulkDeleteResult function argument also from some other functions > such as lazy_parallel_vacuum_indexes() and

Re: a verbose option for autovacuum

2021-03-18 Thread Masahiko Sawada
On Thu, Mar 18, 2021 at 9:13 PM Masahiko Sawada wrote: > > On Thu, Mar 18, 2021 at 3:41 PM Michael Paquier wrote: > > > > On Wed, Mar 17, 2021 at 08:50:26AM -0300, Euler Taveira wrote: > > > Since commit 5f8727f5a6, this patch doesn't apply anymore. Fortunately, > > > it is > > > just a small

Re: a verbose option for autovacuum

2021-03-18 Thread Masahiko Sawada
On Thu, Mar 18, 2021 at 3:41 PM Michael Paquier wrote: > > On Wed, Mar 17, 2021 at 08:50:26AM -0300, Euler Taveira wrote: > > Since commit 5f8727f5a6, this patch doesn't apply anymore. Fortunately, it > > is > > just a small hunk. I reviewed this patch and it looks good to me. There is > > just

Re: a verbose option for autovacuum

2021-03-18 Thread Michael Paquier
On Wed, Mar 17, 2021 at 08:50:26AM -0300, Euler Taveira wrote: > Since commit 5f8727f5a6, this patch doesn't apply anymore. Fortunately, it is > just a small hunk. I reviewed this patch and it looks good to me. There is > just > a small issue (double space after 'if') that I fixed in the attached

Re: a verbose option for autovacuum

2021-03-17 Thread Euler Taveira
On Wed, Mar 10, 2021, at 12:46 AM, Masahiko Sawada wrote: > Attached a patch. I've slightly modified the format for consistency > with heap statistics. Since commit 5f8727f5a6, this patch doesn't apply anymore. Fortunately, it is just a small hunk. I reviewed this patch and it looks good to me.

Re: a verbose option for autovacuum

2021-03-09 Thread Masahiko Sawada
On Tue, Mar 9, 2021 at 12:58 AM Euler Taveira wrote: > > On Mon, Mar 8, 2021, at 2:32 AM, Masahiko Sawada wrote: > > * Proposed idea > LOG: automatic vacuum of table "postgres.public.test": index scans: 1 > pages: 0 removed, 443 remain, 0 skipped due to pins, 0 skipped frozen > tuples: 1000

Re: a verbose option for autovacuum

2021-03-08 Thread Euler Taveira
On Mon, Mar 8, 2021, at 2:32 AM, Masahiko Sawada wrote: > * Proposed idea > LOG: automatic vacuum of table "postgres.public.test": index scans: 1 > pages: 0 removed, 443 remain, 0 skipped due to pins, 0 skipped frozen > tuples: 1000 removed, 99000 remain, 0 are dead but not yet removable, >

Re: a verbose option for autovacuum

2021-03-07 Thread Masahiko Sawada
On Tue, Feb 2, 2021 at 10:51 AM Euler Taveira wrote: > > On Fri, Jan 29, 2021, at 4:35 AM, Masahiko Sawada wrote: > > I agree to have autovacuum log more information, especially index > vacuums. Currently, the log related to index vacuum is only the number > of index scans. I think it would be

Re: a verbose option for autovacuum

2021-03-07 Thread Masahiko Sawada
On Tue, Feb 2, 2021 at 9:59 AM Tommy Li wrote: > > Hi Masahiko > > > If we set > > it per-table basis, it’s useful when the user already knows which > > tables are likely to take a long time for autovacuum > > I would assume that's the default case, most apps I've seen are designed > around a

Re: a verbose option for autovacuum

2021-02-01 Thread Euler Taveira
On Fri, Jan 29, 2021, at 4:35 AM, Masahiko Sawada wrote: > I agree to have autovacuum log more information, especially index > vacuums. Currently, the log related to index vacuum is only the number > of index scans. I think it would be helpful if the log has more > details about each index vacuum.

Re: a verbose option for autovacuum

2021-02-01 Thread Tommy Li
Hi Masahiko > If we set > it per-table basis, it’s useful when the user already knows which > tables are likely to take a long time for autovacuum I would assume that's the default case, most apps I've seen are designed around a small number of large tables that take up most of the maintenance

Re: a verbose option for autovacuum

2021-01-28 Thread Masahiko Sawada
On Tue, Jan 26, 2021 at 2:46 AM Tommy Li wrote: > > Hi Stephen > > > ... can set vacuum options on a table level which autovacuum should respect, > > such as vacuum_index_cleanup and vacuum_truncate. For skip locked, > > autovacuum already will automatically release it's attempt to acquire a > >

Re: a verbose option for autovacuum

2021-01-25 Thread Tommy Li
Hi Stephen > ... can set vacuum options on a table level which autovacuum should respect, > such as vacuum_index_cleanup and vacuum_truncate. For skip locked, > autovacuum already will automatically release it's attempt to acquire a > lock if someone backs up behind it for too long. This is

Re: a verbose option for autovacuum

2021-01-23 Thread Stephen Frost
Greetings, On Fri, Jan 22, 2021 at 2:33 PM Tom Lane wrote: > Tommy Li writes: > > Additionally, is there any interest in exposing more vacuum options to be > > run by autovac? Right now it runs FREEZE and ANALYZE, which leaves the > > VERBOSE, SKIP_LOCKED, INDEX_CLEANUP, and TRUNCATE

Re: a verbose option for autovacuum

2021-01-22 Thread Tommy Li
Hey Tom > Seems like that would very soon feel like log spam. What would be the > use case for having this on? If you want one-off results you could > run VACUUM manually. In my case I have a fairly large, fairly frequently updated table with a large number of indexes where autovacuum's

Re: a verbose option for autovacuum

2021-01-22 Thread Tom Lane
Tommy Li writes: > I was surprised to see that there's no way to get `VACUUM VERBOSE`-like > output from autovacuum. Is there any interest in enabling this? Seems like that would very soon feel like log spam. What would be the use case for having this on? If you want one-off results you could

a verbose option for autovacuum

2021-01-22 Thread Tommy Li
Hi all I was surprised to see that there's no way to get `VACUUM VERBOSE`-like output from autovacuum. Is there any interest in enabling this? Additionally, is there any interest in exposing more vacuum options to be run by autovac? Right now it runs FREEZE and ANALYZE, which leaves the VERBOSE,