Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

2022-08-18 Thread Peter Geoghegan
On Thu, Aug 11, 2022 at 1:48 AM Matthias van de Meent wrote: > I think I understand your reasoning, but I don't agree with the > conclusion. The attached patch 0002 does fix that skew too, at what I > consider negligible cost. 0001 is your patch with a new version > number. Your patch added allow

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

2022-08-11 Thread Matthias van de Meent
On Mon, 8 Aug 2022 at 18:48, Peter Geoghegan wrote: > > On Mon, Aug 8, 2022 at 9:17 AM Matthias van de Meent > wrote: > > Because if a subset of the pages of a relation contains more tuples > > than your current total expected tuples in the table, you should > > update your expectations regardles

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

2022-08-08 Thread Peter Geoghegan
On Mon, Aug 8, 2022 at 9:17 AM Matthias van de Meent wrote: > Because if a subset of the pages of a relation contains more tuples > than your current total expected tuples in the table, you should > update your expectations regardless of which blocks or which number of > blocks you've scanned - th

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

2022-08-08 Thread Matthias van de Meent
On Mon, 8 Aug 2022 at 17:49, Peter Geoghegan wrote: > > On Mon, Aug 8, 2022 at 8:33 AM Matthias van de Meent > wrote: > > For example, if currently the measured 2% of the pages contains more > > than 100% of the previous count of tuples, or with your patch the last > > page contains more than 100

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

2022-08-08 Thread Peter Geoghegan
On Mon, Aug 8, 2022 at 8:33 AM Matthias van de Meent wrote: > For example, if currently the measured 2% of the pages contains more > than 100% of the previous count of tuples, or with your patch the last > page contains more than 100% of the previous count of the tuples, that > new count is ignore

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

2022-08-08 Thread Matthias van de Meent
On Mon, 8 Aug 2022 at 17:26, Peter Geoghegan wrote: > > On Mon, Aug 8, 2022 at 8:14 AM Matthias van de Meent > wrote: > > I do not have intimate knowledge of this code, but shouldn't we also > > add some sefety guarantees like the following in these blocks? Right > > now, we'll keep underestimati

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

2022-08-08 Thread Peter Geoghegan
On Mon, Aug 8, 2022 at 8:14 AM Matthias van de Meent wrote: > I do not have intimate knowledge of this code, but shouldn't we also > add some sefety guarantees like the following in these blocks? Right > now, we'll keep underestimating the table size even when we know that > the count is incorrect

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

2022-08-08 Thread Matthias van de Meent
On Mon, 8 Aug 2022 at 16:52, Peter Geoghegan wrote: > > On Fri, Aug 5, 2022 at 5:39 PM Peter Geoghegan wrote: > > Attached patch fixes closes the remaining gap. With the patch applied, > > the second and subsequent vacuum verbose operations from the test case > > will show that reltuples is still

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

2022-08-08 Thread Peter Geoghegan
On Fri, Aug 5, 2022 at 5:39 PM Peter Geoghegan wrote: > Attached patch fixes closes the remaining gap. With the patch applied, > the second and subsequent vacuum verbose operations from the test case > will show that reltuples is still 1 (it won't ever change). The > patch just extends an old

Remaining case where reltuples can become distorted across multiple VACUUM operations

2022-08-05 Thread Peter Geoghegan
My bugfix commit 74388a1a (which was pushed back in February) added heuristics to VACUUM's reltuples calculation/estimate. This prevented VACUUM from distorting our estimate of reltuples over time, across successive VACUUM operations run against the same table. The problem was that VACUUM could sca