Re: Remove size limitations of vacuums dead_tuples array

2019-11-29 Thread Michael Paquier
On Fri, Oct 11, 2019 at 04:49:11PM +0300, Ants Aasma wrote: > Thanks for the pointer, wow that's a long thread. For some reason it did > not consider lifting the INT_MAX tuples/12GB limitation. I'll see if I can > pick up where that thread left off and push it along. Hmm. Okay.. Then I have mark

Re: Remove size limitations of vacuums dead_tuples array

2019-10-11 Thread Ants Aasma
On Thu, 10 Oct 2019 at 17:05, Tomas Vondra wrote: > There already was a attempt to make this improvement, see [1]. There was > a fairly long discussion about how to best do that (using other data > structure, not just a simple array). It kinda died about a year ago, but > I suppose there's a lot

Re: Remove size limitations of vacuums dead_tuples array

2019-10-10 Thread Tomas Vondra
On Wed, Oct 09, 2019 at 03:58:11PM +0300, Ants Aasma wrote: When dealing with a case where a 2TB table had 3 billion dead tuples I discovered that vacuum currently can't make use of more than 1GB of maintenance_work_mem - 179M tuples. This caused excessive amounts of index scanning even though th

Remove size limitations of vacuums dead_tuples array

2019-10-09 Thread Ants Aasma
When dealing with a case where a 2TB table had 3 billion dead tuples I discovered that vacuum currently can't make use of more than 1GB of maintenance_work_mem - 179M tuples. This caused excessive amounts of index scanning even though there was plenty of memory available. I didn't see any good rea