Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-14 Thread Thomas Munro
On Wed, Mar 15, 2023 at 7:54 AM Nathan Bossart wrote: > Here is roughly what I had in mind: > > NOTE: Although the delay is specified in microseconds, older Unixen > and > Windows use periodic kernel ticks to wake up, which might increase the > delay time significantly.

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-14 Thread Nathan Bossart
On Tue, Mar 14, 2023 at 03:38:45PM +1300, Thomas Munro wrote: > On Tue, Mar 14, 2023 at 12:10 PM Nathan Bossart > wrote: >> > * NOTE: although the delay is specified in microseconds, the effective >> > - * resolution is only 1/HZ, or 10 milliseconds, on most Unixen. Expect >> > - * the

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-13 Thread Thomas Munro
On Tue, Mar 14, 2023 at 12:10 PM Nathan Bossart wrote: > > * NOTE: although the delay is specified in microseconds, the effective > > - * resolution is only 1/HZ, or 10 milliseconds, on most Unixen. Expect > > - * the requested delay to be rounded up to the next resolution boundary. > > + *

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-13 Thread Nathan Bossart
> * NOTE: although the delay is specified in microseconds, the effective > - * resolution is only 1/HZ, or 10 milliseconds, on most Unixen. Expect > - * the requested delay to be rounded up to the next resolution boundary. > + * resolution is only 1/HZ on systems that use periodic kernel ticks

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-12 Thread Thomas Munro
On Sat, Mar 11, 2023 at 11:49 AM Tom Lane wrote: > Thomas Munro writes: > > I think this is the minimal back-patchable change. I propose to go > > ahead and do that, and then to kick the ideas about latch API changes > > into a new thread for the next commitfest. > > OK by me, but then again

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-11 Thread Thomas Munro
On Fri, Mar 10, 2023 at 1:05 PM Thomas Munro wrote: > On Fri, Mar 10, 2023 at 11:37 AM Nathan Bossart > wrote: > > On Thu, Mar 09, 2023 at 05:27:08PM -0500, Tom Lane wrote: > > > Is it reasonable to assume that all modern platforms can time > > > millisecond delays accurately? Ten years ago I'd

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-10 Thread Tom Lane
Thomas Munro writes: > I think this is the minimal back-patchable change. I propose to go > ahead and do that, and then to kick the ideas about latch API changes > into a new thread for the next commitfest. OK by me, but then again 4753ef37 wasn't my patch. regards, tom

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-10 Thread Thomas Munro
On Fri, Mar 10, 2023 at 6:58 PM Thomas Munro wrote: > ... Perhaps something like 0004, which also shows the sort > of thing that we might consider back-patching to 14 and 15 (next > revision I'll move that up the front and put it in back-patchable > form). I think this is the minimal

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Thomas Munro
On Fri, Mar 10, 2023 at 2:45 PM Thomas Munro wrote: > I'll go and see about usec latch waits. More soon. Here are some experimental patches along those lines. Seems good locally, but I saw a random failure I don't understand on CI so apparently I need to find a bug; at least this gives an idea

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Thomas Munro
On Fri, Mar 10, 2023 at 2:21 PM Tom Lane wrote: > Thomas Munro writes: > > OK. One idea is to provide a WaitLatchUsec(), which is just some > > cross platform donkeywork that I think I know how to type in, and it > > would have to round up on poll() and Windows builds. Then we could > > either

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Tom Lane
Thomas Munro writes: > OK. One idea is to provide a WaitLatchUsec(), which is just some > cross platform donkeywork that I think I know how to type in, and it > would have to round up on poll() and Windows builds. Then we could > either also provide WaitEventSetResolution() that returns 1000 or

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Thomas Munro
On Fri, Mar 10, 2023 at 1:46 PM Tom Lane wrote: > >> I propose therefore that instead of increasing vacuum_cost_limit, > >> what we ought to be doing is reducing vacuum_cost_delay by a similar > >> factor. And, to provide some daylight for people to reduce it even > >> more, we ought to arrange

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Tom Lane
Thomas Munro writes: > Erm, but maybe I'm just looking at this too myopically. Is there > really any point in letting people set it to 0.5, if it behaves as if > you'd set it to 1 and doubled the cost limit? Isn't it just more > confusing? I haven't read the discussion from when fractional

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Thomas Munro
Erm, but maybe I'm just looking at this too myopically. Is there really any point in letting people set it to 0.5, if it behaves as if you'd set it to 1 and doubled the cost limit? Isn't it just more confusing? I haven't read the discussion from when fractional delays came in, where I imagine

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Thomas Munro
On Fri, Mar 10, 2023 at 11:37 AM Nathan Bossart wrote: > > On Thu, Mar 09, 2023 at 05:27:08PM -0500, Tom Lane wrote: > > Is it reasonable to assume that all modern platforms can time > > millisecond delays accurately? Ten years ago I'd have suggested > > truncating the delay to a multiple of

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Tom Lane
Melanie Plageman writes: > On Thu, Mar 9, 2023 at 5:27 PM Tom Lane wrote: >> A minimalistic fix could be as attached. I'm not sure if it's worth >> making the state variable global so that it can be reset to zero in >> the places where we zero out VacuumCostBalance etc. Also note that >> this

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Melanie Plageman
On Thu, Mar 9, 2023 at 5:27 PM Tom Lane wrote: > > Thomas Munro writes: > > On Fri, Mar 10, 2023 at 11:02 AM Tom Lane wrote: > >> The caf626b2c code would only work well on platforms that have > >> microsecond-based sleep primitives, so it was already not too portable. > > > Also, the previous

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Melanie Plageman
On Thu, Mar 9, 2023 at 5:27 PM Tom Lane wrote: > > Thomas Munro writes: > > On Fri, Mar 10, 2023 at 11:02 AM Tom Lane wrote: > >> The caf626b2c code would only work well on platforms that have > >> microsecond-based sleep primitives, so it was already not too portable. > > > Also, the previous

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Nathan Bossart
On Thu, Mar 09, 2023 at 05:27:08PM -0500, Tom Lane wrote: > Is it reasonable to assume that all modern platforms can time > millisecond delays accurately? Ten years ago I'd have suggested > truncating the delay to a multiple of 10msec and using this logic > to track the remainder, but maybe now

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Tom Lane
Thomas Munro writes: > On Fri, Mar 10, 2023 at 11:02 AM Tom Lane wrote: >> The caf626b2c code would only work well on platforms that have >> microsecond-based sleep primitives, so it was already not too portable. > Also, the previous coding was already b0rked, because pg_usleep() > rounds up to

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Melanie Plageman
On Thu, Mar 9, 2023 at 5:10 PM Thomas Munro wrote: > > On Fri, Mar 10, 2023 at 11:02 AM Tom Lane wrote: > > Thomas Munro writes: > > > On Fri, Mar 10, 2023 at 10:26 AM Melanie Plageman > > > wrote: > > >> I think that 4753ef37e0ed undid the work caf626b2c did to support > > >> sub-millisecond

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Thomas Munro
On Fri, Mar 10, 2023 at 11:02 AM Tom Lane wrote: > Thomas Munro writes: > > On Fri, Mar 10, 2023 at 10:26 AM Melanie Plageman > > wrote: > >> I think that 4753ef37e0ed undid the work caf626b2c did to support > >> sub-millisecond delays for vacuum and autovacuum. > > > Given that some of the

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Stephen Frost
Greetings, * Thomas Munro (thomas.mu...@gmail.com) wrote: > On Fri, Mar 10, 2023 at 10:26 AM Melanie Plageman > wrote: > > I think that 4753ef37e0ed undid the work caf626b2c did to support > > sub-millisecond delays for vacuum and autovacuum. > > > > After 4753ef37e0ed, vacuum_delay_point()'s

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Tom Lane
Thomas Munro writes: > On Fri, Mar 10, 2023 at 10:26 AM Melanie Plageman > wrote: >> I think that 4753ef37e0ed undid the work caf626b2c did to support >> sub-millisecond delays for vacuum and autovacuum. > Given that some of the clunkier underlying kernel primitives have > milliseconds in their

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Thomas Munro
On Fri, Mar 10, 2023 at 10:26 AM Melanie Plageman wrote: > I think that 4753ef37e0ed undid the work caf626b2c did to support > sub-millisecond delays for vacuum and autovacuum. > > After 4753ef37e0ed, vacuum_delay_point()'s local variable msec is a > double which, after being passed to

Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-09 Thread Melanie Plageman
Hi, I think that 4753ef37e0ed undid the work caf626b2c did to support sub-millisecond delays for vacuum and autovacuum. After 4753ef37e0ed, vacuum_delay_point()'s local variable msec is a double which, after being passed to WaitLatch() as timeout, which is a long, ends up being 0, so we don't