Re: Automatically sizing the IO worker pool

2025-07-11 Thread Thomas Munro
On Wed, May 28, 2025 at 5:55 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > I probably had to start with a statement that I find the current approach > reasonable, and I'm only curious if there is more to get out of it. I haven't > benchmarked the patch yet (plan getting to it when I'll get ba

Re: Automatically sizing the IO worker pool

2025-05-27 Thread Dmitry Dolgov
On Mon, May 26, 2025, 8:01 AM Thomas Munro wrote: > But ... I'm not even sure if we can say that our > I/O arrivals have a Poisson distribution, since they are not all > independent. > Yeah, a good point, one have to be careful with assumptions about distribution -- from what I've read many proc

Re: Automatically sizing the IO worker pool

2025-05-26 Thread Thomas Munro
BTW I would like to push 0001 and 0002 to master/18. They are are not behaviour changes, they just fix up a bunch of inconsistent (0001) and misleading (0002) variable naming and comments to reflect reality (in AIO v1 the postmaster used to assign those I/O worker numbers, now the postmaster has i

Re: Automatically sizing the IO worker pool

2025-05-26 Thread Thomas Munro
On Sun, May 25, 2025 at 7:20 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Sun, Apr 13, 2025 at 04:59:54AM GMT, Thomas Munro wrote: > > It's hard to know how to set io_workers=3. If it's too small, > > io_method=worker's small submission queue overflows and it silently > > falls back to s

Re: Automatically sizing the IO worker pool

2025-05-25 Thread wenhui qiu
HI > On Sun, Apr 13, 2025 at 04:59:54AM GMT, Thomas Munro wrote: > It's hard to know how to set io_workers=3. If it's too small, > io_method=worker's small submission queue overflows and it silently > falls back to synchronous IO. If it's too high, it generates a lot of > pointless wakeups and sc

Re: Automatically sizing the IO worker pool

2025-05-24 Thread Dmitry Dolgov
> On Sun, Apr 13, 2025 at 04:59:54AM GMT, Thomas Munro wrote: > It's hard to know how to set io_workers=3. If it's too small, > io_method=worker's small submission queue overflows and it silently > falls back to synchronous IO. If it's too high, it generates a lot of > pointless wakeups and sched

Re: Automatically sizing the IO worker pool

2025-04-13 Thread Thomas Munro
On Mon, Apr 14, 2025 at 5:45 AM Jose Luis Tallon wrote: > On 12/4/25 18:59, Thomas Munro wrote: > > It's hard to know how to set io_workers=3. > > Hmmm enable the below behaviour if "io_workers=auto" (default) ? Why not just delete io_workers? If you really want a fixed number, you can set i

Re: Automatically sizing the IO worker pool

2025-04-13 Thread Jose Luis Tallon
On 12/4/25 18:59, Thomas Munro wrote: It's hard to know how to set io_workers=3. Hmmm enable the below behaviour if "io_workers=auto" (default) ? Sometimes being able to set this kind of parameters manually helps tremendously with specific workloads... :S [snip] Here's a patch to repla

Automatically sizing the IO worker pool

2025-04-12 Thread Thomas Munro
It's hard to know how to set io_workers=3. If it's too small, io_method=worker's small submission queue overflows and it silently falls back to synchronous IO. If it's too high, it generates a lot of pointless wakeups and scheduling overhead, which might be considered an independent problem or no