On Mon, 21 Jul 2025 at 06:27, Greg Hennessy <greg.henne...@gmail.com> wrote:
> test=# show parallel_worker_algorithm ;
> parallel_worker_algorithm
> ---------------------------
> log3
> (1 row)
I don't think having a GUC which allows exactly two settings is
anywhere near as flexible as you could make this. You're complaining
that the current calculation isn't ideal for you and are proposing a
new one which seemingly is more to your liking, but who's to say that
someone else would find either useful? With this method, if someone
else comes along complaining, do we give them what they want by adding
a new enum? That does not sound great :(
One thought I had was that you could adjust the "divide by 3" to be
"divide by <some_new_GUC>" and make that calculation floating point.
For people who want really aggressive scaling, that could set it to
something a little over 1.0. The problem with that is numbers really
close to 1.0 would cause that loop to iterate an excessive number of
times and that would be bad for performance.
On rethink, maybe you can use the log() function to get something
close to what's being generated today. The fact that the
heap_parallel_threshold is taken into account makes that a little
tricky as it's not a simple log3. I experimented with the attached .c
file. When I set the GUC to 3.0, it's pretty close to what we get
today. If I drop it to 1.01, I get something closer to your version.
Do you want to play around with the code in the attached .c file and
add a GUC like the parallel_worker_scaling_logarithm I have in the .c
file? Feel free to tweak the algorithm. I won't pretend it's the best
it can be. I didn't spend much time on it. There might be a better way
to get numbers closer to what we have today that takes into account
the (heap|index)_parallel_threshold and doesn't need a while loop.
David
#include <stdio.h>
#include <limits.h>
#include <math.h>
#define Max(x, y) ((x) > (y) ? (x) : (y))
/* GUC */
double parallel_worker_scaling_logarithm = 3.0;
int masters_algoriothm(unsigned int heap_pages)
{
int heap_parallel_workers = 1;
int heap_parallel_threshold = 128;
while (heap_pages >= (unsigned int) (heap_parallel_threshold * 3))
{
heap_parallel_workers++;
heap_parallel_threshold *= 3;
if (heap_parallel_threshold > INT_MAX / 3)
break; /* avoid overflow */
}
return heap_parallel_workers;
}
int workers_via_log(unsigned int heap_pages)
{
int heap_parallel_threshold = 128;
double log10scale;
int parallel_workers;
if (heap_pages <= heap_parallel_threshold)
return 1;
log10scale = log(parallel_worker_scaling_logarithm);
parallel_workers = (int) ceil(log(heap_pages - heap_parallel_threshold)
/ log10scale) - floor(log(heap_parallel_threshold) / log10scale);
return Max(parallel_workers, 1);
}
int main(void)
{
for (unsigned long long pages = 128; pages <= 0xffffffff; pages *= 1.1)
{
printf("Table Size = %llu MB pages = %u old workers = %d, new
workers = %d\n",
((unsigned long long) pages) * 8192 / 1024 / 1024,
pages, masters_algoriothm(pages), workers_via_log(pages));
}
return 0;
}
