Re: [HACKERS] pgbench progress report improvements - split 2
On Tue, Sep 24, 2013 at 08:42:15PM +0200, Fabien COELHO wrote: >> meet all those goals simultaneously with simpler code, can we not? >> >> int64 wait = (int64) (throttle_delay * >> Min(7.0, -log(1 - pg_erand48(thread->random_state; > > If you truncate roughly the multipler, as it is done here with "min", you > necessarily create a bias, my random guess would be a 0.5% under > estimation, but maybe it is more... Thus you would have to compute and > the correcting factor as well. Its computation is a little bit less easy > than with the quantized formula where I just used a simple SUM, and you > have to really do the maths here. So I would keep the simple solution, > but it is fine with me if you do the maths! Ah, true; I guess each approach has different advantages. I've committed your latest version. -- Noah Misch EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgbench progress report improvements - split 2
Hello Noah, meet all those goals simultaneously with simpler code, can we not? int64 wait = (int64) (throttle_delay * Min(7.0, -log(1 - pg_erand48(thread->random_state; If you truncate roughly the multipler, as it is done here with "min", you necessarily create a bias, my random guess would be a 0.5% under estimation, but maybe it is more... Thus you would have to compute and the correcting factor as well. Its computation is a little bit less easy than with the quantized formula where I just used a simple SUM, and you have to really do the maths here. So I would keep the simple solution, but it is fine with me if you do the maths! -- Fabien. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgbench progress report improvements - split 2
On Sun, Sep 22, 2013 at 08:46:55PM +0200, Fabien wrote: > pgbench: reduce and compensate throttling underestimation bias. > > This is a consequence of relying on an integer random generator, > which allow to ensure that delays inserted stay reasonably in > range of the target average delay. > > The bias was about 0.5% with 1000 distinct values. Multiplier added > to compensate the 0.05% bias with 1 distinct integer values, > so there is no bias now. > --- a/contrib/pgbench/pgbench.c > +++ b/contrib/pgbench/pgbench.c > @@ -929,13 +929,17 @@ top: >* that the series of delays will approximate a Poisson > distribution >* centered on the throttle_delay time. > * > - * 1000 implies a 6.9 (-log(1/1000)) to 0.0 (log 1.0) delay > multiplier. > + * 1 implies a 9.2 (-log(1/1)) to 0.0 (log 1) delay > multiplier, > + * and results in a 0.055 % target underestimation bias: > + * > + * SELECT 1.0/AVG(-LN(i/1.0)) FROM generate_series(1,1) > AS i; > + * = 1.00055271703266335474 >* >* If transactions are too slow or a given wait is shorter than >* a transaction, the next transaction will start right away. >*/ > - int64 wait = (int64) > - throttle_delay * -log(getrand(thread, 1, 1000)/1000.0); > + int64 wait = (int64) (throttle_delay * > + 1.00055271703 * -log(getrand(thread, 1, > 1)/1.0)); > > thread->throttle_trigger += wait; > On Sun, Sep 22, 2013 at 10:08:54AM +0200, Fabien COELHO wrote: >> Then I understand why you want to remove the bias, but >> why do you also increase the upper bound? > > Because the bias was significantly larger for 1000 (about 0.5%), so this > also contributed to reduce said bias, and 9.2 times the average target > seems as reasonnable a bound as 6.9. The magnitude of the bias is unimportant if known exactly, as you do here. Smaller quanta do give you more possible distinct sleep durations, which is a nice bonus. What I'm hearing is that you don't especially want to change the upper bound, but changing it was an acceptable side effect. However, we can meet all those goals simultaneously with simpler code, can we not? int64 wait = (int64) (throttle_delay * Min(7.0, -log(1 - pg_erand48(thread->random_state; -- Noah Misch EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgbench progress report improvements - split 2
Split 2 of the initial submission pgbench: reduce and compensate throttling underestimation bias. This is a consequence of relying on an integer random generator, which allow to ensure that delays inserted stay reasonably in range of the target average delay. The bias was about 0.5% with 1000 distinct values. Multiplier added to compensate the 0.05% bias with 1 distinct integer values, so there is no bias now. -- Fabien.diff --git a/contrib/pgbench/pgbench.c b/contrib/pgbench/pgbench.c index ad8e272..572573a 100644 --- a/contrib/pgbench/pgbench.c +++ b/contrib/pgbench/pgbench.c @@ -929,13 +929,17 @@ top: * that the series of delays will approximate a Poisson distribution * centered on the throttle_delay time. * - * 1000 implies a 6.9 (-log(1/1000)) to 0.0 (log 1.0) delay multiplier. + * 1 implies a 9.2 (-log(1/1)) to 0.0 (log 1) delay multiplier, + * and results in a 0.055 % target underestimation bias: + * + * SELECT 1.0/AVG(-LN(i/1.0)) FROM generate_series(1,1) AS i; + * = 1.00055271703266335474 * * If transactions are too slow or a given wait is shorter than * a transaction, the next transaction will start right away. */ - int64 wait = (int64) - throttle_delay * -log(getrand(thread, 1, 1000)/1000.0); + int64 wait = (int64) (throttle_delay * + 1.00055271703 * -log(getrand(thread, 1, 1)/1.0)); thread->throttle_trigger += wait; -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
