On Tue, Oct 01, 2013 at 02:50:29PM +0200, Fabien COELHO wrote: > I do not think that there is a clean and simple way to take the > start/stop period into account when computing global performances of > a run. The TPC-C benchmark tells to ignore the warmup/closure > period, whatever they are, and only perform measures within the > steady state. However the full graph must be provided when the bench > is provided.
That makes sense to me. "pgbench --progress" and "pgbench --log --aggregate-interval" are good tools for excluding non-steady-state periods. > About better measures: If I could rely on having threads, I would > simply synchronise the threads at the beginning so that they > actually start after they are all created, and one thread would > decide when to stop and set a shared volatile variable to stop all > transactions more or less at once. In this case, the thread start > time would be taken just after the synchronization, and maybe only > by thread 0 would be enough. > > Note that this is pretty standard stuff with threads, ISTM that it > would solve most of the issues, *but* this is not possible with the > "thread fork emulation" implemented by pgbench, which really means > no threads at all. You could do those same things in the fork emulation mode using anonymous shared memory, like we do in the server. That would permit removing the current "#ifdef PTHREAD_FORK_EMULATION" wart, too. For the time being, I propose the attached comment patch. -- Noah Misch EnterpriseDB http://www.enterprisedb.com
diff --git a/contrib/pgbench/pgbench.c b/contrib/pgbench/pgbench.c index 926246e..816400f 100644 --- a/contrib/pgbench/pgbench.c +++ b/contrib/pgbench/pgbench.c @@ -2896,7 +2896,16 @@ main(int argc, char **argv) } disconnect_all(state, nclients); - /* get end time */ + /* + * XXX We compute results as though every client of every thread started + * and finished at the same time. That model can diverge noticeably from + * reality for a short benchmark run involving relatively many threads. + * The first thread may process notably many transactions before the last + * thread begins. Improving the model alone would bring limited benefit, + * because performance during those periods of partial thread count can + * easily exceed steady state performance. This is one of the many ways + * short runs convey deceptive performance figures. + */ INSTR_TIME_SET_CURRENT(total_time); INSTR_TIME_SUBTRACT(total_time, start_time); printResults(ttype, total_xacts, nclients, threads, nthreads,
-- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers