Oops, as usual I forgot something...
This v2 removes old stats code that was put in comment and simplify the
logic when counting lag times, as they are now taken into account at the
end of the transaction instead of at the beginning.
This patch adds per-script statistics & other improvements to pgbench
Rationale: Josh asked for the per-script stats:-)
Some restructuring is done so that all stats (-l --aggregate-interval
--progress --per-script-stats, latency & lag...) share the same structures
and functions to accumulate data. This limits a lot the growth of pgbench
from this patch (+17 lines).
In passing, remove the distinction between internal and external scripts.
Pgbench just execute scripts, some of them may be internal...
As a side effect, all scripts can be accumulated "pgbench -B -N -S -f ..."
would execute 4 scripts, 3 of which internal (tpc-b, simple-update,
select-only and another externally supplied one).
Also add a weight option to change the probability of choosing some scripts
when several are available.
Hmmm... Not sure that the --per-script-stats option is really useful. The
stats could always be shown when several scripts are executed?
sh> ./pgbench -T 3 -B -N -w 2 -S -w 7 --per-script-stats
starting vacuum...end.
transaction type: multiple scripts
scaling factor: 1
query mode: simple
number of clients: 1
number of threads: 1
duration: 3 s
number of transactions actually processed: 3192
latency average: 0.940 ms
tps = 1063.756045 (including connections establishing)
tps = 1065.412737 (excluding connections establishing)
SQL script 0: <builtin: TPC-B (sort of)>
- weight is 1
- 297 transactions (tps = 98.977301)
- latency average = 3.001 ms
- latency stddev = 1.320 ms
SQL script 1: <builtin: simple update>
- weight is 2
- 621 transactions (tps = 206.952539)
- latency average = 2.506 ms
- latency stddev = 1.194 ms
SQL script 2: <builtin: select only>
- weight is 7
- 2274 transactions (tps = 757.826205)
- latency average = 0.236 ms
- latency stddev = 0.083 ms
--
Fabien.
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index 2517a3a..eb571a8 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -261,6 +261,18 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
benchmarking arguments:
<variablelist>
+ <varlistentry>
+ <term><option>-B</option></term>
+ <term><option>--tpc-b</option></term>
+ <listitem>
+ <para>
+ Built-in TPC-B like test which updates three tables and performs
+ an insert on a fourth. See below for details.
+ This is the default if no bench is explicitely specified.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-c</option> <replaceable>clients</></term>
@@ -313,8 +325,7 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
<para>
Read transaction script from <replaceable>filename</>.
See below for details.
- <option>-N</option>, <option>-S</option>, and <option>-f</option>
- are mutually exclusive.
+ Multiple <option>-f</> options are allowed.
</para>
</listitem>
</varlistentry>
@@ -404,10 +415,10 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
<term><option>--skip-some-updates</option></term>
<listitem>
<para>
- Do not update <structname>pgbench_tellers</> and
- <structname>pgbench_branches</>.
- This will avoid update contention on these tables, but
- it makes the test case even less like TPC-B.
+ Built-in test which updates only one table compared to <option>-B</>.
+ This avoids update contention on <structname>pgbench_tellers</>
+ and <structname>pgbench_branches</>, but it makes the test case
+ even less like TPC-B.
</para>
</listitem>
</varlistentry>
@@ -499,9 +510,9 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
Report the specified scale factor in <application>pgbench</>'s
output. With the built-in tests, this is not necessary; the
correct scale factor will be detected by counting the number of
- rows in the <structname>pgbench_branches</> table. However, when testing
- custom benchmarks (<option>-f</> option), the scale factor
- will be reported as 1 unless this option is used.
+ rows in the <structname>pgbench_branches</> table.
+ However, when testing only custom benchmarks (<option>-f</> option),
+ the scale factor will be reported as 1 unless this option is used.
</para>
</listitem>
</varlistentry>
@@ -511,7 +522,7 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
<term><option>--select-only</option></term>
<listitem>
<para>
- Perform select-only transactions instead of TPC-B-like test.
+ Built-in test with select-only transactions.
</para>
</listitem>
</varlistentry>
@@ -552,6 +563,20 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
</varlistentry>
<varlistentry>
+ <term><option>-w</option></term>
+ <term><option>--weight</option></term>
+ <listitem>
+ <para>
+ Set the integer weight for the previous test script
+ (<option>-B</>, <option>-N</>, <option>-S</> or <option>-f</>).
+ When several test scripts are used, the relative probability of
+ drawing each test is proportional to these weights.
+ The default weight is <literal>1</>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>--aggregate-interval=<replaceable>seconds</></option></term>
<listitem>
<para>
@@ -567,6 +592,16 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
</varlistentry>
<varlistentry>
+ <term><option>--per-script-stats</option></term>
+ <listitem>
+ <para>
+ Report some statistics per script run by pgbench.
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+ <varlistentry>
<term><option>--sampling-rate=<replaceable>rate</></option></term>
<listitem>
<para>
@@ -661,7 +696,17 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
<title>What is the <quote>Transaction</> Actually Performed in pgbench?</title>
<para>
- The default transaction script issues seven commands per transaction:
+ Pgbench executes test scripts chosen randomly from a specified list.
+ They include built-in scripts (<option>-B</>, <option>-N</> and
+ <option>-S</>) and custom scripts provided by the user with <option>-f</>.
+ Each test may be given a relative weight (<option>-w</>)
+ specified <emphasis>after</> the test about which it applies so as
+ to change its draw probability.
+ </para>
+
+ <para>
+ The default transaction script (also invoked with <option>-B</>)
+ issues seven commands per transaction:
</para>
<orderedlist>
@@ -689,10 +734,7 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
benchmark scenarios by replacing the default transaction script
(described above) with a transaction script read from a file
(<option>-f</option> option). In this case a <quote>transaction</>
- counts as one execution of a script file. You can even specify
- multiple scripts (multiple <option>-f</option> options), in which
- case a random one of the scripts is chosen each time a client session
- starts a new transaction.
+ counts as one execution of a script file.
</para>
<para>
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index e839fa3..bbf483e 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -162,11 +162,10 @@ bool use_log; /* log transaction latencies to a file */
bool use_quiet; /* quiet logging onto stderr */
int agg_interval; /* log aggregates instead of individual
* transactions */
+bool per_script_stats = false; /* whether to collect stats per script */
int progress = 0; /* thread progress report every this seconds */
-int progress_nclients = 0; /* number of clients for progress
- * report */
-int progress_nthreads = 0; /* number of threads for progress
- * report */
+int nclients = 1; /* number of clients */
+int nthreads = 1; /* number of threads */
bool is_connect; /* establish connection for each transaction */
bool is_latencies; /* report per-command latencies */
int main_pid; /* main process id used in log filename */
@@ -186,13 +185,39 @@ typedef struct
char *value; /* its value */
} Variable;
-#define MAX_FILES 128 /* max number of SQL script files allowed */
+#define MAX_SCRIPTS 128 /* max number of SQL script files allowed */
#define SHELL_COMMAND_SIZE 256 /* maximum size allowed for shell command */
/*
- * structures used in custom query mode
+ * simple data structure to keep stats about something.
+ * probably the first value should be kept and used as an offset for
+ * better numerical stability...
+ */
+typedef struct
+{
+ int64_t count; /* how many values where encountered */
+ double min; /* the minimum seen */
+ double max; /* the maximum seen */
+ double sum; /* sum of values */
+ double sum2; /* sum of squared values */
+} SimpleStats;
+
+/* data structure to hold various statistics.
+ * it is used for interval statistics as well as file statistics.
*/
+typedef struct
+{
+ long start_time; /* when the interval starts, for aggregates */
+ int64 cnt; /* number of transactions */
+ int64 skipped; /* number of transactions skipped under --rate
+ * and --latency-limit */
+ SimpleStats latency;
+ SimpleStats lag;
+} StatsData;
+/*
+ * structures used in custom query mode
+ */
typedef struct
{
PGconn *con; /* connection handle to DB */
@@ -200,22 +225,20 @@ typedef struct
int state; /* state No. */
int listen; /* 0 indicates that an async query has been
* sent */
- int sleeping; /* 1 indicates that the client is napping */
+ bool is_throttled; /* whether transaction throttling is done */
+ bool sleeping; /* whether the client is napping */
bool throttling; /* whether nap is for throttling */
Variable *variables; /* array of variable definitions */
int nvariables;
int64 txn_scheduled; /* scheduled start time of transaction (usec) */
instr_time txn_begin; /* used for measuring schedule lag times */
instr_time stmt_begin; /* used for measuring statement latencies */
- bool is_throttled; /* whether transaction throttling is done */
- int use_file; /* index in sql_files for this client */
- bool prepared[MAX_FILES];
+ int use_file; /* index in sql_scripts for this client */
+ bool prepared[MAX_SCRIPTS]; /* whether client prepared the script */
/* per client collected stats */
- int cnt; /* xacts count */
+ int64 cnt; /* transaction count */
int ecnt; /* error count */
- int64 txn_latencies; /* cumulated latencies */
- int64 txn_sqlats; /* cumulated square latencies */
} CState;
/*
@@ -227,19 +250,16 @@ typedef struct
pthread_t thread; /* thread handle */
CState *state; /* array of CState */
int nstate; /* length of state[] */
- instr_time start_time; /* thread start time */
- instr_time *exec_elapsed; /* time spent executing cmds (per Command) */
- int *exec_count; /* number of cmd executions (per Command) */
- unsigned short random_state[3]; /* separate randomness for each thread */
- int64 throttle_trigger; /* previous/next throttling (us) */
+ unsigned short random_state[3]; /* separate randomness for each thread */
+ int64 throttle_trigger; /* previous/next throttling (us) */
/* per thread collected stats */
+ instr_time start_time; /* thread start time */
instr_time conn_time;
- int64 throttle_lag; /* total transaction lag behind throttling */
- int64 throttle_lag_max; /* max transaction lag */
- int64 throttle_latency_skipped; /* lagging transactions
- * skipped */
- int64 latency_late; /* late transactions */
+ instr_time *exec_elapsed; /* time spent executing cmds (per Command) */
+ int *exec_count; /* number of cmd executions (per Command) */
+ StatsData stats;
+ int64 latency_late; /* executed but late transactions */
} TState;
#define INVALID_THREAD ((pthread_t) 0)
@@ -273,33 +293,26 @@ typedef struct
PgBenchExpr *expr; /* parsed expression */
} Command;
+/* SQL script to be executed, either a file or an internal script
+ */
typedef struct
{
-
- long start_time; /* when does the interval start */
- int cnt; /* number of transactions */
- int skipped; /* number of transactions skipped under --rate
- * and --latency-limit */
-
- double min_latency; /* min/max latencies */
- double max_latency;
- double sum_latency; /* sum(latency), sum(latency^2) - for
- * estimates */
- double sum2_latency;
-
- double min_lag;
- double max_lag;
- double sum_lag; /* sum(lag) */
- double sum2_lag; /* sum(lag*lag) */
-} AggVals;
-
-static Command **sql_files[MAX_FILES]; /* SQL script files */
-static int num_files; /* number of script files */
+ const char *name;
+ int weight;
+ Command **commands;
+ StatsData stats;
+} SQLScript;
+
+static SQLScript sql_script[MAX_SCRIPTS];
+static int num_scripts; /* number of script in sql_script[] */
static int num_commands = 0; /* total number of Command structs */
+static int total_weight = 0;
+
static int debug = 0; /* debug flag */
-/* default scenario */
-static char *tpc_b = {
+/* -B case, default scenario */
+static const char *b_bench = "<builtin: TPC-B (sort of)>";
+static const char *tpc_b = {
"\\set nbranches " CppAsString2(nbranches) " * :scale\n"
"\\set ntellers " CppAsString2(ntellers) " * :scale\n"
"\\set naccounts " CppAsString2(naccounts) " * :scale\n"
@@ -317,7 +330,8 @@ static char *tpc_b = {
};
/* -N case */
-static char *simple_update = {
+static const char *n_bench = "<builtin: simple update>";
+static const char *simple_update = {
"\\set nbranches " CppAsString2(nbranches) " * :scale\n"
"\\set ntellers " CppAsString2(ntellers) " * :scale\n"
"\\set naccounts " CppAsString2(naccounts) " * :scale\n"
@@ -333,7 +347,8 @@ static char *simple_update = {
};
/* -S case */
-static char *select_only = {
+static const char *s_bench = "<builtin: select only>";
+static const char *select_only = {
"\\set naccounts " CppAsString2(naccounts) " * :scale\n"
"\\setrandom aid 1 :naccounts\n"
"SELECT abalance FROM pgbench_accounts WHERE aid = :aid;\n"
@@ -343,8 +358,7 @@ static char *select_only = {
static void setalarm(int seconds);
static void *threadRun(void *arg);
-static void doLog(TState *thread, CState *st, FILE *logfile, instr_time *now,
- AggVals *agg, bool skipped);
+static void doTxStats(TState*, CState*, instr_time*, bool, FILE*, StatsData*);
static void
usage(void)
@@ -364,11 +378,12 @@ usage(void)
" --tablespace=TABLESPACE create tables in the specified tablespace\n"
" --unlogged-tables create tables as unlogged tables\n"
"\nBenchmarking options:\n"
+ " -B, --tpc-b add TPC-B (sort-of) internal bench\n"
" -c, --client=NUM number of concurrent database clients (default: 1)\n"
" -C, --connect establish new connection for each transaction\n"
" -D, --define=VARNAME=VALUE\n"
" define variable for use by custom script\n"
- " -f, --file=FILENAME read transaction script from FILENAME\n"
+ " -f, --file=FILENAME add transaction script from FILENAME\n"
" -j, --jobs=NUM number of threads (default: 1)\n"
" -l, --log write transaction times to log file\n"
" -L, --latency-limit=NUM count transactions lasting more than NUM ms\n"
@@ -376,17 +391,19 @@ usage(void)
" -M, --protocol=simple|extended|prepared\n"
" protocol for submitting queries (default: simple)\n"
" -n, --no-vacuum do not run VACUUM before tests\n"
- " -N, --skip-some-updates skip updates of pgbench_tellers and pgbench_branches\n"
+ " -N, --skip-some-updates add simple updates internal bench\n"
" -P, --progress=NUM show thread progress report every NUM seconds\n"
" -r, --report-latencies report average latency per command\n"
" -R, --rate=NUM target rate in transactions per second\n"
" -s, --scale=NUM report this scale factor in output\n"
- " -S, --select-only perform SELECT-only transactions\n"
+ " -S, --select-only add SELECT-only internal bench\n"
" -t, --transactions=NUM number of transactions each client runs (default: 10)\n"
" -T, --time=NUM duration of benchmark test in seconds\n"
" -v, --vacuum-all vacuum all four standard tables before tests\n"
+ " -w, --weight=NUM set weight for previous script\n"
" --aggregate-interval=NUM aggregate data over NUM seconds\n"
" --sampling-rate=NUM fraction of transactions to log (e.g. 0.01 for 1%%)\n"
+ " --per-file-stats report per-file statistics\n"
"\nCommon options:\n"
" -d, --debug print debugging output\n"
" -h, --host=HOSTNAME database server host or socket directory\n"
@@ -578,6 +595,49 @@ getPoissonRand(TState *thread, int64 center)
return (int64) (-log(uniform) * ((double) center) + 0.5);
}
+static void
+initSimpleStats(SimpleStats * ss)
+{
+ memset(ss, 0, sizeof(SimpleStats));
+}
+
+static void
+doSimpleStats(SimpleStats *ss, double val)
+{
+ if (ss->count == 0 || val < ss->min)
+ ss->min = val;
+ if (ss->count == 0 || val > ss->max)
+ ss->max = val;
+ ss->count ++;
+ ss->sum += val;
+ ss->sum2 += val * val;
+}
+
+static void
+appendSimpleStats(SimpleStats *acc, SimpleStats *ss)
+{
+ if (acc->count == 0 || ss->min < acc->min)
+ acc->min = ss->min;
+ if (acc->count == 0 || ss->max > acc->max)
+ acc->max = ss->max;
+ acc->count += ss->count;
+ acc->sum += ss->sum;
+ acc->sum2 += ss->sum2;
+}
+
+static void
+initStats(StatsData *sd, double start_time)
+{
+ sd->cnt = 0;
+ sd->skipped = 0;
+ initSimpleStats(& sd->latency);
+ initSimpleStats(& sd->lag);
+
+ /* not necessarily overriden? */
+ if (start_time)
+ sd->start_time = start_time;
+}
+
/* call PQexec() and exit() on failure */
static void
executeStatement(PGconn *con, const char *sql)
@@ -1097,33 +1157,21 @@ clientDone(CState *st, bool ok)
return false; /* always false */
}
-static void
-agg_vals_init(AggVals *aggs, instr_time start)
+static int
+chooseScript(TState *thread)
{
- /* basic counters */
- aggs->cnt = 0; /* number of transactions (includes skipped) */
- aggs->skipped = 0; /* xacts skipped under --rate --latency-limit */
-
- aggs->sum_latency = 0; /* SUM(latency) */
- aggs->sum2_latency = 0; /* SUM(latency*latency) */
+ int i = 0, w = 0, wc = (int) getrand(thread, 0, total_weight - 1);
- /* min and max transaction duration */
- aggs->min_latency = 0;
- aggs->max_latency = 0;
+ do {
+ w += sql_script[i++].weight;
+ } while (w <= wc);
- /* schedule lag counters */
- aggs->sum_lag = 0;
- aggs->sum2_lag = 0;
- aggs->min_lag = 0;
- aggs->max_lag = 0;
-
- /* start of the current interval */
- aggs->start_time = INSTR_TIME_GET_DOUBLE(start);
+ return i - 1;
}
/* return false iff client should be disconnected */
static bool
-doCustom(TState *thread, CState *st, instr_time *conn_time, FILE *logfile, AggVals *agg)
+doCustom(TState *thread, CState *st, instr_time *conn_time, FILE *logfile, StatsData *agg)
{
PGresult *res;
Command **commands;
@@ -1140,7 +1188,7 @@ doCustom(TState *thread, CState *st, instr_time *conn_time, FILE *logfile, AggVa
INSTR_TIME_SET_ZERO(now);
top:
- commands = sql_files[st->use_file];
+ commands = sql_script[st->use_file].commands;
/*
* Handle throttling once per transaction by sleeping. It is simpler to
@@ -1176,18 +1224,15 @@ top:
now_us = INSTR_TIME_GET_MICROSEC(now);
while (thread->throttle_trigger < now_us - latency_limit)
{
- thread->throttle_latency_skipped++;
-
- if (logfile)
- doLog(thread, st, logfile, &now, agg, true);
-
+ doTxStats(thread, st, &now, true, logfile, agg);
+ /* next rendez-vous */
wait = getPoissonRand(thread, throttle_delay);
thread->throttle_trigger += wait;
st->txn_scheduled = thread->throttle_trigger;
}
}
- st->sleeping = 1;
+ st->sleeping = true;
st->throttling = true;
st->is_throttled = true;
if (debug)
@@ -1197,27 +1242,13 @@ top:
if (st->sleeping)
{ /* are we sleeping? */
- int64 now_us;
-
if (INSTR_TIME_IS_ZERO(now))
INSTR_TIME_SET_CURRENT(now);
- now_us = INSTR_TIME_GET_MICROSEC(now);
- if (st->txn_scheduled <= now_us)
- {
- st->sleeping = 0; /* Done sleeping, go ahead with next command */
- if (st->throttling)
- {
- /* Measure lag of throttled transaction relative to target */
- int64 lag = now_us - st->txn_scheduled;
-
- thread->throttle_lag += lag;
- if (lag > thread->throttle_lag_max)
- thread->throttle_lag_max = lag;
- st->throttling = false;
- }
- }
- else
+ if (INSTR_TIME_GET_MICROSEC(now) < st->txn_scheduled)
return true; /* Still sleeping, nothing to do here */
+ /* Else done sleeping, go ahead with next command */
+ st->sleeping = false;
+ st->throttling = false;
}
if (st->listen)
@@ -1253,35 +1284,10 @@ top:
/* transaction finished: calculate latency and log the transaction */
if (commands[st->state + 1] == NULL)
{
- /* only calculate latency if an option is used that needs it */
- if (progress || throttle_delay || latency_limit)
- {
- int64 latency;
-
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
-
- latency = INSTR_TIME_GET_MICROSEC(now) - st->txn_scheduled;
-
- st->txn_latencies += latency;
-
- /*
- * XXX In a long benchmark run of high-latency transactions,
- * this int64 addition eventually overflows. For example, 100
- * threads running 10s transactions will overflow it in 2.56
- * hours. With a more-typical OLTP workload of .1s
- * transactions, overflow would take 256 hours.
- */
- st->txn_sqlats += latency * latency;
-
- /* record over the limit transactions if needed. */
- if (latency_limit && latency > latency_limit)
- thread->latency_late++;
- }
-
- /* record the time it took in the log */
- if (logfile)
- doLog(thread, st, logfile, &now, agg, false);
+ if (progress || throttle_delay || latency_limit || per_script_stats || logfile)
+ doTxStats(thread, st, &now, false, logfile, agg);
+ else
+ thread->stats.cnt ++;
}
if (commands[st->state]->type == SQL_COMMAND)
@@ -1324,8 +1330,9 @@ top:
if (commands[st->state] == NULL)
{
st->state = 0;
- st->use_file = (int) getrand(thread, 0, num_files - 1);
- commands = sql_files[st->use_file];
+ st->use_file = num_scripts==1? 0: chooseScript(thread);
+
+ commands = sql_script[st->use_file].commands;
st->is_throttled = false;
/*
@@ -1368,7 +1375,7 @@ top:
}
/* Record transaction start time under logging, progress or throttling */
- if ((logfile || progress || throttle_delay || latency_limit) && st->state == 0)
+ if ((logfile || progress || throttle_delay || latency_limit || per_script_stats) && st->state == 0)
{
INSTR_TIME_SET_CURRENT(st->txn_begin);
@@ -1656,7 +1663,7 @@ top:
INSTR_TIME_SET_CURRENT(now);
st->txn_scheduled = INSTR_TIME_GET_MICROSEC(now) + usec;
- st->sleeping = 1;
+ st->sleeping = true;
st->listen = 1;
}
@@ -1694,16 +1701,31 @@ top:
return true;
}
+static void
+doStats(StatsData *stats, bool skipped, double lat, double lag)
+{
+ stats->cnt ++;
+
+ if (skipped)
+ /* no latency to record on skipped transactions */
+ stats->skipped ++;
+ else
+ {
+ doSimpleStats(& stats->latency, lat);
+
+ /* and possibly the same for schedule lag */
+ if (throttle_delay)
+ doSimpleStats(& stats->lag, lag);
+ }
+}
+
/*
* print log entry after completing one transaction.
*/
static void
-doLog(TState *thread, CState *st, FILE *logfile, instr_time *now, AggVals *agg,
- bool skipped)
+doLog(TState *thread, CState *st, FILE *logfile, instr_time *now,
+ StatsData *agg, bool skipped, double latency, double lag)
{
- double lag;
- double latency;
-
/*
* Skip the log entry if sampling is enabled and this row doesn't belong
* to the random sample.
@@ -1712,15 +1734,6 @@ doLog(TState *thread, CState *st, FILE *logfile, instr_time *now, AggVals *agg,
pg_erand48(thread->random_state) > sample_rate)
return;
- if (INSTR_TIME_IS_ZERO(*now))
- INSTR_TIME_SET_CURRENT(*now);
-
- latency = (double) (INSTR_TIME_GET_MICROSEC(*now) - st->txn_scheduled);
- if (skipped)
- lag = latency;
- else
- lag = (double) (INSTR_TIME_GET_MICROSEC(st->txn_begin) - st->txn_scheduled);
-
/* should we aggregate the results or not? */
if (agg_interval > 0)
{
@@ -1730,39 +1743,7 @@ doLog(TState *thread, CState *st, FILE *logfile, instr_time *now, AggVals *agg,
*/
if (agg->start_time + agg_interval >= INSTR_TIME_GET_DOUBLE(*now))
{
- agg->cnt += 1;
- if (skipped)
- {
- /*
- * there is no latency to record if the transaction was
- * skipped
- */
- agg->skipped += 1;
- }
- else
- {
- agg->sum_latency += latency;
- agg->sum2_latency += latency * latency;
-
- /* first in this aggregation interval */
- if ((agg->cnt == 1) || (latency < agg->min_latency))
- agg->min_latency = latency;
-
- if ((agg->cnt == 1) || (latency > agg->max_latency))
- agg->max_latency = latency;
-
- /* and the same for schedule lag */
- if (throttle_delay)
- {
- agg->sum_lag += lag;
- agg->sum2_lag += lag * lag;
-
- if ((agg->cnt == 1) || (lag < agg->min_lag))
- agg->min_lag = lag;
- if ((agg->cnt == 1) || (lag > agg->max_lag))
- agg->max_lag = lag;
- }
- }
+ doStats(agg, skipped, latency, lag);
}
else
{
@@ -1777,52 +1758,34 @@ doLog(TState *thread, CState *st, FILE *logfile, instr_time *now, AggVals *agg,
* usage), so we don't need to handle this in a special way
* (see below).
*/
- fprintf(logfile, "%ld %d %.0f %.0f %.0f %.0f",
+ fprintf(logfile, "%ld " INT64_FORMAT " %.0f %.0f %.0f %.0f",
agg->start_time,
agg->cnt,
- agg->sum_latency,
- agg->sum2_latency,
- agg->min_latency,
- agg->max_latency);
+ agg->latency.sum,
+ agg->latency.sum2,
+ agg->latency.min,
+ agg->latency.max);
if (throttle_delay)
{
fprintf(logfile, " %.0f %.0f %.0f %.0f",
- agg->sum_lag,
- agg->sum2_lag,
- agg->min_lag,
- agg->max_lag);
+ agg->lag.sum,
+ agg->lag.sum2,
+ agg->lag.min,
+ agg->lag.max);
if (latency_limit)
- fprintf(logfile, " %d", agg->skipped);
+ fprintf(logfile, " " INT64_FORMAT, agg->skipped);
}
fputc('\n', logfile);
- /* move to the next inteval */
- agg->start_time = agg->start_time + agg_interval;
+ /* move to the next interval */
+ agg->start_time += agg_interval;
/* reset for "no transaction" intervals */
- agg->cnt = 0;
- agg->skipped = 0;
- agg->min_latency = 0;
- agg->max_latency = 0;
- agg->sum_latency = 0;
- agg->sum2_latency = 0;
- agg->min_lag = 0;
- agg->max_lag = 0;
- agg->sum_lag = 0;
- agg->sum2_lag = 0;
+ initStats(agg, 0.0);
}
/* reset the values to include only the current transaction. */
- agg->cnt = 1;
- agg->skipped = skipped ? 1 : 0;
- agg->min_latency = latency;
- agg->max_latency = latency;
- agg->sum_latency = skipped ? 0.0 : latency;
- agg->sum2_latency = skipped ? 0.0 : latency * latency;
- agg->min_lag = lag;
- agg->max_lag = lag;
- agg->sum_lag = lag;
- agg->sum2_lag = lag * lag;
+ doStats(agg, skipped, latency, lag);
}
}
else
@@ -1832,21 +1795,21 @@ doLog(TState *thread, CState *st, FILE *logfile, instr_time *now, AggVals *agg,
/* This is more than we really ought to know about instr_time */
if (skipped)
- fprintf(logfile, "%d %d skipped %d %ld %ld",
+ fprintf(logfile, "%d " INT64_FORMAT " skipped %d %ld %ld",
st->id, st->cnt, st->use_file,
(long) now->tv_sec, (long) now->tv_usec);
else
- fprintf(logfile, "%d %d %.0f %d %ld %ld",
+ fprintf(logfile, "%d " INT64_FORMAT " %.0f %d %ld %ld",
st->id, st->cnt, latency, st->use_file,
(long) now->tv_sec, (long) now->tv_usec);
#else
/* On Windows, instr_time doesn't provide a timestamp anyway */
if (skipped)
- fprintf(logfile, "%d %d skipped %d 0 0",
+ fprintf(logfile, "%d "INT64_FORMAT" skipped %d 0 0",
st->id, st->cnt, st->use_file);
else
- fprintf(logfile, "%d %d %.0f %d 0 0",
+ fprintf(logfile, "%d "INT64_FORMAT" %.0f %d 0 0",
st->id, st->cnt, latency, st->use_file);
#endif
if (throttle_delay)
@@ -1855,6 +1818,44 @@ doLog(TState *thread, CState *st, FILE *logfile, instr_time *now, AggVals *agg,
}
}
+/*
+ * end of transaction statistics
+ */
+static void
+doTxStats(TState *thread, CState *st, instr_time *now,
+ bool skipped, FILE *logfile, StatsData *agg)
+{
+ double latency = 0.0, lag = 0.0;
+
+ if ((!skipped || agg_interval) && INSTR_TIME_IS_ZERO(*now))
+ INSTR_TIME_SET_CURRENT(*now);
+
+ if (!skipped)
+ {
+ /* compute latency & lag if needed */
+ latency = INSTR_TIME_GET_MICROSEC(*now) - st->txn_scheduled;
+ lag = INSTR_TIME_GET_MICROSEC(st->txn_begin) - st->txn_scheduled;
+ }
+
+ if (progress || throttle_delay || latency_limit)
+ {
+ doStats(& thread->stats, skipped, latency, lag);
+
+ /* record over the limit transactions if needed. */
+ if (latency_limit && latency > latency_limit)
+ thread->latency_late++;
+ }
+ else
+ thread->stats.cnt ++;
+
+ if (use_log)
+ doLog(thread, st, logfile, now, agg, skipped, latency, lag);
+
+ if (per_script_stats) /* mutex? hmmm... these are only statistics */
+ doStats(& sql_script[st->use_file].stats, skipped, latency, lag);
+}
+
+
/* discard connections */
static void
disconnect_all(CState *state, int length)
@@ -2475,7 +2476,7 @@ read_line_from_file(FILE *fd)
return NULL;
}
-static int
+static Command **
process_file(char *filename)
{
#define COMMANDS_ALLOC_NUM 128
@@ -2487,12 +2488,6 @@ process_file(char *filename)
char *buf;
int alloc_num;
- if (num_files >= MAX_FILES)
- {
- fprintf(stderr, "at most %d SQL files are allowed\n", MAX_FILES);
- exit(1);
- }
-
alloc_num = COMMANDS_ALLOC_NUM;
my_commands = (Command **) pg_malloc(sizeof(Command *) * alloc_num);
@@ -2503,7 +2498,7 @@ process_file(char *filename)
fprintf(stderr, "could not open file \"%s\": %s\n",
filename, strerror(errno));
pg_free(my_commands);
- return false;
+ return NULL;
}
lineno = 0;
@@ -2535,13 +2530,11 @@ process_file(char *filename)
my_commands[index] = NULL;
- sql_files[num_files++] = my_commands;
-
- return true;
+ return my_commands;
}
static Command **
-process_builtin(char *tb, const char *source)
+process_builtin(const char *tb, const char *source)
{
#define COMMANDS_ALLOC_NUM 128
@@ -2595,35 +2588,54 @@ process_builtin(char *tb, const char *source)
return my_commands;
}
+static void
+addScript(const char *name, Command ** commands)
+{
+ if (commands == NULL)
+ {
+ fprintf(stderr, "empty commands for %s", name);
+ exit(1);
+ }
+
+ if (num_scripts >= MAX_SCRIPTS)
+ {
+ fprintf(stderr, "at most %d SQL scripts are allowed\n", MAX_SCRIPTS);
+ exit(1);
+ }
+
+ sql_script[num_scripts].name = name;
+ sql_script[num_scripts].weight = 1;
+ sql_script[num_scripts].commands = commands;
+ initStats(& sql_script[num_scripts].stats, 0.0);
+ num_scripts++;
+}
+
+static void
+printSimpleStats(char *prefix, SimpleStats *ss)
+{
+ /* print NaN if no transactions where executed */
+ double latency = ss->sum / ss->count;
+ double stddev = sqrt(ss->sum2 / ss->count - latency*latency);
+ printf("%s average = %.3f ms\n", prefix, 0.001 * latency);
+ printf("%s stddev = %.3f ms\n", prefix, 0.001 * stddev);
+}
+
/* print out results */
static void
-printResults(int ttype, int64 normal_xacts, int nclients,
- TState *threads, int nthreads,
- instr_time total_time, instr_time conn_total_time,
- int64 total_latencies, int64 total_sqlats,
- int64 throttle_lag, int64 throttle_lag_max,
- int64 throttle_latency_skipped, int64 latency_late)
+printResults(TState *threads, StatsData *total, instr_time total_time,
+ instr_time conn_total_time, int latency_late)
{
double time_include,
tps_include,
tps_exclude;
- char *s;
time_include = INSTR_TIME_GET_DOUBLE(total_time);
- tps_include = normal_xacts / time_include;
- tps_exclude = normal_xacts / (time_include -
+ tps_include = total->cnt / time_include;
+ tps_exclude = total->cnt / (time_include -
(INSTR_TIME_GET_DOUBLE(conn_total_time) / nthreads));
- if (ttype == 0)
- s = "TPC-B (sort of)";
- else if (ttype == 2)
- s = "Update only pgbench_accounts";
- else if (ttype == 1)
- s = "SELECT only";
- else
- s = "Custom query";
-
- printf("transaction type: %s\n", s);
+ printf("transaction type: %s\n",
+ num_scripts == 1? sql_script[0].name: "multiple scripts");
printf("scaling factor: %d\n", scale);
printf("query mode: %s\n", QUERYMODE[querymode]);
printf("number of clients: %d\n", nclients);
@@ -2631,49 +2643,38 @@ printResults(int ttype, int64 normal_xacts, int nclients,
if (duration <= 0)
{
printf("number of transactions per client: %d\n", nxacts);
- printf("number of transactions actually processed: " INT64_FORMAT "/" INT64_FORMAT "\n",
- normal_xacts, (int64) nxacts * nclients);
+ printf("number of transactions actually processed: "INT64_FORMAT"/%d\n",
+ total->cnt, nxacts * nclients);
}
else
{
printf("duration: %d s\n", duration);
- printf("number of transactions actually processed: " INT64_FORMAT "\n",
- normal_xacts);
+ printf("number of transactions actually processed: "INT64_FORMAT"\n",
+ total->cnt);
}
/* Remaining stats are nonsensical if we failed to execute any xacts */
- if (normal_xacts <= 0)
+ if (total->cnt <= 0)
return;
if (throttle_delay && latency_limit)
- printf("number of transactions skipped: " INT64_FORMAT " (%.3f %%)\n",
- throttle_latency_skipped,
- 100.0 * throttle_latency_skipped / (throttle_latency_skipped + normal_xacts));
+ printf("number of transactions skipped: "INT64_FORMAT" (%.3f %%)\n",
+ total->skipped,
+ 100.0 * total->skipped / (total->skipped + total->cnt));
if (latency_limit)
- printf("number of transactions above the %.1f ms latency limit: " INT64_FORMAT " (%.3f %%)\n",
+ printf("number of transactions above the %.1f ms latency limit: %d (%.3f %%)\n",
latency_limit / 1000.0, latency_late,
- 100.0 * latency_late / (throttle_latency_skipped + normal_xacts));
+ 100.0 * latency_late / (total->skipped + total->cnt));
if (throttle_delay || progress || latency_limit)
- {
- /* compute and show latency average and standard deviation */
- double latency = 0.001 * total_latencies / normal_xacts;
- double sqlat = (double) total_sqlats / normal_xacts;
-
- printf("latency average: %.3f ms\n"
- "latency stddev: %.3f ms\n",
- latency, 0.001 * sqrt(sqlat - 1000000.0 * latency * latency));
- }
+ printSimpleStats("latency", & total->latency);
else
- {
/* only an average latency computed from the duration is available */
printf("latency average: %.3f ms\n",
- 1000.0 * duration * nclients / normal_xacts);
- }
+ 1000.0 * duration * nclients / total->cnt);
if (throttle_delay)
- {
/*
* Report average transaction lag under rate limit throttling. This
* is the delay between scheduled and actual start times for the
@@ -2681,27 +2682,48 @@ printResults(int ttype, int64 normal_xacts, int nclients,
* the database load, or the Poisson throttling process.
*/
printf("rate limit schedule lag: avg %.3f (max %.3f) ms\n",
- 0.001 * throttle_lag / normal_xacts, 0.001 * throttle_lag_max);
- }
+ 0.001 * total->lag.sum / total->cnt, 0.001 * total->lag.max);
printf("tps = %f (including connections establishing)\n", tps_include);
printf("tps = %f (excluding connections establishing)\n", tps_exclude);
+ /* Report per-file data */
+ if (per_script_stats)
+ {
+ int i;
+
+ for (i = 0; i < num_scripts; i++)
+ {
+ printf("SQL script %d, weight %d: %s\n"
+ " - "INT64_FORMAT" transactions (tps = %f)\n",
+ i, sql_script[i].weight, sql_script[i].name,
+ sql_script[i].stats.cnt, sql_script[i].stats.cnt / time_include);
+
+ if (latency_limit)
+ printf(" - number of transactions skipped: "INT64_FORMAT" (%.3f%%)\n",
+ sql_script[i].stats.skipped,
+ 100.0 * sql_script[i].stats.skipped /
+ (sql_script[i].stats.skipped + sql_script[i].stats.cnt));
+
+ printSimpleStats(" - latency", & sql_script[i].stats.latency);
+ }
+ }
+
/* Report per-command latencies */
if (is_latencies)
{
int i;
- for (i = 0; i < num_files; i++)
+ for (i = 0; i < num_scripts; i++)
{
Command **commands;
- if (num_files > 1)
+ if (num_scripts > 1)
printf("statement latencies in milliseconds, file %d:\n", i + 1);
else
printf("statement latencies in milliseconds:\n");
- for (commands = sql_files[i]; *commands != NULL; commands++)
+ for (commands = sql_script[i].commands; *commands != NULL; commands++)
{
Command *command = *commands;
int cnum = command->command_num;
@@ -2739,6 +2761,7 @@ main(int argc, char **argv)
{
static struct option long_options[] = {
/* systematic long/short named options */
+ {"tpc-b", no_argument, NULL, 'B'},
{"client", required_argument, NULL, 'c'},
{"connect", no_argument, NULL, 'C'},
{"debug", no_argument, NULL, 'd'},
@@ -2749,12 +2772,14 @@ main(int argc, char **argv)
{"initialize", no_argument, NULL, 'i'},
{"jobs", required_argument, NULL, 'j'},
{"log", no_argument, NULL, 'l'},
+ {"latency-limit", required_argument, NULL, 'L'},
{"no-vacuum", no_argument, NULL, 'n'},
{"port", required_argument, NULL, 'p'},
{"progress", required_argument, NULL, 'P'},
{"protocol", required_argument, NULL, 'M'},
{"quiet", no_argument, NULL, 'q'},
{"report-latencies", no_argument, NULL, 'r'},
+ {"rate", required_argument, NULL, 'R'},
{"scale", required_argument, NULL, 's'},
{"select-only", no_argument, NULL, 'S'},
{"skip-some-updates", no_argument, NULL, 'N'},
@@ -2762,6 +2787,7 @@ main(int argc, char **argv)
{"transactions", required_argument, NULL, 't'},
{"username", required_argument, NULL, 'U'},
{"vacuum-all", no_argument, NULL, 'v'},
+ {"weight", required_argument, NULL, 'w'},
/* long-named only options */
{"foreign-keys", no_argument, &foreign_keys, 1},
{"index-tablespace", required_argument, NULL, 3},
@@ -2769,25 +2795,20 @@ main(int argc, char **argv)
{"unlogged-tables", no_argument, &unlogged_tables, 1},
{"sampling-rate", required_argument, NULL, 4},
{"aggregate-interval", required_argument, NULL, 5},
- {"rate", required_argument, NULL, 'R'},
- {"latency-limit", required_argument, NULL, 'L'},
+ {"per-script-stats", no_argument, NULL, 6},
{NULL, 0, NULL, 0}
};
int c;
- int nclients = 1; /* default number of simulated clients */
- int nthreads = 1; /* default number of threads */
int is_init_mode = 0; /* initialize mode? */
int is_no_vacuum = 0; /* no vacuum at all before testing? */
int do_vacuum_accounts = 0; /* do vacuum accounts before testing? */
- int ttype = 0; /* transaction type. 0: TPC-B, 1: SELECT only,
- * 2: skip update of branches and tellers */
int optindex;
- char *filename = NULL;
bool scale_given = false;
bool benchmarking_option_set = false;
bool initialization_option_set = false;
+ bool internal_script_used = false;
CState *state; /* status of clients */
TState *threads; /* array of thread */
@@ -2795,13 +2816,8 @@ main(int argc, char **argv)
instr_time start_time; /* start up time */
instr_time total_time;
instr_time conn_total_time;
- int64 total_xacts = 0;
- int64 total_latencies = 0;
- int64 total_sqlats = 0;
- int64 throttle_lag = 0;
- int64 throttle_lag_max = 0;
- int64 throttle_latency_skipped = 0;
int64 latency_late = 0;
+ StatsData stats;
int i;
int nclients_dealt;
@@ -2847,7 +2863,7 @@ main(int argc, char **argv)
state = (CState *) pg_malloc(sizeof(CState));
memset(state, 0, sizeof(CState));
- while ((c = getopt_long(argc, argv, "ih:nvp:dqSNc:j:Crs:t:T:U:lf:D:F:M:P:R:L:", long_options, &optindex)) != -1)
+ while ((c = getopt_long(argc, argv, "ih:nvp:dqBSNw:c:j:Crs:t:T:U:lf:D:F:M:P:R:L:", long_options, &optindex)) != -1)
{
switch (c)
{
@@ -2869,14 +2885,6 @@ main(int argc, char **argv)
case 'd':
debug++;
break;
- case 'S':
- ttype = 1;
- benchmarking_option_set = true;
- break;
- case 'N':
- ttype = 2;
- benchmarking_option_set = true;
- break;
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
@@ -2979,12 +2987,44 @@ main(int argc, char **argv)
initialization_option_set = true;
use_quiet = true;
break;
+ /* what to run*/
case 'f':
+ addScript(optarg, process_file(optarg));
benchmarking_option_set = true;
- ttype = 3;
- filename = pg_strdup(optarg);
- if (process_file(filename) == false || *sql_files[num_files - 1] == NULL)
- exit(1);
+ break;
+ case 'B':
+ addScript(b_bench, process_builtin(tpc_b, b_bench));
+ benchmarking_option_set = true;
+ internal_script_used = true;
+ break;
+ case 'S':
+ addScript(s_bench, process_builtin(select_only, s_bench));
+ benchmarking_option_set = true;
+ internal_script_used = true;
+ break;
+ case 'N':
+ addScript(n_bench, process_builtin(simple_update, n_bench));
+ benchmarking_option_set = true;
+ internal_script_used = true;
+ break;
+ case 'w':
+ {
+ int weight = atoi(optarg);
+ if (weight <= 0)
+ {
+ fprintf(stderr,
+ "invalid weight value: \"%s\"\n", optarg);
+ exit(1);
+ }
+ if (num_scripts == 0)
+ {
+ fprintf(stderr,
+ "weight must come after its script selection"
+ " (-B -N -S or -f ...)\n");
+ exit(1);
+ }
+ sql_script[num_scripts - 1].weight = weight;
+ }
break;
case 'D':
{
@@ -3015,7 +3055,7 @@ main(int argc, char **argv)
break;
case 'M':
benchmarking_option_set = true;
- if (num_files > 0)
+ if (num_scripts > 0)
{
fprintf(stderr, "query mode (-M) should be specified before any transaction scripts (-f)\n");
exit(1);
@@ -3107,6 +3147,9 @@ main(int argc, char **argv)
}
#endif
break;
+ case 6:
+ per_script_stats = true;
+ break;
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
exit(1);
@@ -3114,6 +3157,18 @@ main(int argc, char **argv)
}
}
+ /* set default script if none */
+ if (num_scripts == 0 && !initialization_option_set)
+ {
+ addScript(b_bench, process_builtin(tpc_b, b_bench));
+ benchmarking_option_set = true;
+ internal_script_used = true;
+ }
+
+ /* compute total_weight */
+ for (i = 0; i < num_scripts; i++)
+ total_weight += sql_script[i].weight;
+
/*
* Don't need more threads than there are clients. (This is not merely an
* optimization; throttle_delay is calculated incorrectly below if some
@@ -3198,8 +3253,6 @@ main(int argc, char **argv)
* changed after fork.
*/
main_pid = (int) getpid();
- progress_nclients = nclients;
- progress_nthreads = nthreads;
if (nclients > 1)
{
@@ -3242,7 +3295,7 @@ main(int argc, char **argv)
exit(1);
}
- if (ttype != 3)
+ if (internal_script_used)
{
/*
* get the scaling factor that should be same as count(*) from
@@ -3319,31 +3372,6 @@ main(int argc, char **argv)
INSTR_TIME_SET_CURRENT(start_time);
srandom((unsigned int) INSTR_TIME_GET_MICROSEC(start_time));
- /* process builtin SQL scripts */
- switch (ttype)
- {
- case 0:
- sql_files[0] = process_builtin(tpc_b,
- "<builtin: TPC-B (sort of)>");
- num_files = 1;
- break;
-
- case 1:
- sql_files[0] = process_builtin(select_only,
- "<builtin: select only>");
- num_files = 1;
- break;
-
- case 2:
- sql_files[0] = process_builtin(simple_update,
- "<builtin: simple update>");
- num_files = 1;
- break;
-
- default:
- break;
- }
-
/* set up thread data structures */
threads = (TState *) pg_malloc(sizeof(TState) * nthreads);
nclients_dealt = 0;
@@ -3359,8 +3387,8 @@ main(int argc, char **argv)
thread->random_state[0] = random();
thread->random_state[1] = random();
thread->random_state[2] = random();
- thread->throttle_latency_skipped = 0;
thread->latency_late = 0;
+ initStats(& thread->stats, 0.0);
nclients_dealt += thread->nstate;
@@ -3427,11 +3455,11 @@ main(int argc, char **argv)
#endif /* ENABLE_THREAD_SAFETY */
/* wait for threads and accumulate results */
+ initStats(& stats, 0.0);
INSTR_TIME_SET_ZERO(conn_total_time);
for (i = 0; i < nthreads; i++)
{
TState *thread = &threads[i];
- int j;
#ifdef ENABLE_THREAD_SAFETY
if (threads[i].thread == INVALID_THREAD)
@@ -3444,21 +3472,13 @@ main(int argc, char **argv)
(void) threadRun(thread);
#endif /* ENABLE_THREAD_SAFETY */
- /* thread level stats */
- throttle_lag += thread->throttle_lag;
- throttle_latency_skipped = threads->throttle_latency_skipped;
- latency_late = thread->latency_late;
- if (throttle_lag_max > thread->throttle_lag_max)
- throttle_lag_max = thread->throttle_lag_max;
+ /* aggregate thread level stats */
+ appendSimpleStats(& stats.latency, & thread->stats.latency);
+ appendSimpleStats(& stats.lag, & thread->stats.lag);
+ stats.cnt += thread->stats.cnt;
+ stats.skipped += thread->stats.skipped;
+ latency_late += thread->latency_late;
INSTR_TIME_ADD(conn_total_time, thread->conn_time);
-
- /* client-level stats */
- for (j = 0; j < thread->nstate; j++)
- {
- total_xacts += thread->state[j].cnt;
- total_latencies += thread->state[i].txn_latencies;
- total_sqlats += thread->state[i].txn_sqlats;
- }
}
disconnect_all(state, nclients);
@@ -3474,10 +3494,7 @@ main(int argc, char **argv)
*/
INSTR_TIME_SET_CURRENT(total_time);
INSTR_TIME_SUBTRACT(total_time, start_time);
- printResults(ttype, total_xacts, nclients, threads, nthreads,
- total_time, conn_total_time, total_latencies, total_sqlats,
- throttle_lag, throttle_lag_max, throttle_latency_skipped,
- latency_late);
+ printResults(threads, &stats, total_time, conn_total_time, latency_late);
return 0;
}
@@ -3498,13 +3515,7 @@ threadRun(void *arg)
int64 thread_start = INSTR_TIME_GET_MICROSEC(thread->start_time);
int64 last_report = thread_start;
int64 next_report = last_report + (int64) progress * 1000000;
- int64 last_count = 0,
- last_lats = 0,
- last_sqlats = 0,
- last_lags = 0,
- last_skipped = 0;
-
- AggVals aggs;
+ StatsData last, aggs;
/*
* Initialize throttling rate target for all of the thread's clients. It
@@ -3514,8 +3525,6 @@ threadRun(void *arg)
*/
INSTR_TIME_SET_CURRENT(start);
thread->throttle_trigger = INSTR_TIME_GET_MICROSEC(start);
- thread->throttle_lag = 0;
- thread->throttle_lag_max = 0;
INSTR_TIME_SET_ZERO(thread->conn_time);
@@ -3552,16 +3561,17 @@ threadRun(void *arg)
INSTR_TIME_SET_CURRENT(thread->conn_time);
INSTR_TIME_SUBTRACT(thread->conn_time, thread->start_time);
- agg_vals_init(&aggs, thread->start_time);
+ initStats(&aggs, INSTR_TIME_GET_DOUBLE(thread->start_time));
+ last = aggs;
/* send start up queries in async manner */
for (i = 0; i < nstate; i++)
{
CState *st = &state[i];
- Command **commands = sql_files[st->use_file];
+ Command **commands = sql_script[st->use_file].commands;
int prev_ecnt = st->ecnt;
- st->use_file = getrand(thread, 0, num_files - 1);
+ st->use_file = num_scripts==1? 0: chooseScript(thread);
if (!doCustom(thread, st, &thread->conn_time, logfile, &aggs))
remains--; /* I've aborted */
@@ -3589,7 +3599,7 @@ threadRun(void *arg)
for (i = 0; i < nstate; i++)
{
CState *st = &state[i];
- Command **commands = sql_files[st->use_file];
+ Command **commands = sql_script[st->use_file].commands;
int sock;
if (st->con == NULL)
@@ -3602,7 +3612,7 @@ threadRun(void *arg)
{
/* interrupt client which has not started a transaction */
remains--;
- st->sleeping = 0;
+ st->sleeping = false;
st->throttling = false;
PQfinish(st->con);
st->con = NULL;
@@ -3695,7 +3705,7 @@ threadRun(void *arg)
for (i = 0; i < nstate; i++)
{
CState *st = &state[i];
- Command **commands = sql_files[st->use_file];
+ Command **commands = sql_script[st->use_file].commands;
int prev_ecnt = st->ecnt;
if (st->con && (FD_ISSET(PQsocket(st->con), &input_mask)
@@ -3726,11 +3736,7 @@ threadRun(void *arg)
if (now >= next_report)
{
/* generate and show report */
- int64 count = 0,
- lats = 0,
- sqlats = 0,
- lags = 0,
- skipped = 0;
+ StatsData cur;
int64 run = now - last_report;
double tps,
total_run,
@@ -3750,23 +3756,24 @@ threadRun(void *arg)
* (If a read from a 64-bit integer is not atomic, you might
* get a "torn" read and completely bogus latencies though!)
*/
- for (i = 0; i < progress_nclients; i++)
+ initStats(& cur, 0.0);
+ for (i = 0; i < nthreads; i++)
{
- count += state[i].cnt;
- lats += state[i].txn_latencies;
- sqlats += state[i].txn_sqlats;
+ appendSimpleStats(& cur.latency, & thread[i].stats.latency);
+ appendSimpleStats(& cur.lag, & thread[i].stats.lag);
+ cur.cnt += thread[i].stats.cnt;
+ cur.skipped += thread[i].stats.skipped;
}
- for (i = 0; i < progress_nthreads; i++)
- lags += thread[i].throttle_lag;
-
total_run = (now - thread_start) / 1000000.0;
- tps = 1000000.0 * (count - last_count) / run;
- latency = 0.001 * (lats - last_lats) / (count - last_count);
- sqlat = 1.0 * (sqlats - last_sqlats) / (count - last_count);
+ tps = 1000000.0 * (cur.cnt - last.cnt) / run;
+ latency = 0.001 * (cur.latency.sum - last.latency.sum) /
+ (cur.cnt - last.cnt);
+ sqlat = 1.0 * (cur.latency.sum2 - last.latency.sum2)
+ / (cur.cnt - last.cnt);
stdev = 0.001 * sqrt(sqlat - 1000000.0 * latency * latency);
- lag = 0.001 * (lags - last_lags) / (count - last_count);
- skipped = thread->throttle_latency_skipped - last_skipped;
+ lag = 0.001 * (cur.lag.sum - last.lag.sum) /
+ (cur.cnt - last.cnt);
fprintf(stderr,
"progress: %.1f s, %.1f tps, "
@@ -3776,16 +3783,13 @@ threadRun(void *arg)
{
fprintf(stderr, ", lag %.3f ms", lag);
if (latency_limit)
- fprintf(stderr, ", " INT64_FORMAT " skipped", skipped);
+ fprintf(stderr, ", "INT64_FORMAT" skipped",
+ cur.skipped - last.skipped);
}
fprintf(stderr, "\n");
- last_count = count;
- last_lats = lats;
- last_sqlats = sqlats;
- last_lags = lags;
+ last = cur;
last_report = now;
- last_skipped = thread->throttle_latency_skipped;
/*
* Ensure that the next report is in the future, in case
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
