Re: pgbench - implement strict TPC-B benchmark

2019-08-28 Thread Dmitry Dolgov
> On Wed, Aug 28, 2019 at 7:37 AM Fabien COELHO wrote: > > > While doing benchmarking using different tools, including pgbench, I found > > it > > useful as a temporary hack to add copy freeze and maintenance_work_mem > > options > > (the last one not as an env variable, just as a set before,

Re: pgbench - implement strict TPC-B benchmark

2019-08-27 Thread Fabien COELHO
Hello Dmitry, Well, it could be added? While doing benchmarking using different tools, including pgbench, I found it useful as a temporary hack to add copy freeze and maintenance_work_mem options (the last one not as an env variable, just as a set before, although not sure if it's a best

Re: pgbench - implement strict TPC-B benchmark

2019-08-27 Thread Dmitry Dolgov
> On Mon, Aug 5, 2019 at 10:46 PM Fabien COELHO wrote: > > > The index builds are done serially. The vacuum could be replaced by COPY > > FREEZE. > > Well, it could be added? While doing benchmarking using different tools, including pgbench, I found it useful as a temporary hack to add copy

Re: pgbench - implement strict TPC-B benchmark

2019-08-06 Thread Fabien COELHO
Hello Robert, Ok, one thread cannot feed an N core server if enough client are executed per thread and the server has few things to do. Right ... where N is, uh, TWO. Yes, two indeed… For low-work cpu-bound load, given libpq & system overheads, you cannot really hope for a better deal.

Re: pgbench - implement strict TPC-B benchmark

2019-08-05 Thread Fabien COELHO
Hello Andres, Which is a (somehow disappointing) * 3.3 speedup. The impact on the 3 complex expressions tests is not measurable, though. I don't know why that could be disappointing. We put in much more work for much smaller gains in other places. Probably, but I thought I would have a

Re: pgbench - implement strict TPC-B benchmark

2019-08-05 Thread Robert Haas
On Fri, Aug 2, 2019 at 2:38 AM Fabien COELHO wrote: > Ok, one thread cannot feed an N core server if enough client are executed > per thread and the server has few things to do. Right ... where N is, uh, TWO. > The point I'm clumsily trying to make is that pgbench-specific overheads > are quite

Re: pgbench - implement strict TPC-B benchmark

2019-08-05 Thread Andres Freund
Hi, On 2019-08-05 17:38:23 +0200, Fabien COELHO wrote: > Which is a (somehow disappointing) * 3.3 speedup. The impact on the 3 > complex expressions tests is not measurable, though. I don't know why that could be disappointing. We put in much more work for much smaller gains in other places. >

Re: pgbench - implement strict TPC-B benchmark

2019-08-05 Thread Fabien COELHO
Hello Andres, If not, do you think advisable to spend time improving the evaluator & variable stuff and possibly other places for an overall 15% gain? Also, what would be the likelyhood of such optimization patch to pass? I could do a limited variable management improvement patch,

Re: pgbench - implement strict TPC-B benchmark

2019-08-03 Thread Fabien COELHO
Hello Andres, Using pgbench -Mprepared -n -c 8 -j 8 -S pgbench_100 -T 10 -r -P1 e.g. shows pgbench to use 189% CPU in my 4/8 core/thread laptop. That's a pretty significant share. Fine, but what is the corresponding server load? 211%? 611%? And what actual time is spent in pgbench itself, vs

Re: pgbench - implement strict TPC-B benchmark

2019-08-02 Thread Andres Freund
Hi, On 2019-08-02 10:34:24 +0200, Fabien COELHO wrote: > > Hello Andres, > > Thanks a lot for these feedbacks and comments. > > > Using pgbench -Mprepared -n -c 8 -j 8 -S pgbench_100 -T 10 -r -P1 > > e.g. shows pgbench to use 189% CPU in my 4/8 core/thread laptop. That's > > a pretty

Re: pgbench - implement strict TPC-B benchmark

2019-08-02 Thread Fabien COELHO
Hello Andres, Thanks a lot for these feedbacks and comments. Using pgbench -Mprepared -n -c 8 -j 8 -S pgbench_100 -T 10 -r -P1 e.g. shows pgbench to use 189% CPU in my 4/8 core/thread laptop. That's a pretty significant share. Fine, but what is the corresponding server load? 211%? 611%? And

Re: pgbench - implement strict TPC-B benchmark

2019-08-02 Thread Fabien COELHO
Hello Robert, All in all, pgbench overheads are small compared to postgres processing times and representative of a reasonably optimized client application. It's pretty easy to devise tests where pgbench is client-limited -- just try running it with threads = clients/4, sometimes even

Re: pgbench - implement strict TPC-B benchmark

2019-08-01 Thread Andres Freund
Hi, On 2019-08-01 08:52:52 +0200, Fabien COELHO wrote: > sh> time pgbench -r -T 30 -M prepared > ... > latency average = 2.425 ms > tps = 412.394420 (including connections establishing) > statement latencies in milliseconds: > 0.001 \set aid random(1, 10 * :scale) > 0.000

Re: pgbench - implement strict TPC-B benchmark

2019-08-01 Thread Robert Haas
On Thu, Aug 1, 2019 at 2:53 AM Fabien COELHO wrote: > All in all, pgbench overheads are small compared to postgres processing > times and representative of a reasonably optimized client application. It's pretty easy to devise tests where pgbench is client-limited -- just try running it with

Re: pgbench - implement strict TPC-B benchmark

2019-07-31 Thread Peter Geoghegan
On Wed, Jul 31, 2019 at 2:11 PM Tom Lane wrote: > > I agree with this. When I was at EnterpriseDB, while it wasn't audited, we > > had to develop an actual TPC-B implementation because pgbench was too > > different. pgbench itself isn't that useful as a benchmark tool, imo, but > > if we have the

Re: pgbench - implement strict TPC-B benchmark

2019-07-31 Thread Tom Lane
"Jonah H. Harris" writes: > On Wed, Jul 31, 2019 at 10:10 AM Fabien COELHO wrote: >> I agree that nobody really cares about TPC-B per se. The point of this >> patch is to provide a built-in example of recent and useful pgbench >> features that match a real specification. > I agree with this.

Re: pgbench - implement strict TPC-B benchmark

2019-07-31 Thread Jonah H. Harris
On Wed, Jul 31, 2019 at 10:10 AM Fabien COELHO wrote: > > Hello Tom, > > >>> I'm also highly dubious about labeling this script "standard TPC-B", > >>> when it resolves only some of the reasons why our traditional script > >>> is not really TPC-B. That's treading on being false advertising. > >

Re: pgbench - implement strict TPC-B benchmark

2019-07-31 Thread Fabien COELHO
Hello Tom, I'm also highly dubious about labeling this script "standard TPC-B", when it resolves only some of the reasons why our traditional script is not really TPC-B. That's treading on being false advertising. IANAL, but it may not even be permissible to claim that we have implemented

Re: pgbench - implement strict TPC-B benchmark

2019-07-30 Thread Tom Lane
Peter Geoghegan writes: > On Tue, Jul 30, 2019 at 3:00 PM Tom Lane wrote: >> I'm also highly dubious about labeling this script "standard TPC-B", >> when it resolves only some of the reasons why our traditional script >> is not really TPC-B. That's treading on being false advertising. > IANAL,

Re: pgbench - implement strict TPC-B benchmark

2019-07-30 Thread Peter Geoghegan
On Tue, Jul 30, 2019 at 3:00 PM Tom Lane wrote: > TBH, I think we should reject this patch. Nobody cares about TPC-B > anymore, and they care even less about differences between one > sort-of-TPC-B test and another sort-of-TPC-B test. (As the lack > of response on this thread shows.) We don't

Re: pgbench - implement strict TPC-B benchmark

2019-07-30 Thread Tom Lane
Fabien COELHO writes: > [ pgbench-strict-tpcb-2.patch ] TBH, I think we should reject this patch. Nobody cares about TPC-B anymore, and they care even less about differences between one sort-of-TPC-B test and another sort-of-TPC-B test. (As the lack of response on this thread shows.) We don't

Re: pgbench - implement strict TPC-B benchmark

2019-07-14 Thread Fabien COELHO
Hello Thomas, Thanks for the feedback. + the account branch has a 15% probability to be in the same branch as the teller (unless I would say "... has a 15% probability of being in the same ...". The same wording appears further down in the comment. Fixed. I see that the parameters you

Re: pgbench - implement strict TPC-B benchmark

2019-07-13 Thread Thomas Munro
On Tue, Apr 9, 2019 at 3:58 AM Fabien COELHO wrote: > The attached patch does $SUBJECT, as a showcase for recently added > features, including advanced expressions (CASE...), \if, \gset, ending SQL > commands at ";"... Hi Fabien, + the account branch has a 15% probability to be in the same

pgbench - implement strict TPC-B benchmark

2019-04-08 Thread Fabien COELHO
Hello devs, The attached patch does $SUBJECT, as a showcase for recently added features, including advanced expressions (CASE...), \if, \gset, ending SQL commands at ";"... There is also a small fix to the doc which describes the tpcb-like implementation but gets one variable name wrong: