Re: [HACKERS] pgbench progress report improvements - split 3 v2 - A

2013-10-08 Thread Fabien COELHO
On Sat, Oct 5, 2013 at 6:10 PM, Noah Misch wrote: The sum of the squares of the latencies wraps after 2^63/(10^12 * avg_latency * nclients) seconds. That's unlikely to come up with the ordinary pgbench script, but one can reach it in a few hours when benchmarking a command that runs for many

Re: [HACKERS] pgbench progress report improvements - split 3 v2 - A

2013-10-07 Thread Robert Haas
On Sat, Oct 5, 2013 at 6:10 PM, Noah Misch wrote: > The sum of the squares of the latencies wraps after 2^63/(10^12 * avg_latency > * nclients) seconds. That's unlikely to come up with the ordinary pgbench > script, but one can reach it in a few hours when benchmarking a command that > runs for m

Re: [HACKERS] pgbench progress report improvements - split 3 v2 - A

2013-10-07 Thread Fabien COELHO
pgbench already offers two schedules of "pgbench --initialize" messaging, message-per-100k-rows and message-per-5s. A user too picky to find satisfaction in either option can filter the messages through grep, sed et al. We patched pgbench on two occasions during the 9.3 cycle to arrive at that

Re: [HACKERS] pgbench progress report improvements - split 3 v2 - A

2013-10-06 Thread Noah Misch
On Sun, Oct 06, 2013 at 06:44:11PM +0200, Fabien COELHO wrote: > >>>Patch (2): Make --initialize mode respect --progress. > >>>Rejected > >> > >>I missed this one... > > > >See the second half of this message, including quoted material: > >http://www.postgresql.org/message-id/CA+TgmoZNXkm-EtszHX=kw

Re: [HACKERS] pgbench progress report improvements - split 3 v2 - A

2013-10-06 Thread Fabien COELHO
Patch (2): Make --initialize mode respect --progress. Rejected I missed this one... See the second half of this message, including quoted material: http://www.postgresql.org/message-id/CA+TgmoZNXkm-EtszHX=kwq34h5ni4cs8dg31no86cmdryaq...@mail.gmail.com I did not read Peter Haas comments as

Re: [HACKERS] pgbench progress report improvements - split 3 v2 - A

2013-10-06 Thread Noah Misch
On Sun, Oct 06, 2013 at 05:04:56PM +0200, Fabien COELHO wrote: > > Patch (2): Make --initialize mode respect --progress. > >Rejected > > I missed this one... See the second half of this message, including quoted material: http://www.postgresql.org/message-id/CA+TgmoZNXkm-EtszHX=kwq34h5ni4cs8dg31n

Re: [HACKERS] pgbench progress report improvements - split 3 v2 - A

2013-10-06 Thread Fabien COELHO
Hello Noah, Patch (1): Change the default from --progress=0 to --progress=5 Rejected Yep. Too bad, esp as the default is meaningless and does not scale. Patch (2): Make --initialize mode respect --progress. Rejected I missed this one... This is just about having the same option (--prog

Re: [HACKERS] pgbench progress report improvements - split 3 v2 - A

2013-10-06 Thread Noah Misch
On Sun, Oct 06, 2013 at 09:40:40AM +0200, Fabien COELHO wrote: > Which part do you want as a next step? I think we're done; here are the distinct changes in your original patch, along with their dispositions: Patch (1): Change the default from --progress=0 to --progress=5 Rejected Patch (2)

Re: [HACKERS] pgbench progress report improvements - split 3 v2 - A

2013-10-06 Thread Fabien COELHO
Hello Noah, Patch (4): Redefine "latency" as reported by pgbench and report "lag" more. Here is a first partial patch, which focusses on measuring latency and reporting the measure under --progress. This patch contains the features pertaining to both hypothetical patches (3) and (4), not ju

Re: [HACKERS] pgbench progress report improvements - split 3 v2 - A

2013-10-05 Thread Noah Misch
On Thu, Sep 26, 2013 at 08:50:15PM +0200, Fabien COELHO wrote: > > Patch (4): Redefine "latency" as reported by pgbench and report "lag" more. > > Here is a first partial patch, which focusses on measuring latency > and reporting the measure under --progress. This patch contains the features pert

Re: [HACKERS] pgbench progress report improvements - split 3 v2 - A

2013-09-26 Thread Fabien COELHO
Patch (4): Redefine "latency" as reported by pgbench and report "lag" more. Here is a first partial patch, which focusses on measuring latency and reporting the measure under --progress. ** Improve pgbench measurements & progress report Measure transaction latency instead under --rate an

Re: [HACKERS] pgbench progress report improvements - split 3 v2

2013-09-26 Thread Fabien COELHO
My feelings on the patch split haven't changed; your three bullet points call for four separate patches. Conflicting patches are bad, but dependent patches are okay; Indeed, this is the only solution if you do not want one patch. Note that it will not possible to backtrack one of the patch b

Re: [HACKERS] pgbench progress report improvements - split 3 v2

2013-09-25 Thread Noah Misch
On Tue, Sep 24, 2013 at 10:41:17PM +0200, Fabien COELHO wrote: > These changes are coupled because measures are changed, and their > reporting as well. Submitting separate patches for these different > features would result in conflicting or dependent patches, so I > wish to avoid that if possible.

Re: [HACKERS] pgbench progress report improvements - split 3 v2

2013-09-24 Thread Fabien COELHO
Split 3 of the initial submission, which actually deal with data measured and reported on stderr under various options. This version currently takes into account many comments by Noah Mish. In particular, the default "no report" behavior under benchmarking is not changed, although I really t

Re: [HACKERS] pgbench progress report improvements - split 3

2013-09-24 Thread Josh Berkus
On 09/22/2013 10:44 PM, Fabien COELHO wrote: > Due to the possibly repetitive table structure of the data, maybe CSV > would make sense as well. It is less extensible, but it is directly > usable by sqlite or pg. I'd be OK with CSV. At least I wouldn't be regexing the output. -- Josh Berkus Pos

Re: [HACKERS] pgbench progress report improvements - split 3

2013-09-23 Thread Fabien COELHO
Dear Peter, Split 3 of the initial submission, which actually deal with data measured and reported on stderr under various options. It seems this patch doesn't apply. Does it need the first two applied first? Oops. Indeed. The patch is fully independent of the two others. It was generated

Re: [HACKERS] pgbench progress report improvements - split 3

2013-09-23 Thread Peter Eisentraut
On 9/22/13 2:58 PM, Fabien wrote: > Split 3 of the initial submission, which actually deal with data > measured and reported on stderr under various options. It seems this patch doesn't apply. Does it need the first two applied first? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postg

Re: [HACKERS] pgbench progress report improvements - split 3

2013-09-22 Thread Fabien COELHO
Hello Josh, As long as you're hacking pgbench output, what about offering a JSON option instead of the text output? I'm working on automating pgbench performance testing, and having the output in a proper delimited format would be really helpful. I'm more a "grep | cut | ..." person, but I d

Re: [HACKERS] pgbench progress report improvements - split 3

2013-09-22 Thread Josh Berkus
Fabien, As long as you're hacking pgbench output, what about offering a JSON option instead of the text output? I'm working on automating pgbench performance testing, and having the output in a proper delimited format would be really helpful. -- Josh Berkus PostgreSQL Experts Inc. http://pgexpe

Re: [HACKERS] pgbench progress report improvements - split 3

2013-09-22 Thread Fabien
Split 3 of the initial submission, which actually deal with data measured and reported on stderr under various options. This version currently takes into account many comments by Noah Mish. In particular, the default "no report" behavior under benchmarking is not changed, although I really t