Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-16 Thread legrand legrand
> I'm instead attaching a v7 which removes the assert in pg_plan_query, and > modify pgss_planner_hook to also ignore queries without a query text, as > this > seems the best option. Ok, it was the second solution, go on ! -- Sent from: https://www.postgresql-archive.org/PostgreSQL-hackers-f19

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-16 Thread Julien Rouhaud
On Mon, Mar 16, 2020 at 01:34:11AM +, imai.yoshik...@fujitsu.com wrote: > On Sat, Mar 14, 2020 at 5:28 PM, Julien Rouhaud wrote: > > I don't think that's a correct assumption. I obviously didn't read all of > > citus extension, but it looks like what's happening is that they get > > generate

RE: Planning counters in pg_stat_statements (using pgss_store)

2020-03-16 Thread imai.yoshik...@fujitsu.com
On Mon, Mar 16, 2020 at 9:49 PM, Julien Rouhaud wrote: > On Mon, Mar 16, 2020 at 01:34:11AM +, imai.yoshik...@fujitsu.com > wrote: > > On Sat, Mar 14, 2020 at 5:28 PM, Julien Rouhaud wrote: > > > I don't think that's a correct assumption. I obviously didn't read > > > all of citus extension, b

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-20 Thread Sergei Kornilov
Hello I was inactive for a while... Sorry. >>  BTW, I recheck the patchset. >>  I think codes are ready for committer but should we modify the >> documentation? >>  {min,max,mean,stddev}_time is now obsoleted so it is better to modify it to >>  {min,max,mean,stddev}_exec_time and add {min,max,me

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-20 Thread Julien Rouhaud
On Fri, Mar 20, 2020 at 05:09:05PM +0300, Sergei Kornilov wrote: > Hello > > Yet another is missed in docs: total_time Oh good catch! I rechecked many time the field, and totally missed that the documentation is referring to the view, which has an additional column, and not the function. Attach

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-25 Thread Fujii Masao
On 2020/03/21 4:30, Julien Rouhaud wrote: On Fri, Mar 20, 2020 at 05:09:05PM +0300, Sergei Kornilov wrote: Hello Yet another is missed in docs: total_time Oh good catch! I rechecked many time the field, and totally missed that the documentation is referring to the view, which has an addit

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-25 Thread Julien Rouhaud
On Wed, Mar 25, 2020 at 10:09:37PM +0900, Fujii Masao wrote: > > On 2020/03/21 4:30, Julien Rouhaud wrote: > > On Fri, Mar 20, 2020 at 05:09:05PM +0300, Sergei Kornilov wrote: > > > Hello > > > > > > Yet another is missed in docs: total_time > > > > Oh good catch! I rechecked many time the fiel

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-25 Thread Sergei Kornilov
Hello > WAL usage patch [1] increments this version to 1_4 instead of 1_8. > I *guess* that's because previously this version was maintained > independently from pg_stat_statements' version. For example, > pg_stat_statements 1.4 seems to have used PGSS_V1_3. As far as I remember, this was my prop

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-25 Thread Fujii Masao
On 2020/03/26 2:17, Sergei Kornilov wrote: Hello WAL usage patch [1] increments this version to 1_4 instead of 1_8. I *guess* that's because previously this version was maintained independently from pg_stat_statements' version. For example, pg_stat_statements 1.4 seems to have used PGSS_V1_3

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-26 Thread Fujii Masao
On 2020/03/25 22:45, Julien Rouhaud wrote: On Wed, Mar 25, 2020 at 10:09:37PM +0900, Fujii Masao wrote: On 2020/03/21 4:30, Julien Rouhaud wrote: On Fri, Mar 20, 2020 at 05:09:05PM +0300, Sergei Kornilov wrote: Hello Yet another is missed in docs: total_time Oh good catch! I rechecked

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-26 Thread Julien Rouhaud
On Thu, Mar 26, 2020 at 08:08:35PM +0900, Fujii Masao wrote: > > On 2020/03/25 22:45, Julien Rouhaud wrote: > > On Wed, Mar 25, 2020 at 10:09:37PM +0900, Fujii Masao wrote: > > > + /* > > > + * We can't process the query if no query_text is provided, as > > > pgss_store > > > + * needs it. We

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-27 Thread Julien Rouhaud
On Thu, Mar 26, 2020 at 02:22:47PM +0100, Julien Rouhaud wrote: > On Thu, Mar 26, 2020 at 08:08:35PM +0900, Fujii Masao wrote: > > > > Here are other comments. > > > > - if (jstate) > > + if (kind == PGSS_JUMBLE) > > > > Why is PGSS_JUMBLE necessary? ISTM that we can still us

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-27 Thread Fujii Masao
On 2020/03/27 19:00, Julien Rouhaud wrote: On Thu, Mar 26, 2020 at 02:22:47PM +0100, Julien Rouhaud wrote: On Thu, Mar 26, 2020 at 08:08:35PM +0900, Fujii Masao wrote: Here are other comments. - if (jstate) + if (kind == PGSS_JUMBLE) Why is PGSS_JUMBLE necessar

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-27 Thread Fujii Masao
On 2020/03/27 19:00, Julien Rouhaud wrote: On Thu, Mar 26, 2020 at 02:22:47PM +0100, Julien Rouhaud wrote: On Thu, Mar 26, 2020 at 08:08:35PM +0900, Fujii Masao wrote: Here are other comments. - if (jstate) + if (kind == PGSS_JUMBLE) Why is PGSS_JUMBLE necessary

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-27 Thread Julien Rouhaud
On Fri, Mar 27, 2020 at 12:02 PM Fujii Masao wrote: > > On 2020/03/27 19:00, Julien Rouhaud wrote: > > On Thu, Mar 26, 2020 at 02:22:47PM +0100, Julien Rouhaud wrote: > >> On Thu, Mar 26, 2020 at 08:08:35PM +0900, Fujii Masao wrote: > >>> > >>> Here are other comments. > >>> > >>> - if (

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-27 Thread Julien Rouhaud
On Fri, Mar 27, 2020 at 2:01 PM Fujii Masao wrote: > > On 2020/03/27 19:00, Julien Rouhaud wrote: > > On Thu, Mar 26, 2020 at 02:22:47PM +0100, Julien Rouhaud wrote: > >> On Thu, Mar 26, 2020 at 08:08:35PM +0900, Fujii Masao wrote: > >>> > >>> Here are other comments. > >>> > >>> - if (j

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-28 Thread Julien Rouhaud
On Fri, Mar 27, 2020 at 03:42:50PM +0100, Julien Rouhaud wrote: > On Fri, Mar 27, 2020 at 2:01 PM Fujii Masao > wrote: > > > > > So what I'd like to say is that the information that users are interested > > in would vary on each situation and case. At least for me it seems > > enough for pgss to

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-29 Thread Fujii Masao
On 2020/03/29 15:15, Julien Rouhaud wrote: On Fri, Mar 27, 2020 at 03:42:50PM +0100, Julien Rouhaud wrote: On Fri, Mar 27, 2020 at 2:01 PM Fujii Masao wrote: So what I'd like to say is that the information that users are interested in would vary on each situation and case. At least for

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-30 Thread Julien Rouhaud
On Mon, Mar 30, 2020 at 01:56:43PM +0900, Fujii Masao wrote: > > > On 2020/03/29 15:15, Julien Rouhaud wrote: > > On Fri, Mar 27, 2020 at 03:42:50PM +0100, Julien Rouhaud wrote: > > > On Fri, Mar 27, 2020 at 2:01 PM Fujii Masao > > > wrote: > > > > > > > > > > > So what I'd like to say is tha

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-30 Thread Fujii Masao
On 2020/03/30 17:03, Julien Rouhaud wrote: On Mon, Mar 30, 2020 at 01:56:43PM +0900, Fujii Masao wrote: On 2020/03/29 15:15, Julien Rouhaud wrote: On Fri, Mar 27, 2020 at 03:42:50PM +0100, Julien Rouhaud wrote: On Fri, Mar 27, 2020 at 2:01 PM Fujii Masao wrote: So what I'd like to s

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-30 Thread Julien Rouhaud
On Mon, Mar 30, 2020 at 6:36 PM Fujii Masao wrote: > > On 2020/03/30 17:03, Julien Rouhaud wrote: > > On Mon, Mar 30, 2020 at 01:56:43PM +0900, Fujii Masao wrote: > >> > >> > >> On 2020/03/29 15:15, Julien Rouhaud wrote: > >>> On Fri, Mar 27, 2020 at 03:42:50PM +0100, Julien Rouhaud wrote: >

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-30 Thread Fujii Masao
On 2020/03/31 3:16, Julien Rouhaud wrote: On Mon, Mar 30, 2020 at 6:36 PM Fujii Masao wrote: On 2020/03/30 17:03, Julien Rouhaud wrote: On Mon, Mar 30, 2020 at 01:56:43PM +0900, Fujii Masao wrote: On 2020/03/29 15:15, Julien Rouhaud wrote: On Fri, Mar 27, 2020 at 03:42:50PM +0100, Juli

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-30 Thread Julien Rouhaud
On Tue, Mar 31, 2020 at 12:21:43PM +0900, Fujii Masao wrote: > > On 2020/03/31 3:16, Julien Rouhaud wrote: > > On Mon, Mar 30, 2020 at 6:36 PM Fujii Masao > > wrote: > > > > > > While testing the patched pgss, I found that the patched version > > > may track the statements that the original ver

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-31 Thread Fujii Masao
On 2020/03/31 15:03, Julien Rouhaud wrote: On Tue, Mar 31, 2020 at 12:21:43PM +0900, Fujii Masao wrote: On 2020/03/31 3:16, Julien Rouhaud wrote: On Mon, Mar 30, 2020 at 6:36 PM Fujii Masao wrote: While testing the patched pgss, I found that the patched version may track the statements t

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-31 Thread Julien Rouhaud
On Tue, Mar 31, 2020 at 04:10:47PM +0900, Fujii Masao wrote: > > > On 2020/03/31 15:03, Julien Rouhaud wrote: > > On Tue, Mar 31, 2020 at 12:21:43PM +0900, Fujii Masao wrote: > > > > > > On 2020/03/31 3:16, Julien Rouhaud wrote: > > > > On Mon, Mar 30, 2020 at 6:36 PM Fujii Masao > > > > wrote

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-31 Thread Fujii Masao
On 2020/03/31 16:33, Julien Rouhaud wrote: On Tue, Mar 31, 2020 at 04:10:47PM +0900, Fujii Masao wrote: On 2020/03/31 15:03, Julien Rouhaud wrote: On Tue, Mar 31, 2020 at 12:21:43PM +0900, Fujii Masao wrote: On 2020/03/31 3:16, Julien Rouhaud wrote: On Mon, Mar 30, 2020 at 6:36 PM Fujii

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-31 Thread Julien Rouhaud
On Wed, Apr 01, 2020 at 02:43:10AM +0900, Fujii Masao wrote: > > > On 2020/03/31 16:33, Julien Rouhaud wrote: > > > > v12 attached! > > Thanks for updating the patch! The patch looks good to me. > > I applied minor and cosmetic changes into the patch. Attached is > the updated version of the patch.

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-04-01 Thread Fujii Masao
On 2020/04/01 3:42, Julien Rouhaud wrote: On Wed, Apr 01, 2020 at 02:43:10AM +0900, Fujii Masao wrote: On 2020/03/31 16:33, Julien Rouhaud wrote: v12 attached! Thanks for updating the patch! The patch looks good to me. I applied minor and cosmetic changes into the patch. Attached is th

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-04-01 Thread Fujii Masao
On 2020/04/01 18:19, Fujii Masao wrote: On 2020/04/01 3:42, Julien Rouhaud wrote: On Wed, Apr 01, 2020 at 02:43:10AM +0900, Fujii Masao wrote: On 2020/03/31 16:33, Julien Rouhaud wrote: v12 attached! Thanks for updating the patch! The patch looks good to me. I applied minor and cosm

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-04-02 Thread legrand legrand
Fujii Masao-4 wrote > On 2020/04/01 18:19, Fujii Masao wrote: > > Finally I pushed the patch! > Many thanks for all involved in this patch! > > As a remaining TODO item, I'm thinking that the document would need to > be improved. For example, previously the query was not stored in pgss > when it

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-04-03 Thread Julien Rouhaud
On Thu, Apr 02, 2020 at 01:04:28PM -0700, legrand legrand wrote: > Fujii Masao-4 wrote > > On 2020/04/01 18:19, Fujii Masao wrote: > > > > Finally I pushed the patch! > > Many thanks for all involved in this patch! > > > > As a remaining TODO item, I'm thinking that the document would need to > >

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-04-08 Thread Fujii Masao
On 2020/04/03 16:26, Julien Rouhaud wrote: On Thu, Apr 02, 2020 at 01:04:28PM -0700, legrand legrand wrote: Fujii Masao-4 wrote On 2020/04/01 18:19, Fujii Masao wrote: Finally I pushed the patch! Many thanks for all involved in this patch! As a remaining TODO item, I'm thinking that the doc

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-04-08 Thread Julien Rouhaud
On Wed, Apr 08, 2020 at 05:37:27PM +0900, Fujii Masao wrote: > > > On 2020/04/03 16:26, Julien Rouhaud wrote: > > On Thu, Apr 02, 2020 at 01:04:28PM -0700, legrand legrand wrote: > > > Fujii Masao-4 wrote > > > > On 2020/04/01 18:19, Fujii Masao wrote: > > > > > > > > Finally I pushed the patch!

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-04-08 Thread legrand legrand
Fujii Masao-4 wrote > On 2020/04/03 16:26 > [...] >> >> "Note that planning and execution statistics are updated only at their >> respective end phase, and only for successful operations. >> For example the execution counters of a long running query >> will only be updated at the execution end, wi

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-04-08 Thread Fujii Masao
On 2020/04/08 18:31, Julien Rouhaud wrote: On Wed, Apr 08, 2020 at 05:37:27PM +0900, Fujii Masao wrote: On 2020/04/03 16:26, Julien Rouhaud wrote: On Thu, Apr 02, 2020 at 01:04:28PM -0700, legrand legrand wrote: Fujii Masao-4 wrote On 2020/04/01 18:19, Fujii Masao wrote: Finally I pushe

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-04-08 Thread Fujii Masao
On 2020/04/08 21:32, legrand legrand wrote: Fujii Masao-4 wrote On 2020/04/03 16:26 [...] "Note that planning and execution statistics are updated only at their respective end phase, and only for successful operations. For example the execution counters of a long running query will only be

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-04-09 Thread Julien Rouhaud
On Thu, Apr 9, 2020 at 5:59 AM Fujii Masao wrote: > > > > On 2020/04/08 18:31, Julien Rouhaud wrote: > > On Wed, Apr 08, 2020 at 05:37:27PM +0900, Fujii Masao wrote: > >> > >> > >> On 2020/04/03 16:26, Julien Rouhaud wrote: > >>> On Thu, Apr 02, 2020 at 01:04:28PM -0700, legrand legrand wrote: > >

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-04-09 Thread Fujii Masao
On 2020/04/09 22:31, Julien Rouhaud wrote: On Thu, Apr 9, 2020 at 5:59 AM Fujii Masao wrote: On 2020/04/08 18:31, Julien Rouhaud wrote: On Wed, Apr 08, 2020 at 05:37:27PM +0900, Fujii Masao wrote: On 2020/04/03 16:26, Julien Rouhaud wrote: On Thu, Apr 02, 2020 at 01:04:28PM -0700, le

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-02-12 Thread Sergei Kornilov
Hello Thank you for picking this up! Did you register patch in CF app? I did not found entry. Currently we have pg_stat_statements 1.7 version and this patch does not apply... My fast and small view: > - errmsg("could not read file \"%s\": %m", > + err

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-02-12 Thread legrand legrand
Hi Sergei, Thank you for this review ! >Did you register patch in CF app? I did not found entry. I think it is related to https://commitfest.postgresql.org/16/1373/ but I don't know how to link it with. Yes, there are many things to improve, but before to go deeper, I would like to know if tha

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-02-12 Thread Sergei Kornilov
Hi > I think it is related to https://commitfest.postgresql.org/16/1373/ > but I don't know how to link it with. You can just add new entry in open commitfest and then attach previous thread. This is usual way for pick up old patches. For example, as i did here: https://commitfest.postgresql.or

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-02-14 Thread legrand legrand
Thank you Sergei for your comments, > Did you register patch in CF app? I did not found entry. created today: https://commitfest.postgresql.org/22/1999/ > Currently we have pg_stat_statements 1.7 version and this patch does not > apply... will rebase and create a 1.8 version > -

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-02-14 Thread Sergei Kornilov
Hi >>  +#define PG_STAT_STATEMENTS_COLS_V1_4 25 > > I thought it was needed when adding new columns, isn't it ? Yes, this is needed. I mean it should be PG_STAT_STATEMENTS_COLS_V1_8: because such change was made for 1.8 pg_stat_statements version. Same thing for other version-specific places.

Re: Planning counters in pg_stat_statements (using pgss_store)

2021-07-25 Thread Tom Lane
[ blast from the past department ] Fujii Masao writes: > Finally I pushed the patch! > Many thanks for all involved in this patch! It turns out that the regression test outputs from this patch are unstable under debug_discard_caches (nee CLOBBER_CACHE_ALWAYS). You can easily check this in HEAD o

Re: Planning counters in pg_stat_statements (using pgss_store)

2021-07-25 Thread Julien Rouhaud
On Sun, Jul 25, 2021 at 12:03:25PM -0400, Tom Lane wrote: > The cause of the failure of course is that cache clobbering includes > plan cache clobbering, so that the prepared statement's plan is > remade each time it's used, not only twice as the test expects. > However, remembering that cache flu

Re: Planning counters in pg_stat_statements (using pgss_store)

2021-07-25 Thread Tom Lane
Julien Rouhaud writes: > On Sun, Jul 25, 2021 at 12:03:25PM -0400, Tom Lane wrote: >> So AFAICS this test is inherently unstable and there is no code bug >> to be fixed. We could drop the "plans" column from this query, or >> print something approximate like "plans > 0 AND plans <= calls". >> Tho

Re: Planning counters in pg_stat_statements (using pgss_store)

2021-07-25 Thread Julien Rouhaud
Le lun. 26 juil. 2021 à 00:59, Tom Lane a écrit : > Julien Rouhaud writes: > > On Sun, Jul 25, 2021 at 12:03:25PM -0400, Tom Lane wrote: > > > Would it be worth to split the query for the prepared statement row vs > the rest > > to keep the full "plans" coverage when possible? > > +1, the same

Re: Planning counters in pg_stat_statements (using pgss_store)

2021-07-25 Thread Andrew Dunstan
On 7/25/21 12:03 PM, Tom Lane wrote: > > So AFAICS this test is inherently unstable and there is no code bug > to be fixed. We could drop the "plans" column from this query, or > print something approximate like "plans > 0 AND plans <= calls". > Thoughts? > Is that likely to tell us anything ve

Re: Planning counters in pg_stat_statements (using pgss_store)

2021-07-25 Thread Tom Lane
Andrew Dunstan writes: > On 7/25/21 12:03 PM, Tom Lane wrote: >> So AFAICS this test is inherently unstable and there is no code bug >> to be fixed. We could drop the "plans" column from this query, or >> print something approximate like "plans > 0 AND plans <= calls". >> Thoughts? > Is that lik

Re: Planning counters in pg_stat_statements (using pgss_store)

2021-07-25 Thread Julien Rouhaud
On Mon, Jul 26, 2021 at 01:08:08AM +0800, Julien Rouhaud wrote: > Le lun. 26 juil. 2021 à 00:59, Tom Lane a écrit : > > > Julien Rouhaud writes: > > > On Sun, Jul 25, 2021 at 12:03:25PM -0400, Tom Lane wrote: > > > > > > Would it be worth to split the query for the prepared statement row vs > >

Re: Planning counters in pg_stat_statements (using pgss_store)

2021-07-25 Thread Tom Lane
Julien Rouhaud writes: > I attach a patch that splits the test and add a comment explaining the > boundaries for the new query. > Checked with and without forced invalidations. Pushed with a little cosmetic fooling-about. regards, tom lane

Re: Planning counters in pg_stat_statements (using pgss_store)

2021-07-25 Thread Julien Rouhaud
On Sun, Jul 25, 2021 at 11:26:02PM -0400, Tom Lane wrote: > Julien Rouhaud writes: > > I attach a patch that splits the test and add a comment explaining the > > boundaries for the new query. > > Checked with and without forced invalidations. > > Pushed with a little cosmetic fooling-about. Than

Re: Planning counters in pg_stat_statements (using pgss_store)

2021-07-25 Thread Fujii Masao
On 2021/07/26 12:32, Julien Rouhaud wrote: On Sun, Jul 25, 2021 at 11:26:02PM -0400, Tom Lane wrote: Julien Rouhaud writes: I attach a patch that splits the test and add a comment explaining the boundaries for the new query. Checked with and without forced invalidations. Pushed with a lit

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-09-04 Thread Sergei Kornilov
Hello I think the most important question for this topic is performance penalty. It was a long story, first test on my desktop was too volatile. I setup separate PC with DB only and test few cases. PC spec: 2-core Intel Core 2 Duo E6550, 4GB ram, mechanical HDD All tests on top 7dedfd22b79822b7f

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-09-06 Thread Tomas Vondra
On Fri, Sep 06, 2019 at 04:19:16PM +0200, Tomas Vondra wrote: FWIW I've done some benchmarking on this too, with a single pgbench client running select-only test on a tiny database, in different modes (simple, extended, prepared). I've done that on two systems with different CPUs (spreadsheet wi

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-09-06 Thread Tomas Vondra
On Wed, Sep 04, 2019 at 07:19:47PM +0300, Sergei Kornilov wrote: ... Results: test | mode | average_tps | degradation_perc --+--+-+-- head_no_pgss | extended | 13816 |1.000 patch_not_loaded

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-09-08 Thread Imai Yoshikazu
Hello On 2019/09/06 23:19, Tomas Vondra wrote: > On Wed, Sep 04, 2019 at 07:19:47PM +0300, Sergei Kornilov wrote: >> >> ... >> >> Results: >> >>     test |   mode   | average_tps | degradation_perc >> --+--+-+-- >> head_no_pgss   

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-09-10 Thread Julien Rouhaud
On Fri, Sep 6, 2019 at 4:19 PM Tomas Vondra wrote: > > On Wed, Sep 04, 2019 at 07:19:47PM +0300, Sergei Kornilov wrote: > > > > ... > > > >Results: > > > > test | mode | average_tps | degradation_perc > >--+--+-+-- > > hea

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-09-10 Thread Julien Rouhaud
Hello, On Sun, Sep 8, 2019 at 11:45 AM Imai Yoshikazu wrote: > > I also saw the codes and have one comment. Thanks for looking at this patch! > [0002 patch] > In pgss_planner_hook: > > + /* calc differences of buffer counters. */ > + bufusage = compute_buffer_counter

RE: Planning counters in pg_stat_statements (using pgss_store)

2019-03-22 Thread legrand legrand
Hi, Here is a rebased and corrected version . Columns naming has not been modified, I would propose to change it to: - plans: ok - planning_time --> plan_time - calls: ok - total_time --> exec_time - {min,max,mean,stddev}_time: ok - new total_time (being the sum of plan_time and exec_time)

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-03-23 Thread Julien Rouhaud
On Fri, Mar 22, 2019 at 11:46 PM legrand legrand wrote: > > Here is a rebased and corrected version . This patch has multiple trailing whitespace, indent and coding style issues. You should consider running pg_indent before submitting a patch. I attach the diff after running pgindent if you wan

RE: Planning counters in pg_stat_statements (using pgss_store)

2019-03-23 Thread legrand legrand
> This patch has multiple trailing whitespace, indent and coding style > issues. You should consider running pg_indent before submitting a > patch. I attach the diff after running pgindent if you want more > details about the various issues. fixed > - * Track statement execution times acro

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-03-24 Thread Julien Rouhaud
On Sat, Mar 23, 2019 at 11:08 PM legrand legrand wrote: > > > This patch has multiple trailing whitespace, indent and coding style > > issues. You should consider running pg_indent before submitting a > > patch. I attach the diff after running pgindent if you want more > > details about the vari

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-03-24 Thread Julien Rouhaud
On Sun, Mar 24, 2019 at 11:24 AM Julien Rouhaud wrote: > > > there are 4 tests to check if planning_time is zero or not, it's quite > > > messy. Could you refactor the code to avoid so many tests? It would > > > probably be useful to add some asserts to check that we don't provide > > > both pla

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-03-25 Thread legrand legrand
As there are now 3 locking times on pgss hash struct, one day or an other, somebody will ask for a GUC to disable this feature (to be able to run pgss unchanged with only one lock as today). With this GUC, pgss_store should be able to store the query text and accumulated execution duration in th

RE: Planning counters in pg_stat_statements (using pgss_store)

2019-03-26 Thread legrand legrand
here is a new version: - "track_planning" GUC added to permit to keep previous behavior unchanged - columns names have been changed / added: total_plan_time, total_exec_time, total_time - trailing whitespaces and comments wider than 80 characters not fixed

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-03-27 Thread Julien Rouhaud
On Wed, Mar 27, 2019 at 12:21 AM legrand legrand wrote: > > here is a new version: > > - "track_planning" GUC added > to permit to keep previous behavior unchanged good > - trailing whitespaces and comments wider than 80 characters > not fixed why? In case it's not cle

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-03-27 Thread legrand legrand
>> - trailing whitespaces and comments wider than 80 characters >> not fixed > why? In case it's not clear, I'm talking about the .c file, not the > regression tests. I work on a poor msys install on windows 7, where perl is broken ;o( So no pgindent available. Will fix that later,

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-03-27 Thread Julien Rouhaud
On Wed, Mar 27, 2019 at 9:36 PM legrand legrand wrote: > > >> - trailing whitespaces and comments wider than 80 characters > >> not fixed > > > why? In case it's not clear, I'm talking about the .c file, not the > > regression tests. > > I work on a poor msys install on windows 7, wh

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-03-27 Thread legrand legrand
Julien Rouhaud wrote > On Wed, Mar 27, 2019 at 9:36 PM legrand legrand > < > legrand_legrand@ > > wrote: >> >> >> - trailing whitespaces and comments wider than 80 characters >> >> not fixed >> >> > why? In case it's not clear, I'm talking about the .c file, not the >> > regression

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-03-28 Thread Sergei Kornilov
Hi >>  Ok, but keep in mind that this is the last commitfest for pg12, and >>  there are only 4 days left. Will you have time to take care of it, or >>  do you need help on it? > > Oups, sorry, I won't have time nor knowledge to finish in time ;o( > Any help is welcome ! No need to rush, this pat

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-03-28 Thread Julien Rouhaud
On Thu, Mar 28, 2019 at 8:45 AM Sergei Kornilov wrote: > > >> Ok, but keep in mind that this is the last commitfest for pg12, and > >> there are only 4 days left. Will you have time to take care of it, or > >> do you need help on it? > > > > Oups, sorry, I won't have time nor knowledge to finis

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-03-28 Thread Julien Rouhaud
On Thu, Mar 28, 2019 at 9:48 AM Julien Rouhaud wrote: > > On Thu, Mar 28, 2019 at 8:45 AM Sergei Kornilov wrote: > > > > >> Ok, but keep in mind that this is the last commitfest for pg12, and > > >> there are only 4 days left. Will you have time to take care of it, or > > >> do you need help o

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-04-01 Thread legrand legrand
Hi, I have played with this patch, it works fine. rem the last position of the "new" total_time column is confusing +CREATE VIEW pg_stat_statements AS + SELECT *, total_plan_time + total_exec_time AS total_time +FROM pg_stat_statements(true); I wanted to perform some benchmark between those

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-04-01 Thread Julien Rouhaud
On Mon, Apr 1, 2019 at 10:35 PM legrand legrand wrote: > > I have played with this patch, it works fine. Thanks for testing! > rem the last position of the "new" total_time column is confusing > +CREATE VIEW pg_stat_statements AS > + SELECT *, total_plan_time + total_exec_time AS total_time > +

RE: Planning counters in pg_stat_statements (using pgss_store)

2019-04-02 Thread legrand legrand
Hi, >> >> case avg_tps pct_diff >> 089 278 -- >> 188 745 0,6% >> 288 282 1,1% >> 386 660 2,9% >> >> This means that even in this extrem test case, the worst degradation is less >> than 3% >> (this overhead can be removed using pg_stat_statements.track_pla

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-02-28 Thread legrand legrand
Hi Julien, >> But I would have prefered this new feature to work the same way with or >> without track_planning activated ;o( > Definitely, but fixing the issue in pgss (ignoring planner calls when > we don't have a query text) means that pgss won't give an exhaustive > view of activity anymore,

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-01 Thread Julien Rouhaud
Hi, On Fri, Feb 28, 2020 at 4:06 PM legrand legrand wrote: > > It seems IVM team does not consider this point as a priority ... Well, IVM is a big project and I agree that fixing this issue isn't the most urgent one, especially since there's no guarantee that this pgss planning patch will be com

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-01 Thread legrand legrand
>> I like the idea of adding a check for a non-zero queryId in the new >> pgss_planner_hook() (zero queryid shouldn't be reserved for >> utility_statements ?). > Some assert hit later, I can say that it's not always true. For > instance a CREATE TABLE AS won't run parse analysis for the underlyin

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-02 Thread Julien Rouhaud
On Sun, Mar 1, 2020 at 3:55 PM legrand legrand wrote: > > >> I like the idea of adding a check for a non-zero queryId in the new > >> pgss_planner_hook() (zero queryid shouldn't be reserved for > >> utility_statements ?). > > > Some assert hit later, I can say that it's not always true. For > > i

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-02 Thread legrand legrand
Julien Rouhaud wrote > On Sun, Mar 1, 2020 at 3:55 PM legrand legrand > < > legrand_legrand@ > > wrote: >> >> >> I like the idea of adding a check for a non-zero queryId in the new >> >> pgss_planner_hook() (zero queryid shouldn't be reserved for >> >> utility_statements ?). >> >> > Some assert h

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-02 Thread Julien Rouhaud
On Mon, Mar 2, 2020 at 1:01 PM legrand legrand wrote: > > Julien Rouhaud wrote > > On Sun, Mar 1, 2020 at 3:55 PM legrand legrand > > < > > > legrand_legrand@ > > > > wrote: > >> > >> >> I like the idea of adding a check for a non-zero queryId in the new > >> >> pgss_planner_hook() (zero queryid s

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-05 Thread legrand legrand
Never mind ... Please consider PG13 shortest path ;o) My one is parse->queryId != UINT64CONST(0) in pgss_planner_hook(). It fixes IVM problem (verified), and keep CTAS equal to pgss without planning counters (verified too). Regards PAscal -- Sent from: https://www.postgresql-archive.org/

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-09 Thread Julien Rouhaud
On Thu, Mar 05, 2020 at 01:26:19PM -0700, legrand legrand wrote: > Please consider PG13 shortest path ;o) > > My one is parse->queryId != UINT64CONST(0) in pgss_planner_hook(). > It fixes IVM problem (verified), > and keep CTAS equal to pgss without planning counters (verified too). I still disag

RE: Planning counters in pg_stat_statements (using pgss_store)

2020-03-11 Thread imai.yoshik...@fujitsu.com
Hi Julien, On Mon, Mar 9, 2020 at 10:32 AM, Julien Rouhaud wrote: > On Thu, Mar 05, 2020 at 01:26:19PM -0700, legrand legrand wrote: > > Please consider PG13 shortest path ;o) > > > > My one is parse->queryId != UINT64CONST(0) in pgss_planner_hook(). > > It fixes IVM problem (verified), > > and k

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-11 Thread Julien Rouhaud
Hi Imai-san, On Thu, Mar 12, 2020 at 05:28:38AM +, imai.yoshik...@fujitsu.com wrote: > Hi Julien, > > On Mon, Mar 9, 2020 at 10:32 AM, Julien Rouhaud wrote: > > On Thu, Mar 05, 2020 at 01:26:19PM -0700, legrand legrand wrote: > > > Please consider PG13 shortest path ;o) > > > > > > My one is

RE: Planning counters in pg_stat_statements (using pgss_store)

2020-03-12 Thread imai.yoshik...@fujitsu.com
On Thu, Mar 12, 2020 at 6:31 AM, Julien Rouhaud wrote: > On Thu, Mar 12, 2020 at 05:28:38AM +, imai.yoshik...@fujitsu.com wrote: > > Hi Julien, > > > > On Mon, Mar 9, 2020 at 10:32 AM, Julien Rouhaud wrote: > > > On Thu, Mar 05, 2020 at 01:26:19PM -0700, legrand legrand wrote: > > > > Please co

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-12 Thread Julien Rouhaud
On Thu, Mar 12, 2020 at 10:19 AM imai.yoshik...@fujitsu.com wrote: > > I'll summary code review of this thread. Thanks for the summary! I just have some minor comments > [Performance] > > If track_planning is not enabled, performance will drop 0.2-0.6% which can be > ignored. If track_planning

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-12 Thread Marco Slot
On Thu, Mar 12, 2020 at 11:31 AM Julien Rouhaud wrote: > There's at least the current version of IVM patchset that lacks the > querytext. Looking at various extensions, I see that pg_background > and pglogical call pg_plan_query internally but shouldn't have any > issue providing the query text.

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-12 Thread Julien Rouhaud
On Thu, Mar 12, 2020 at 1:11 PM Marco Slot wrote: > > On Thu, Mar 12, 2020 at 11:31 AM Julien Rouhaud wrote: > > There's at least the current version of IVM patchset that lacks the > > querytext. Looking at various extensions, I see that pg_background > > and pglogical call pg_plan_query interna

RE: Planning counters in pg_stat_statements (using pgss_store)

2020-03-12 Thread imai.yoshik...@fujitsu.com
On Thu, Mar 12, 2020 at 10:31 AM, Julien Rouhaud wrote: > > * bufusage still only counts the buffer usage during executor. > > > > Now we have the ability to count the buffer usage during planner but we > keep > > the bufusage count the buffer usage during executor for now. > > The bufusage sh

RE: Planning counters in pg_stat_statements (using pgss_store)

2020-03-12 Thread imai.yoshik...@fujitsu.com
On Thu, Mar 12, 2020 at 6:37 PM, Julien Rouhaud wrote: > On Thu, Mar 12, 2020 at 1:11 PM Marco Slot wrote: > > On Thu, Mar 12, 2020 at 11:31 AM Julien Rouhaud > wrote: > > > There's at least the current version of IVM patchset that lacks the > > > querytext. Looking at various extensions, I see

RE: Planning counters in pg_stat_statements (using pgss_store)

2020-03-14 Thread legrand legrand
imai.yoshik...@fujitsu.com wrote > On Thu, Mar 12, 2020 at 6:37 PM, Julien Rouhaud wrote: >> On Thu, Mar 12, 2020 at 1:11 PM Marco Slot < > marco.slot@ > > wrote: >> > On Thu, Mar 12, 2020 at 11:31 AM Julien Rouhaud < > rjuju123@ > > >> wrote: >> > > There's at least the current version of IVM

RE: Planning counters in pg_stat_statements (using pgss_store)

2020-03-14 Thread legrand legrand
> I don't know it is useful but there are also codes that avoid an error when > sourceText is NULL. > executor_errposition(EState *estate, int location) > { > ... >/* Can't do anything if source text is not available */ >if (estate == NULL || estate->es_sourceText == NULL) > } or may

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-14 Thread Julien Rouhaud
On Sat, Mar 14, 2020 at 03:04:00AM -0700, legrand legrand wrote: > imai.yoshik...@fujitsu.com wrote > > On Thu, Mar 12, 2020 at 6:37 PM, Julien Rouhaud wrote: > >> That's very interesting feedback, thanks! I'm not a fan of giving a way > >> for > >> queries to say that they want to be ignored by p

RE: Planning counters in pg_stat_statements (using pgss_store)

2020-03-15 Thread imai.yoshik...@fujitsu.com
On Sat, Mar 14, 2020 at 5:28 PM, Julien Rouhaud wrote: > On Sat, Mar 14, 2020 at 03:04:00AM -0700, legrand legrand wrote: > > imai.yoshik...@fujitsu.com wrote > > > On Thu, Mar 12, 2020 at 6:37 PM, Julien Rouhaud wrote: > > >> That's very interesting feedback, thanks! I'm not a fan of giving a way

RE: Planning counters in pg_stat_statements (using pgss_store)

2019-11-07 Thread imai.yoshik...@fujitsu.com
On Tue, Sept 10, 2019 at 11:27 PM, Julien Rouhaud wrote: > > [0002 patch] > > In pgss_planner_hook: > > > > + /* calc differences of buffer counters. */ > > + bufusage = compute_buffer_counters(bufusage_start, > > pgBufferUsage); > > + > > + /* > > +

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-11-08 Thread Julien Rouhaud
On Fri, Nov 8, 2019 at 5:35 AM imai.yoshik...@fujitsu.com wrote: > > On Tue, Sept 10, 2019 at 11:27 PM, Julien Rouhaud wrote: > > > [0002 patch] > > > In pgss_planner_hook: > > > > > > + /* calc differences of buffer counters. */ > > > + bufusage = compute_buffer_counte

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-11-09 Thread Julien Rouhaud
On Fri, Nov 8, 2019 at 3:31 PM Julien Rouhaud wrote: > > On Fri, Nov 8, 2019 at 5:35 AM imai.yoshik...@fujitsu.com > wrote: > > > > On Tue, Sept 10, 2019 at 11:27 PM, Julien Rouhaud wrote: > > > > [0002 patch] > > > > In pgss_planner_hook: > > > > > > > > + /* calc differences of bu

RE: Planning counters in pg_stat_statements (using pgss_store)

2019-11-11 Thread imai.yoshik...@fujitsu.com
On Sat, Nov 9, 2019 at 1:36 PM, Julien Rouhaud wrote: > On Fri, Nov 8, 2019 at 3:31 PM Julien Rouhaud wrote: > > > > On Fri, Nov 8, 2019 at 5:35 AM imai.yoshik...@fujitsu.com > > wrote: > > > > > > On Tue, Sept 10, 2019 at 11:27 PM, Julien Rouhaud wrote: > > > > > [0002 patch] > > > > > In pgss_p

  1   2   >