Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-27 Thread Amit Kapila
On Wed, Oct 28, 2020 at 5:26 AM Tomas Vondra wrote: > > On Tue, Oct 27, 2020 at 06:19:42PM -0500, Justin Pryzby wrote: > >On Tue, Oct 27, 2020 at 09:17:43AM +0530, Amit Kapila wrote: > >> On Tue, Oct 27, 2020 at 8:51 AM Justin Pryzby wrote: > >> > > >> > On Fri, Oct 23, 2020 at 10:45:34AM +0530,

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-27 Thread Tomas Vondra
On Tue, Oct 27, 2020 at 06:19:42PM -0500, Justin Pryzby wrote: On Tue, Oct 27, 2020 at 09:17:43AM +0530, Amit Kapila wrote: On Tue, Oct 27, 2020 at 8:51 AM Justin Pryzby wrote: > > On Fri, Oct 23, 2020 at 10:45:34AM +0530, Amit Kapila wrote: > > On Fri, Oct 23, 2020 at 8:59 AM Amit Kapila wrot

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-27 Thread Justin Pryzby
On Tue, Oct 27, 2020 at 09:17:43AM +0530, Amit Kapila wrote: > On Tue, Oct 27, 2020 at 8:51 AM Justin Pryzby wrote: > > > > On Fri, Oct 23, 2020 at 10:45:34AM +0530, Amit Kapila wrote: > > > On Fri, Oct 23, 2020 at 8:59 AM Amit Kapila > > > wrote: > > > > > > While updating the streaming stats p

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-26 Thread Amit Kapila
On Tue, Oct 27, 2020 at 8:51 AM Justin Pryzby wrote: > > On Fri, Oct 23, 2020 at 10:45:34AM +0530, Amit Kapila wrote: > > On Fri, Oct 23, 2020 at 8:59 AM Amit Kapila wrote: > > > > While updating the streaming stats patch, it occurred to me that we > > can write a better description spill_bytes a

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-26 Thread Justin Pryzby
On Fri, Oct 23, 2020 at 10:45:34AM +0530, Amit Kapila wrote: > On Fri, Oct 23, 2020 at 8:59 AM Amit Kapila wrote: > > > > On Fri, Oct 23, 2020 at 7:42 AM Masahiko Sawada > > wrote: > > > > > > On Thu, 22 Oct 2020 at 20:34, Amit Kapila wrote: > > > > > > > > > > I have modified the description of

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-22 Thread Amit Kapila
On Fri, Oct 23, 2020 at 8:59 AM Amit Kapila wrote: > > On Fri, Oct 23, 2020 at 7:42 AM Masahiko Sawada > wrote: > > > > On Thu, 22 Oct 2020 at 20:34, Amit Kapila wrote: > > > > > > > > I have modified the description of spill_count and spill_txns to make > > > > > things clear. Any suggestions?

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-22 Thread Amit Kapila
On Fri, Oct 23, 2020 at 7:42 AM Masahiko Sawada wrote: > > On Thu, 22 Oct 2020 at 20:34, Amit Kapila wrote: > > > > > > I have modified the description of spill_count and spill_txns to make > > > > things clear. Any suggestions? > > > > > > Thank you for the patch. > > > > > > -logical de

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-22 Thread Masahiko Sawada
On Thu, 22 Oct 2020 at 20:34, Amit Kapila wrote: > > On Thu, Oct 22, 2020 at 4:09 PM Masahiko Sawada > wrote: > > > > On Wed, 21 Oct 2020 at 12:56, Amit Kapila wrote: > > > > > > On Tue, Oct 13, 2020 at 10:33 AM Amit Kapila > > > wrote: > > > > > > > > On Tue, Oct 13, 2020 at 10:21 AM Tom Lane

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-22 Thread Amit Kapila
On Thu, Oct 22, 2020 at 4:09 PM Masahiko Sawada wrote: > > On Wed, 21 Oct 2020 at 12:56, Amit Kapila wrote: > > > > On Tue, Oct 13, 2020 at 10:33 AM Amit Kapila > > wrote: > > > > > > On Tue, Oct 13, 2020 at 10:21 AM Tom Lane wrote: > > > > > > > > > > > > I know I can go read the source code,

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-22 Thread Masahiko Sawada
On Wed, 21 Oct 2020 at 12:56, Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 10:33 AM Amit Kapila wrote: > > > > On Tue, Oct 13, 2020 at 10:21 AM Tom Lane wrote: > > > > > > > > > I know I can go read the source code, but most users will not want to. > > > Is the documentation in monitoring.sgml

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-20 Thread Amit Kapila
On Tue, Oct 13, 2020 at 10:33 AM Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 10:21 AM Tom Lane wrote: > > > > > > I know I can go read the source code, but most users will not want to. > > Is the documentation in monitoring.sgml really sufficient? If we can't > > explain this with more precis

RE: Resetting spilled txn statistics in pg_stat_replication

2020-10-20 Thread Shinoda, Noriyoshi (PN Japan A&PS Delivery)
uesday, October 20, 2020 9:24 PM To: Amit Kapila Cc: Shinoda, Noriyoshi (PN Japan A&PS Delivery) ; Dilip Kumar ; Magnus Hagander ; Tomas Vondra ; PostgreSQL Hackers ; Ajin Cherian Subject: Re: Resetting spilled txn statistics in pg_stat_replication On Tue, 20 Oct 2020 at 20:11, Amit Kapila wro

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-20 Thread Masahiko Sawada
On Tue, 20 Oct 2020 at 20:11, Amit Kapila wrote: > > On Mon, Oct 19, 2020 at 1:20 PM Masahiko Sawada > wrote: > > > > On Mon, 19 Oct 2020 at 14:24, Amit Kapila wrote: > > > > > > > > So, let's > > > stick with 'text' data type for slot_name which means we should > > > go-ahead with the v3 versio

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-20 Thread Amit Kapila
On Mon, Oct 19, 2020 at 1:20 PM Masahiko Sawada wrote: > > On Mon, 19 Oct 2020 at 14:24, Amit Kapila wrote: > > > > > So, let's > > stick with 'text' data type for slot_name which means we should > > go-ahead with the v3 version of the patch [1] posted by Shinoda-San, > > right? > > Right. +1 >

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-19 Thread Masahiko Sawada
On Mon, 19 Oct 2020 at 14:24, Amit Kapila wrote: > > On Mon, Oct 19, 2020 at 9:04 AM Masahiko Sawada > wrote: > > > > On Thu, 15 Oct 2020 at 17:51, Amit Kapila wrote: > > > > > > > > > AFAICS, we use name data-type in many other similar stats views like > > > pg_stat_subscription, pg_statio_all_

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-18 Thread Amit Kapila
On Mon, Oct 19, 2020 at 9:04 AM Masahiko Sawada wrote: > > On Thu, 15 Oct 2020 at 17:51, Amit Kapila wrote: > > > > > > AFAICS, we use name data-type in many other similar stats views like > > pg_stat_subscription, pg_statio_all_sequences, pg_stat_user_functions, > > pg_stat_all_tables. So, shoul

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-18 Thread Masahiko Sawada
On Thu, 15 Oct 2020 at 17:51, Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 5:41 AM Masahiko Sawada > wrote: > > > > On Mon, 12 Oct 2020 at 23:45, Shinoda, Noriyoshi (PN Japan A&PS > > Delivery) wrote: > > > > > > > > > > As it may have been discussed, I think the 'name' column in > > > > pg_s

RE: Resetting spilled txn statistics in pg_stat_replication

2020-10-18 Thread Shinoda, Noriyoshi (PN Japan A&PS Delivery)
gards, Noriyoshi Shinoda -Original Message- From: Amit Kapila [mailto:amit.kapil...@gmail.com] Sent: Thursday, October 15, 2020 5:52 PM To: Masahiko Sawada Cc: Shinoda, Noriyoshi (PN Japan A&PS Delivery) ; Dilip Kumar ; Magnus Hagander ; Tomas Vondra ; PostgreSQL Hackers ; Ajin Cherian Subjec

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-15 Thread Amit Kapila
On Tue, Oct 13, 2020 at 5:41 AM Masahiko Sawada wrote: > > On Mon, 12 Oct 2020 at 23:45, Shinoda, Noriyoshi (PN Japan A&PS > Delivery) wrote: > > > > > > > As it may have been discussed, I think the 'name' column in > > > pg_stat_replication_slots is more consistent with the column name and > >

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-14 Thread Masahiko Sawada
On Wed, 14 Oct 2020 at 12:03, Shinoda, Noriyoshi (PN Japan A&PS Delivery) wrote: > > Thanks for your comment. > > > 8.3. Character Types: > > The name type exists only for the storage of identifiers in the internal > > system catalogs > > I didn't know the policy about data types. Thank you. > Bu

RE: Resetting spilled txn statistics in pg_stat_replication

2020-10-13 Thread Shinoda, Noriyoshi (PN Japan A&PS Delivery)
der ; Tomas Vondra ; PostgreSQL Hackers ; Ajin Cherian Subject: Re: Resetting spilled txn statistics in pg_stat_replication (Please avoid top-posting) On Mon, 12 Oct 2020 at 23:45, Shinoda, Noriyoshi (PN Japan A&PS Delivery) wrote: > > Sawada-san, Thank you your comments. > &g

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-13 Thread Amit Kapila
On Tue, Oct 13, 2020 at 7:57 PM Tom Lane wrote: > > Amit Kapila writes: > > I am able to reproduce this problem via debugger. Basically, execute > > the Insert mentioned above from one the psql sessions and in > > ExecInsert() stop the execution once 'estate->es_processed > 4000' and > > then fro

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-13 Thread Tom Lane
Amit Kapila writes: > I am able to reproduce this problem via debugger. Basically, execute > the Insert mentioned above from one the psql sessions and in > ExecInsert() stop the execution once 'estate->es_processed > 4000' and > then from another psql terminal execute some DDL which will be ignore

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-13 Thread Amit Kapila
On Tue, Oct 13, 2020 at 12:51 PM Masahiko Sawada wrote: > > > > I'll push the patch as I am more > > confident now. > > +1. Let's check how the tests are going to be. > Okay, thanks. I have pushed it. -- With Regards, Amit Kapila.

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-13 Thread Masahiko Sawada
On Tue, 13 Oct 2020 at 16:12, Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 12:17 PM Masahiko Sawada > wrote: > > > > On Tue, 13 Oct 2020 at 15:27, Amit Kapila wrote: > > > > > > On Tue, Oct 13, 2020 at 11:49 AM Masahiko Sawada > > > wrote: > > > > > > > > On Tue, 13 Oct 2020 at 14:53, Amit Ka

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-13 Thread Amit Kapila
On Tue, Oct 13, 2020 at 12:42 PM Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 12:17 PM Masahiko Sawada > wrote: > > > > I agree to check if the spill_counts and spill_txns are positive. > > > > I am able to reproduce this problem via debugger. Basically, execute > the Insert mentioned above fro

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-13 Thread Amit Kapila
On Tue, Oct 13, 2020 at 12:17 PM Masahiko Sawada wrote: > > On Tue, 13 Oct 2020 at 15:27, Amit Kapila wrote: > > > > On Tue, Oct 13, 2020 at 11:49 AM Masahiko Sawada > > wrote: > > > > > > On Tue, 13 Oct 2020 at 14:53, Amit Kapila wrote: > > > > > > > > The original theory I have given above [1

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Masahiko Sawada
On Tue, 13 Oct 2020 at 15:27, Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 11:49 AM Masahiko Sawada > wrote: > > > > On Tue, 13 Oct 2020 at 14:53, Amit Kapila wrote: > > > > > > On Tue, Oct 13, 2020 at 11:05 AM Tom Lane wrote: > > > > > > > > Amit Kapila writes: > > > > >> It is possible tha

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 11:49 AM Masahiko Sawada wrote: > > On Tue, 13 Oct 2020 at 14:53, Amit Kapila wrote: > > > > On Tue, Oct 13, 2020 at 11:05 AM Tom Lane wrote: > > > > > > Amit Kapila writes: > > > >> It is possible that MAXALIGN stuff is playing a role here and or the > > > >> background

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Masahiko Sawada
On Tue, 13 Oct 2020 at 14:53, Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 11:05 AM Tom Lane wrote: > > > > Amit Kapila writes: > > >> It is possible that MAXALIGN stuff is playing a role here and or the > > >> background transaction stuff. I think if we go with the idea of > > >> testing spil

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 11:05 AM Tom Lane wrote: > > Amit Kapila writes: > >> It is possible that MAXALIGN stuff is playing a role here and or the > >> background transaction stuff. I think if we go with the idea of > >> testing spill_txns and spill_count being positive then the results > >> will

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Tom Lane
Amit Kapila writes: >> It is possible that MAXALIGN stuff is playing a role here and or the >> background transaction stuff. I think if we go with the idea of >> testing spill_txns and spill_count being positive then the results >> will be stable. I'll write a patch for that. Here's our first fai

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 10:33 AM Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 10:21 AM Tom Lane wrote: > > > > Amit Kapila writes: > > > On Tue, Oct 13, 2020 at 9:25 AM Tom Lane wrote: > > >> It's not very clear what spill_count actually counts (and the > > >> documentation sure does nothing

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Tom Lane
I wrote: > prairiedog just failed in not-quite-the-same way, which reinforces the > idea that this test is dependent on MAXALIGN, which determines physical > tuple size. (I just checked the buildfarm, and the four active members > that report MAXALIGN 4 during configure are florican, lapwing, locu

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 10:21 AM Tom Lane wrote: > > Amit Kapila writes: > > On Tue, Oct 13, 2020 at 9:25 AM Tom Lane wrote: > >> It's not very clear what spill_count actually counts (and the > >> documentation sure does nothing to clarify that), but if it has anything > >> to do with WAL volume

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Tom Lane
Amit Kapila writes: > On Tue, Oct 13, 2020 at 9:25 AM Tom Lane wrote: >> It's not very clear what spill_count actually counts (and the >> documentation sure does nothing to clarify that), but if it has anything >> to do with WAL volume, the explanation might be that florican is 32-bit. >> All the

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 9:25 AM Tom Lane wrote: > > Amit Kapila writes: > > I have pushed this but it failed in one of the BF. See > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=florican&dt=2020-10-13%2003%3A07%3A25 > > The failure is shown below and I am analyzing it. See, if you ca

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Tom Lane
Amit Kapila writes: > I have pushed this but it failed in one of the BF. See > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=florican&dt=2020-10-13%2003%3A07%3A25 > The failure is shown below and I am analyzing it. See, if you can > provide any insights. It's not very clear what spill_c

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 9:11 AM Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 9:02 AM Amit Kapila wrote: > > > > On Tue, Oct 13, 2020 at 4:54 AM Masahiko Sawada > > wrote: > > > > > > Attached the updated version patch. Please review it. > > > > > > > I have pushed this but it failed in one of

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 9:02 AM Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 4:54 AM Masahiko Sawada > wrote: > > > > Attached the updated version patch. Please review it. > > > > I have pushed this but it failed in one of the BF. See > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fl

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 4:54 AM Masahiko Sawada wrote: > > Attached the updated version patch. Please review it. > I have pushed this but it failed in one of the BF. See https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=florican&dt=2020-10-13%2003%3A07%3A25 The failure is shown below and I

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Masahiko Sawada
eSQL Hackers > ; Ajin Cherian > Subject: Re: Resetting spilled txn statistics in pg_stat_replication > > On Mon, 12 Oct 2020 at 18:29, Shinoda, Noriyoshi (PN Japan A&PS > Delivery) wrote: > > > > Hi, thank you for the awesome feature. > > &

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Masahiko Sawada
On Mon, 12 Oct 2020 at 19:24, Amit Kapila wrote: > > On Mon, Oct 12, 2020 at 10:59 AM Masahiko Sawada > wrote: > > > > On Thu, 8 Oct 2020 at 17:59, Amit Kapila wrote: > > > > > > On Thu, Oct 8, 2020 at 1:55 PM Masahiko Sawada > > > wrote: > > > > > > > > On Thu, 8 Oct 2020 at 14:10, Amit Kapila

RE: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Shinoda, Noriyoshi (PN Japan A&PS Delivery)
, Noriyoshi (PN Japan A&PS Delivery) Cc: Amit Kapila ; Dilip Kumar ; Magnus Hagander ; Tomas Vondra ; PostgreSQL Hackers ; Ajin Cherian Subject: Re: Resetting spilled txn statistics in pg_stat_replication On Mon, 12 Oct 2020 at 18:29, Shinoda, Noriyoshi (PN Japan A&PS Delivery) wrote: >

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Mon, Oct 12, 2020 at 11:52 AM Masahiko Sawada wrote: > > On Thu, 8 Oct 2020 at 22:57, Amit Kapila wrote: > > > > On Thu, Oct 8, 2020 at 7:46 AM Masahiko Sawada > > wrote: > > > > I have rebased the stream stats patch and made minor modifications. I > > haven't done a detailed review but one t

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Masahiko Sawada
On Mon, 12 Oct 2020 at 18:29, Shinoda, Noriyoshi (PN Japan A&PS Delivery) wrote: > > Hi, thank you for the awesome feature. > Thank you for reporting! > As it may have been discussed, I think the 'name' column in > pg_stat_replication_slots is more consistent with the column name and data > ty

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Mon, Oct 12, 2020 at 10:59 AM Masahiko Sawada wrote: > > On Thu, 8 Oct 2020 at 17:59, Amit Kapila wrote: > > > > On Thu, Oct 8, 2020 at 1:55 PM Masahiko Sawada > > wrote: > > > > > > On Thu, 8 Oct 2020 at 14:10, Amit Kapila wrote: > > > > > > > > > > > > We can write if we want but there are

RE: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Shinoda, Noriyoshi (PN Japan A&PS Delivery)
ober 12, 2020 3:22 PM To: Amit Kapila Cc: Dilip Kumar ; Magnus Hagander ; Tomas Vondra ; PostgreSQL Hackers ; Ajin Cherian Subject: Re: Resetting spilled txn statistics in pg_stat_replication On Thu, 8 Oct 2020 at 22:57, Amit Kapila wrote: > > On Thu, Oct 8, 2020 at 7:46 AM Masahiko Sawada

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-11 Thread Masahiko Sawada
On Thu, 8 Oct 2020 at 22:57, Amit Kapila wrote: > > On Thu, Oct 8, 2020 at 7:46 AM Masahiko Sawada > wrote: > > > > On Wed, 7 Oct 2020 at 17:52, Amit Kapila wrote: > > > > > > > > I think after we are done with this the next > > > step would be to finish the streaming stats work [1]. We probably

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-11 Thread Masahiko Sawada
On Thu, 8 Oct 2020 at 17:59, Amit Kapila wrote: > > On Thu, Oct 8, 2020 at 1:55 PM Masahiko Sawada > wrote: > > > > On Thu, 8 Oct 2020 at 14:10, Amit Kapila wrote: > > > > > > > > > We can write if we want but there are few things we need to do for > > > that like maybe a new function like wait_

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-08 Thread Amit Kapila
On Thu, Oct 8, 2020 at 7:46 AM Masahiko Sawada wrote: > > On Wed, 7 Oct 2020 at 17:52, Amit Kapila wrote: > > > > > I think after we are done with this the next > > step would be to finish the streaming stats work [1]. We probably need > > to review and add the test case in that patch. If nobody

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-08 Thread Amit Kapila
On Thu, Oct 8, 2020 at 1:55 PM Masahiko Sawada wrote: > > On Thu, 8 Oct 2020 at 14:10, Amit Kapila wrote: > > > > > > We can write if we want but there are few things we need to do for > > that like maybe a new function like wait_for_spill_stats which will > > check if the counters have become ze

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-08 Thread Masahiko Sawada
On Thu, 8 Oct 2020 at 14:10, Amit Kapila wrote: > > On Thu, Oct 8, 2020 at 7:46 AM Masahiko Sawada > wrote: > > > > On Wed, 7 Oct 2020 at 17:52, Amit Kapila wrote: > > > > > > On Wed, Oct 7, 2020 at 11:24 AM Masahiko Sawada > > > wrote: > > > > > > > > On Tue, 6 Oct 2020 at 17:56, Amit Kapila

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-07 Thread Amit Kapila
On Thu, Oct 8, 2020 at 7:46 AM Masahiko Sawada wrote: > > On Wed, 7 Oct 2020 at 17:52, Amit Kapila wrote: > > > > On Wed, Oct 7, 2020 at 11:24 AM Masahiko Sawada > > wrote: > > > > > > On Tue, 6 Oct 2020 at 17:56, Amit Kapila wrote: > > > > > > > > On Tue, Oct 6, 2020 at 9:34 AM Masahiko Sawada

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-07 Thread Masahiko Sawada
On Wed, 7 Oct 2020 at 17:52, Amit Kapila wrote: > > On Wed, Oct 7, 2020 at 11:24 AM Masahiko Sawada > wrote: > > > > On Tue, 6 Oct 2020 at 17:56, Amit Kapila wrote: > > > > > > On Tue, Oct 6, 2020 at 9:34 AM Masahiko Sawada > > > wrote: > > > > > > > > Looking at pgstat_reset_replslot_counter()

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-07 Thread Amit Kapila
On Wed, Oct 7, 2020 at 11:24 AM Masahiko Sawada wrote: > > On Tue, 6 Oct 2020 at 17:56, Amit Kapila wrote: > > > > On Tue, Oct 6, 2020 at 9:34 AM Masahiko Sawada > > wrote: > > > > > > Looking at pgstat_reset_replslot_counter() in the v8 patch, even if we > > > pass a physical slot name to pg_st

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-06 Thread Masahiko Sawada
On Tue, 6 Oct 2020 at 17:56, Amit Kapila wrote: > > On Tue, Oct 6, 2020 at 9:34 AM Masahiko Sawada > wrote: > > > > Looking at pgstat_reset_replslot_counter() in the v8 patch, even if we > > pass a physical slot name to pg_stat_reset_replication_slot() a > > PgStat_MsgResetreplslotcounter is sent

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-06 Thread Amit Kapila
On Tue, Oct 6, 2020 at 9:34 AM Masahiko Sawada wrote: > > Looking at pgstat_reset_replslot_counter() in the v8 patch, even if we > pass a physical slot name to pg_stat_reset_replication_slot() a > PgStat_MsgResetreplslotcounter is sent to the stats collector. I’m > okay with not raising an error b

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-05 Thread Masahiko Sawada
On Mon, 5 Oct 2020 at 17:50, Amit Kapila wrote: > > On Mon, Oct 5, 2020 at 1:26 PM Masahiko Sawada > wrote: > > > > On Sat, 3 Oct 2020 at 16:55, Amit Kapila wrote: > > > > > > On Sat, Oct 3, 2020 at 9:26 AM Masahiko Sawada > > > wrote: > > > > > > > > When we discussed this before, I was thinki

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-05 Thread Amit Kapila
On Mon, Oct 5, 2020 at 1:26 PM Masahiko Sawada wrote: > > On Sat, 3 Oct 2020 at 16:55, Amit Kapila wrote: > > > > On Sat, Oct 3, 2020 at 9:26 AM Masahiko Sawada > > wrote: > > > > > > When we discussed this before, I was thinking that we could have other > > > statistics for physical slots in th

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-05 Thread Masahiko Sawada
On Sat, 3 Oct 2020 at 16:55, Amit Kapila wrote: > > On Sat, Oct 3, 2020 at 9:26 AM Masahiko Sawada > wrote: > > > > When we discussed this before, I was thinking that we could have other > > statistics for physical slots in the same statistics view in the > > future. Having the view show only log

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-03 Thread Amit Kapila
On Sat, Oct 3, 2020 at 9:26 AM Masahiko Sawada wrote: > > When we discussed this before, I was thinking that we could have other > statistics for physical slots in the same statistics view in the > future. Having the view show only logical slots also makes sense to me > but I’m concerned a bit tha

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-02 Thread Masahiko Sawada
On Wed, 30 Sep 2020 at 18:10, Amit Kapila wrote: > > On Wed, Sep 30, 2020 at 1:12 PM Dilip Kumar wrote: > > > > On Fri, Sep 25, 2020 at 4:33 PM Amit Kapila wrote: > > > > > > On Thu, Sep 24, 2020 at 5:44 PM Amit Kapila > > > wrote: > > > > > > I have done some more testing of this patch especi

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-30 Thread Amit Kapila
On Thu, Oct 1, 2020 at 12:06 PM Amit Kapila wrote: > > Thanks for the review. > oops, forgot to attach the updated patches, doing now. -- With Regards, Amit Kapila. v7-0001-Track-statistics-for-spilling-of-changes-from-Reo.patch Description: Binary data v7-0002-Test-stats.patch Description:

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-30 Thread Amit Kapila
On Wed, Sep 30, 2020 at 4:34 PM Dilip Kumar wrote: > > On Wed, Sep 30, 2020 at 2:40 PM Amit Kapila wrote: > > > > On Wed, Sep 30, 2020 at 1:12 PM Dilip Kumar wrote: > > > > > > On Fri, Sep 25, 2020 at 4:33 PM Amit Kapila > > > wrote: > > > > > > > > On Thu, Sep 24, 2020 at 5:44 PM Amit Kapila

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-30 Thread Dilip Kumar
On Wed, Sep 30, 2020 at 2:40 PM Amit Kapila wrote: > > On Wed, Sep 30, 2020 at 1:12 PM Dilip Kumar wrote: > > > > On Fri, Sep 25, 2020 at 4:33 PM Amit Kapila wrote: > > > > > > On Thu, Sep 24, 2020 at 5:44 PM Amit Kapila > > > wrote: > > > > > > I have done some more testing of this patch espe

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-30 Thread Amit Kapila
On Wed, Sep 30, 2020 at 1:12 PM Dilip Kumar wrote: > > On Fri, Sep 25, 2020 at 4:33 PM Amit Kapila wrote: > > > > On Thu, Sep 24, 2020 at 5:44 PM Amit Kapila wrote: > > > > I have done some more testing of this patch especially for the case > > where we spill before streaming the transaction and

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-30 Thread Dilip Kumar
On Fri, Sep 25, 2020 at 4:33 PM Amit Kapila wrote: > > On Thu, Sep 24, 2020 at 5:44 PM Amit Kapila wrote: > > > > On Sat, Sep 19, 2020 at 1:48 PM Amit Kapila wrote: > > > > > > On Tue, Sep 8, 2020 at 7:02 PM Amit Kapila > > > wrote: > > > > > > > > On Tue, Sep 8, 2020 at 7:53 AM Masahiko Sawad

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-25 Thread Amit Kapila
On Thu, Sep 24, 2020 at 5:44 PM Amit Kapila wrote: > > On Sat, Sep 19, 2020 at 1:48 PM Amit Kapila wrote: > > > > On Tue, Sep 8, 2020 at 7:02 PM Amit Kapila wrote: > > > > > > On Tue, Sep 8, 2020 at 7:53 AM Masahiko Sawada > > > wrote: > > > > I have fixed these review comments in the attached

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-24 Thread Amit Kapila
On Sat, Sep 19, 2020 at 1:48 PM Amit Kapila wrote: > > On Tue, Sep 8, 2020 at 7:02 PM Amit Kapila wrote: > > > > On Tue, Sep 8, 2020 at 7:53 AM Masahiko Sawada > > wrote: > > I have fixed these review comments in the attached patch. > > > Apart from the above, > (a) fixed one bug in ReorderBuffe

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-19 Thread Amit Kapila
On Tue, Sep 8, 2020 at 7:02 PM Amit Kapila wrote: > > On Tue, Sep 8, 2020 at 7:53 AM Masahiko Sawada > wrote: I have fixed these review comments in the attached patch. > > Comments on the latest patch: > = > 1. > +CREATE VIEW pg_stat_replication_slots AS > +SELEC

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-09 Thread Amit Kapila
On Wed, Sep 9, 2020 at 3:20 PM Amit Kapila wrote: > > On Tue, Sep 8, 2020 at 7:02 PM Amit Kapila wrote: > > > > Comments on the latest patch: > > = > > > > Apart from the comments I gave yesterday, another thing I was > wondering is how to write some tests for this pat

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-09 Thread Amit Kapila
On Tue, Sep 8, 2020 at 7:02 PM Amit Kapila wrote: > > Comments on the latest patch: > = > Apart from the comments I gave yesterday, another thing I was wondering is how to write some tests for this patch. The two ideas I could think of are as follows: 1. One idea was

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-08 Thread Amit Kapila
On Tue, Sep 8, 2020 at 7:53 AM Masahiko Sawada wrote: > > On Mon, 7 Sep 2020 at 15:24, Amit Kapila wrote: > > > > This patch needs a rebase. I don't see this patch in the CF app. I > > hope you are still interested in working on this. > > Thank you for reviewing this patch! > > I'm still going to

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-07 Thread Amit Kapila
On Tue, Sep 8, 2020 at 7:53 AM Masahiko Sawada wrote: > > On Mon, 7 Sep 2020 at 15:24, Amit Kapila wrote: > > I'm still going to work on this patch although I might be slow > response this month. > This is a quite fast response. Thanks for staying on top of it. > > > > > 9. While reviewing this

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-07 Thread Masahiko Sawada
On Mon, 7 Sep 2020 at 15:24, Amit Kapila wrote: > > On Thu, Jul 23, 2020 at 11:46 AM Masahiko Sawada > wrote: > > > > I've updated the patch so that the stats collector ignores the > > 'update' message if the slot stats array is already full. > > > > This patch needs a rebase. I don't see this pa

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-06 Thread Amit Kapila
On Thu, Jul 23, 2020 at 11:46 AM Masahiko Sawada wrote: > > I've updated the patch so that the stats collector ignores the > 'update' message if the slot stats array is already full. > This patch needs a rebase. I don't see this patch in the CF app. I hope you are still interested in working on t

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-22 Thread Masahiko Sawada
On Thu, 16 Jul 2020 at 20:01, Amit Kapila wrote: > > On Thu, Jul 16, 2020 at 4:04 PM Masahiko Sawada > wrote: > > > > On Thu, 16 Jul 2020 at 18:16, Amit Kapila wrote: > > > > > > On Thu, Jul 16, 2020 at 1:45 PM Masahiko Sawada > > > wrote: > > > > > > > > A possible solution would be to add an

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-16 Thread Amit Kapila
On Thu, Jul 16, 2020 at 4:04 PM Masahiko Sawada wrote: > > On Thu, 16 Jul 2020 at 18:16, Amit Kapila wrote: > > > > On Thu, Jul 16, 2020 at 1:45 PM Masahiko Sawada > > wrote: > > > > > > A possible solution would be to add an in-use flag to > > > PgStat_ReplSlotStats indicating whether the stats

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-16 Thread Masahiko Sawada
On Thu, 16 Jul 2020 at 18:16, Amit Kapila wrote: > > On Thu, Jul 16, 2020 at 1:45 PM Masahiko Sawada > wrote: > > > > On Thu, 16 Jul 2020 at 15:53, Amit Kapila wrote: > > > > > > On Fri, Jul 10, 2020 at 2:42 PM Amit Kapila > > > wrote: > > > > > > > > On Fri, Jul 10, 2020 at 7:23 AM Masahiko S

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-16 Thread Amit Kapila
On Thu, Jul 16, 2020 at 1:45 PM Masahiko Sawada wrote: > > On Thu, 16 Jul 2020 at 15:53, Amit Kapila wrote: > > > > On Fri, Jul 10, 2020 at 2:42 PM Amit Kapila wrote: > > > > > > On Fri, Jul 10, 2020 at 7:23 AM Masahiko Sawada > > > wrote: > > > > > > > > On Thu, 9 Jul 2020 at 12:11, Amit Kapil

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-16 Thread Masahiko Sawada
On Thu, 16 Jul 2020 at 15:53, Amit Kapila wrote: > > On Fri, Jul 10, 2020 at 2:42 PM Amit Kapila wrote: > > > > On Fri, Jul 10, 2020 at 7:23 AM Masahiko Sawada > > wrote: > > > > > > On Thu, 9 Jul 2020 at 12:11, Amit Kapila wrote: > > > > > > > > On Wed, Jul 8, 2020 at 1:14 PM Masahiko Sawada >

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-15 Thread Amit Kapila
On Fri, Jul 10, 2020 at 2:42 PM Amit Kapila wrote: > > On Fri, Jul 10, 2020 at 7:23 AM Masahiko Sawada > wrote: > > > > On Thu, 9 Jul 2020 at 12:11, Amit Kapila wrote: > > > > > > On Wed, Jul 8, 2020 at 1:14 PM Masahiko Sawada > > > wrote: > > > > > > > > > > > > I think that using oids has ano

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-13 Thread Amit Kapila
On Fri, Jul 10, 2020 at 2:39 PM Amit Kapila wrote: > > On Fri, Jul 10, 2020 at 7:19 AM Masahiko Sawada > wrote: > > > > On Thu, 9 Jul 2020 at 16:09, Amit Kapila wrote: > > > > > > > > > Fair enough. The attached patch reverts the commits related to these > > > stats. Sawada-San, can you please

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-10 Thread Amit Kapila
On Fri, Jul 10, 2020 at 7:23 AM Masahiko Sawada wrote: > > On Thu, 9 Jul 2020 at 12:11, Amit Kapila wrote: > > > > On Wed, Jul 8, 2020 at 1:14 PM Masahiko Sawada > > wrote: > > > > > > > > > I think that using oids has another benefit that we don't need to send > > > slot name to the stats colle

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-10 Thread Amit Kapila
On Fri, Jul 10, 2020 at 7:19 AM Masahiko Sawada wrote: > > On Thu, 9 Jul 2020 at 16:09, Amit Kapila wrote: > > > > > > Fair enough. The attached patch reverts the commits related to these > > stats. Sawada-San, can you please once see if I have missed anything > > apart from catversion bump whi

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-09 Thread Masahiko Sawada
On Thu, 9 Jul 2020 at 12:11, Amit Kapila wrote: > > On Wed, Jul 8, 2020 at 1:14 PM Masahiko Sawada > wrote: > > > > On Wed, 8 Jul 2020 at 16:04, Amit Kapila wrote: > > > > > > On Wed, Jul 8, 2020 at 11:28 AM Masahiko Sawada > > > wrote: > > > > > > > > > > > > > > If we need them to be persiste

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-09 Thread Masahiko Sawada
On Thu, 9 Jul 2020 at 16:09, Amit Kapila wrote: > > On Tue, Jul 7, 2020 at 2:20 PM Magnus Hagander wrote: > > > > On Tue, Jul 7, 2020 at 5:10 AM Amit Kapila wrote: > >> > >> > > I think it depends on the final patch. My initial thought was that we > >> > > should do this for PG14 but if you are

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-09 Thread Amit Kapila
On Tue, Jul 7, 2020 at 2:20 PM Magnus Hagander wrote: > > On Tue, Jul 7, 2020 at 5:10 AM Amit Kapila wrote: >> >> > > I think it depends on the final patch. My initial thought was that we >> > > should do this for PG14 but if you are suggesting removing the changes >> > > done by commit 9290ad198

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-08 Thread Amit Kapila
On Wed, Jul 8, 2020 at 1:14 PM Masahiko Sawada wrote: > > On Wed, 8 Jul 2020 at 16:04, Amit Kapila wrote: > > > > On Wed, Jul 8, 2020 at 11:28 AM Masahiko Sawada > > wrote: > > > > > > > > > > > If we need them to be persistent across time like that, perhaps we > > > > simply need to assign oid

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-08 Thread Masahiko Sawada
On Wed, 8 Jul 2020 at 16:04, Amit Kapila wrote: > > On Wed, Jul 8, 2020 at 11:28 AM Masahiko Sawada > wrote: > > > > On Tue, 7 Jul 2020 at 17:50, Magnus Hagander wrote: > > > > > > > > > > > > On Tue, Jul 7, 2020 at 5:10 AM Amit Kapila > > > wrote: > > >> > > >> On Tue, Jul 7, 2020 at 7:07 AM

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-08 Thread Amit Kapila
On Wed, Jul 8, 2020 at 11:28 AM Masahiko Sawada wrote: > > On Tue, 7 Jul 2020 at 17:50, Magnus Hagander wrote: > > > > > > > > On Tue, Jul 7, 2020 at 5:10 AM Amit Kapila wrote: > >> > >> On Tue, Jul 7, 2020 at 7:07 AM Masahiko Sawada > >> wrote: > >> > > >> > On Mon, 6 Jul 2020 at 20:45, Amit K

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-07 Thread Masahiko Sawada
On Tue, 7 Jul 2020 at 17:50, Magnus Hagander wrote: > > > > On Tue, Jul 7, 2020 at 5:10 AM Amit Kapila wrote: >> >> On Tue, Jul 7, 2020 at 7:07 AM Masahiko Sawada >> wrote: >> > >> > On Mon, 6 Jul 2020 at 20:45, Amit Kapila wrote: >> > > >> > > > Second, as long as the unique identifier is the

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-07 Thread Magnus Hagander
On Tue, Jul 7, 2020 at 5:10 AM Amit Kapila wrote: > On Tue, Jul 7, 2020 at 7:07 AM Masahiko Sawada > wrote: > > > > On Mon, 6 Jul 2020 at 20:45, Amit Kapila > wrote: > > > > > > > Second, as long as the unique identifier is the slot name there is no > > > > convenient way to distinguish between

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-06 Thread Amit Kapila
On Tue, Jul 7, 2020 at 7:07 AM Masahiko Sawada wrote: > > On Mon, 6 Jul 2020 at 20:45, Amit Kapila wrote: > > > > > Second, as long as the unique identifier is the slot name there is no > > > convenient way to distinguish between the same name old and new > > > replication slots, so the backend p

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-06 Thread Masahiko Sawada
On Mon, 6 Jul 2020 at 20:45, Amit Kapila wrote: > > On Thu, Jul 2, 2020 at 9:01 AM Masahiko Sawada > wrote: > > > > I've attached PoC patch that implements a simple approach. I'd like to > > discuss how we collect the replication slot statistics in the stats > > collector before I bring the patch

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-06 Thread Amit Kapila
On Thu, Jul 2, 2020 at 9:01 AM Masahiko Sawada wrote: > > I've attached PoC patch that implements a simple approach. I'd like to > discuss how we collect the replication slot statistics in the stats > collector before I bring the patch to completion. > > In this PoC patch, we have the array of PgS

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-06 Thread Masahiko Sawada
On Sat, 4 Jul 2020 at 22:13, Ajin Cherian wrote: > > > > On Thu, Jul 2, 2020 at 1:31 PM Masahiko Sawada > wrote: >> >> >> >> Thanks! Yes, I'm working on this patch while considering to send the >> stats to stats collector. >> >> I've attached PoC patch that implements a simple approach. I'd like

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-04 Thread Ajin Cherian
On Thu, Jul 2, 2020 at 1:31 PM Masahiko Sawada < masahiko.saw...@2ndquadrant.com> wrote: > > > Thanks! Yes, I'm working on this patch while considering to send the > stats to stats collector. > > I've attached PoC patch that implements a simple approach. I'd like to > discuss how we collect the re

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-01 Thread Masahiko Sawada
On Tue, 30 Jun 2020 at 12:58, Amit Kapila wrote: > > On Tue, Jun 30, 2020 at 6:38 AM Masahiko Sawada > wrote: > > > > On Mon, 29 Jun 2020 at 20:37, Amit Kapila wrote: > > > > > > On Mon, Jun 29, 2020 at 10:26 AM Masahiko Sawada > > > wrote: > > > > > > > > I agree that it would not be a common

  1   2   >