Should rename "startup process" to something else?

2021-11-15 Thread Rushabh Lathia
Hi, Robert and I wondered whether we would like to rename the startup process. The reason for doing this is that the current name doesn't make any sense, as in the stand-by mode replay loop as the main loop, the startup process stays around. I think a better name for the process may be “reco

Re: Commitfest 2021-11 Patch Triage - Part 2

2021-11-15 Thread Robert Haas
On Tue, Nov 9, 2021 at 9:12 AM Daniel Gustafsson wrote: > 2773: libpq compression > === > This patch intended to provide libpq connection compression to "replace SSL > compression" which was doomed when the patch was written, and have since been > removed altogether. The initi

Re: RFC: Logging plan of the running query

2021-11-15 Thread Bharath Rupireddy
On Mon, Nov 15, 2021 at 6:29 PM torikoshia wrote: > > 3) I think SendProcSignalForLogInfo can be more generic, meaning, it > > can also send signal to auxiliary processes if asked to do this will > > simplify the things for pg_log_backend_memory_contexts and other > > patches like pg_print_backtra

pg_waldump stucks with options --follow or -f and --stats or -z

2021-11-15 Thread Bharath Rupireddy
Hi, pg_waldump options, --follow or -f(to keep polling once per second for new WAL to appear) and --stats or -z don't work well together i.e. the command stucks [1]. I think the pg_waldump should emit an error. Note that the pg_basebakup does error out on incompatible options. Here's a small patc

Re: RFC: Logging plan of the running query

2021-11-15 Thread torikoshia
On 2021-11-13 03:37, Justin Pryzby wrote: I reviewed this version of the patch - I have some language fixes. Thanks for your review! Attached patch that reflects your comments. Regards, -- Atsushi Torikoshi NTT DATA CORPORATIONFrom b8367e22d7a9898e4b85627ba8c203be273fc22f Mon Sep 17 00:00:0

Re: pg_get_publication_tables() output duplicate relid

2021-11-15 Thread Alvaro Herrera
> On Mon, Nov 15, 2021 at 1:48 PM houzj.f...@fujitsu.com > wrote: > > create table tbl1 (a int) partition by range (a); > > create table tbl1_part1 partition of tbl1 for values from (1) to (10); > > create table tbl1_part2 partition of tbl1 for values from (10) to (20); > > create publication pub

Re: pgbench: using prepared BEGIN statement in a pipeline could cause an error

2021-11-15 Thread Daniel Gustafsson
> On 21 Jul 2021, at 03:49, Yugo NAGATA wrote: > I attached the updated patch v2, which includes a comment fix and a TAP test. This patch fails the TAP test for pgbench: # Tests were run but no plan was declared and done_testing() was not seen. # Looks like your test exited with 25 just aft

Re: Partial aggregates pushdown

2021-11-15 Thread Alexander Pyhalov
Daniel Gustafsson писал 2021-11-15 13:16: On 3 Nov 2021, at 15:50, Alexander Pyhalov wrote: Daniel Gustafsson писал 2021-11-03 16:45: On 2 Nov 2021, at 10:12, Alexander Pyhalov wrote: Updated and rebased patch. + state = (Int128AggState *) palloc0(sizeof(Int128AggState)); + sta

Re: RFC: Logging plan of the running query

2021-11-15 Thread torikoshia
On 2021-11-13 22:29, Bharath Rupireddy wrote: Thanks for your review! On Wed, Oct 13, 2021 at 7:58 PM Ekaterina Sokolova wrote: Thank you for working on this issue. I would be glad to continue to follow the development of this issue. Thanks for the patch. I'm not sure if v11 is the latest pa

Re: parallel vacuum comments

2021-11-15 Thread Masahiko Sawada
On Mon, Nov 15, 2021 at 8:04 PM Amit Kapila wrote: > > On Mon, Nov 15, 2021 at 2:01 PM Masahiko Sawada wrote: > > > > On Thu, Nov 11, 2021 at 6:38 PM Amit Kapila wrote: > > > > > > On Thu, Nov 11, 2021 at 8:11 AM Masahiko Sawada > > > wrote: > > > > > > > > I've attached a draft patch that ref

RE: Failed transaction statistics to measure the logical replication progress

2021-11-15 Thread osumi.takami...@fujitsu.com
On Monday, November 15, 2021 6:28 PM I wrote: > On Thursday, November 11, 2021 9:47 PM vignesh C > wrote: > > Few more comments: ... > Fixed. Also, its name was too long > when aligned with other PgStat_StatDBEntry memebers. > Thus I renamed it as subworkers_preparedsizes. > > This depends on v21

Re: Add connection active, idle time to pg_stat_activity

2021-11-15 Thread Dilip Kumar
On Mon, Nov 15, 2021 at 4:46 PM Rafia Sabih wrote: > > On Mon, 15 Nov 2021 at 10:24, Dilip Kumar wrote: > > > > On Wed, Nov 10, 2021 at 1:47 PM Rafia Sabih > > wrote: > > > > > > > It seems that in beentry->st_idle_time, you want to compute the > > > > STATE_IDLE, but that state is not handled

Re: Add connection active, idle time to pg_stat_activity

2021-11-15 Thread Rafia Sabih
On Mon, 15 Nov 2021 at 10:24, Dilip Kumar wrote: > > On Wed, Nov 10, 2021 at 1:47 PM Rafia Sabih wrote: > > > > > It seems that in beentry->st_idle_time, you want to compute the > > > STATE_IDLE, but that state is not handled in the outer "if", that > > > means whenever it comes out of the > > >

Re: parallel vacuum comments

2021-11-15 Thread Amit Kapila
On Mon, Nov 15, 2021 at 2:01 PM Masahiko Sawada wrote: > > On Thu, Nov 11, 2021 at 6:38 PM Amit Kapila wrote: > > > > On Thu, Nov 11, 2021 at 8:11 AM Masahiko Sawada > > wrote: > > > > > > I've attached a draft patch that refactors parallel vacuum and > > > separates parallel-vacuum-related cod

Re: WIP: System Versioned Temporal Table

2021-11-15 Thread Simon Riggs
On Mon, 15 Nov 2021 at 09:47, Daniel Gustafsson wrote: > > > On 19 Sep 2021, at 20:32, Simon Riggs wrote: > > > My preferred approach would be to do this "for free" in the table > > access method, but we're a long way from this in terms of actual > > implementation. When Corey suggested earlier

Re: [Proposal] Global temporary tables

2021-11-15 Thread Andrew Bille
Thanks for the patches. The feature has become much more stable. However, there is another simple case that generates an error: Master with v61 patches CREATE GLOBAL TEMPORARY TABLE t AS SELECT 1 AS a; ERROR: could not open file "base/13560/t3_16384": No such file or directory Andrew On Thu, Nov

Re: row filtering for logical replication

2021-11-15 Thread Amit Kapila
On Wed, Nov 10, 2021 at 12:36 PM Peter Smith wrote: > > On Mon, Nov 8, 2021 at 5:53 PM houzj.f...@fujitsu.com > wrote: > > > > 3) v37-0005 > > > > - no parse nodes of any kind other than Var, OpExpr, Const, BoolExpr, > > FuncExpr > > > > I think there could be other node type which can also be c

Re: WIP: WAL prefetch (another approach)

2021-11-15 Thread Daniel Gustafsson
> On 10 May 2021, at 06:11, Thomas Munro wrote: > On Thu, Apr 22, 2021 at 11:22 AM Stephen Frost wrote: >> I tend to agree with the idea to revert it, perhaps a +0 on that, but if >> others argue it should be fixed in-place, I wouldn’t complain about it. > > Reverted. > > Note: eelpout may re

Re: pg_get_publication_tables() output duplicate relid

2021-11-15 Thread Bharath Rupireddy
On Mon, Nov 15, 2021 at 1:48 PM houzj.f...@fujitsu.com wrote: > > Hi hackers, > > In another thread[1], we found the pg_get_publication_tables function will > output > duplicate partition relid when adding both child and parent table to the > publication(pubviaroot = false). > > Example: > create

Re: Partial aggregates pushdown

2021-11-15 Thread Daniel Gustafsson
> On 3 Nov 2021, at 15:50, Alexander Pyhalov wrote: > > Daniel Gustafsson писал 2021-11-03 16:45: >>> On 2 Nov 2021, at 10:12, Alexander Pyhalov wrote: >>> Updated and rebased patch. >> +state = (Int128AggState *) palloc0(sizeof(Int128AggState)); >> +state->calcSumX2 = false; >> + >> +

Re: Split xlog.c

2021-11-15 Thread Daniel Gustafsson
> On 5 Oct 2021, at 03:09, Jaime Casanova wrote: > Are we waiting a rebased version? Currently this does not apply to head. > I'll mark this as WoA and move it to necxt CF. This patch still doesn't apply, exacerbated by the recent ThisTimelineID changes in xlog.c. I'm marking this Returned with

Re: Unnecessary delay in streaming replication due to replay lag

2021-11-15 Thread Daniel Gustafsson
> On 10 Nov 2021, at 00:41, Soumyadeep Chakraborty > wrote: > Thanks for the detailed review! Attached is a rebased patch that addresses > most of the feedback. This patch no longer applies after e997a0c64 and associated follow-up commits, please submit a rebased version. -- Daniel Gustafsson

Re: row filtering for logical replication

2021-11-15 Thread Greg Nancarrow
On Mon, Nov 15, 2021 at 5:09 PM tanghy.f...@fujitsu.com wrote: > > I met a problem when using "ALTER PUBLICATION ... SET TABLE ... WHERE ...", > the > publisher was crashed after executing this statement. > > > > Backtrace is attached. I think maybe the problem is related to the below > change i

Re: WIP: System Versioned Temporal Table

2021-11-15 Thread Vik Fearing
On 11/15/21 10:47 AM, Daniel Gustafsson wrote: >> On 19 Sep 2021, at 20:32, Simon Riggs wrote: > >> My preferred approach would be to do this "for free" in the table >> access method, but we're a long way from this in terms of actual >> implementation. When Corey suggested earlier that we just p

Re: WIP: System Versioned Temporal Table

2021-11-15 Thread Daniel Gustafsson
> On 19 Sep 2021, at 20:32, Simon Riggs wrote: > My preferred approach would be to do this "for free" in the table > access method, but we're a long way from this in terms of actual > implementation. When Corey suggested earlier that we just put the > syntax in there, this was the direction I wa

Re: row filtering for logical replication

2021-11-15 Thread Dilip Kumar
On Mon, 15 Nov 2021 at 3:07 PM, Amit Kapila wrote: > On Mon, Nov 15, 2021 at 2:44 PM Dilip Kumar wrote: > > > > On Fri, Nov 12, 2021 at 3:49 PM Ajin Cherian wrote: > > > > > > > This function definition header is too long to fit in one line, so > > better to break it. I think running will be a

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-11-15 Thread Bharath Rupireddy
On Mon, Nov 15, 2021 at 2:14 PM Daniel Gustafsson wrote: > > > On 15 Nov 2021, at 09:15, Peter Eisentraut > > wrote: > > > I think this is not an improvement. It loses the ability of the caller the > > specify exactly why a relation is not acceptable. > > > Agreed. +1. > > I think a separate

Re: row filtering for logical replication

2021-11-15 Thread Amit Kapila
On Mon, Nov 15, 2021 at 2:44 PM Dilip Kumar wrote: > > On Fri, Nov 12, 2021 at 3:49 PM Ajin Cherian wrote: > > > > This function definition header is too long to fit in one line, so > better to break it. I think running will be a good idea. > It seems in the last line you are suggesting to run

RE: Failed transaction statistics to measure the logical replication progress

2021-11-15 Thread osumi.takami...@fujitsu.com
On Wednesday, November 10, 2021 6:13 PM I wrote: > On Wednesday, November 10, 2021 3:43 PM Dilip Kumar > wrote: > > On Tue, Nov 9, 2021 at 5:05 PM osumi.takami...@fujitsu.com > > wrote: > > > On Tuesday, November 9, 2021 12:08 PM Greg Nancarrow > > wrote: > > > > On Fri, Nov 5, 2021 at 7:11 PM o

RE: Failed transaction statistics to measure the logical replication progress

2021-11-15 Thread osumi.takami...@fujitsu.com
On Wednesday, November 10, 2021 7:13 PM vignesh C wrote: > On Tue, Nov 9, 2021 at 5:05 PM osumi.takami...@fujitsu.com > wrote: > > Yes. I've rebased and updated the patch, paying attention to this point. > > Attached the updated version. > > Thanks for the updated patch, few comments: > 1) you c

RE: Failed transaction statistics to measure the logical replication progress

2021-11-15 Thread osumi.takami...@fujitsu.com
On Thursday, November 11, 2021 9:47 PM vignesh C wrote: > Few more comments: > 1) Here the tuple length is not considered in the calculation, else it will > always > show the fixed size for any size tuple. Ex varchar insert with 1 byte or > varchar > insert with 100's of bytes. So I feel we shou

Re: Write visibility map during CLUSTER/VACUUM FULL

2021-11-15 Thread Daniel Gustafsson
> On 30 Aug 2021, at 02:26, Justin Pryzby wrote: > Rebased on f10f0ae420ee62400876ab34dca2c09c20dcd030. This patch no longer applies, please submit a rebased version. -- Daniel Gustafsson https://vmware.com/

Re: Add connection active, idle time to pg_stat_activity

2021-11-15 Thread Dilip Kumar
On Wed, Nov 10, 2021 at 1:47 PM Rafia Sabih wrote: > > > It seems that in beentry->st_idle_time, you want to compute the > > STATE_IDLE, but that state is not handled in the outer "if", that > > means whenever it comes out of the > > STATE_IDLE, it will not enter inside this if check. You can run

Re: Skipping logical replication transactions on subscriber side

2021-11-15 Thread Masahiko Sawada
On Mon, Nov 15, 2021 at 4:49 PM Greg Nancarrow wrote: > > On Mon, Nov 15, 2021 at 1:49 PM Masahiko Sawada wrote: > > > > I've attached an updated patch that incorporates all comments I got so > > far. Please review it. > > > > Thanks for the updated patch. > A few minor comments: > > doc/src/sgml

Re: row filtering for logical replication

2021-11-15 Thread Dilip Kumar
On Fri, Nov 12, 2021 at 3:49 PM Ajin Cherian wrote: > > Attaching version 39- > Some comments on 0006 -- /* + * Write UPDATE to the output stream using cached virtual slots. + * Cached updates will have both old tuple and new tuple. + */ +void +logicalrep_write_update_cached(StringInfo out, Tra

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-11-15 Thread Daniel Gustafsson
> On 15 Nov 2021, at 09:15, Peter Eisentraut > wrote: > I think this is not an improvement. It loses the ability of the caller the > specify exactly why a relation is not acceptable. Agreed. > I think a separate errdetail_relpersistence_not_supported() would be a better > solution (assuming

Re: parallel vacuum comments

2021-11-15 Thread Masahiko Sawada
On Thu, Nov 11, 2021 at 6:38 PM Amit Kapila wrote: > > On Thu, Nov 11, 2021 at 8:11 AM Masahiko Sawada wrote: > > > > I've attached a draft patch that refactors parallel vacuum and > > separates parallel-vacuum-related code to new file vacuumparallel.c. > > After discussion, I'll divide the patch

pg_get_publication_tables() output duplicate relid

2021-11-15 Thread houzj.f...@fujitsu.com
Hi hackers, In another thread[1], we found the pg_get_publication_tables function will output duplicate partition relid when adding both child and parent table to the publication(pubviaroot = false). Example: create table tbl1 (a int) partition by range (a); create table tbl1_part1 partition of

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-11-15 Thread Peter Eisentraut
On 14.11.21 13:18, Bharath Rupireddy wrote: PSA v11 patch with 2 APIs with much simpler parameters and small function names: int errdetail_rel(Form_pg_class rd_rel); int errdetail_rel_v2(Oid relid, char relkind, char relpersistence); Please review it. I think this is not an improvement. It

Re: Invalid Unicode escape value at or near "\u0000"

2021-11-15 Thread Peter Eisentraut
On 13.11.21 06:40, Thomas Munro wrote: Yes, it is a valid codepoint, but unfortunately PostgreSQL can't support it because it sometimes deals in null terminated string, even though internally it does track string data and length separately. We have to do that to use libc facilities like strcoll_

<    1   2