Re: Why does creating logical replication subscriptions require superuser?

2021-01-21 Thread Andrey Borodin
Hi! > 21 янв. 2021 г., в 21:20, Paul Martinez написал(а): > > Hey, all, > > I'm working with native logical replication, and I don't fully understand > why logical replication subscribers need to be superusers, nor do I fully > understand the implication of some of the comments made on this pag

RE: Parallel INSERT (INTO ... SELECT ...)

2021-01-21 Thread Hou, Zhijie
> > And there seems another solution for this: > > > > In the patch, We only use the { ii_Expressions , ii_NumIndexAttrs , > > ii_IndexAttrNumbers } from the IndexInfo, which seems can get from > "Relation-> rd_index". > > > > Based on above, May be we do not need to call BuildIndexInfo to build >

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-21 Thread Greg Nancarrow
On Fri, Jan 22, 2021 at 4:49 PM Amit Kapila wrote: > > > > > Unfortunately, this change results in a single test failure in the > > "with" tests when "force_parallel_mode=regress" is in effect. > > > > I have reproduced the problem, by extracting relevant SQL from those > > tests, as follows: > >

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-21 Thread Bharath Rupireddy
On Fri, Jan 22, 2021 at 10:14 AM japin wrote: > > 2) Can't we know whether the publications exist on the publisher with > > the existing (or modifying it a bit if required) query in > > fetch_table_list(), so that we can avoid making another connection to > > the publisher system from the subscrib

Re: [PATCH] More docs on what to do and not do in extension code

2021-01-21 Thread Craig Ringer
Hi Thanks so much for reading over this! Would you mind attaching a revised version of the patch with your edits? Otherwise I'll go and merge them in once you've had your say on my comments inline below. Bruce, Robert, can I have an opinion from you on how best to locate and structure these docs

Re: Very misleading documentation for PQreset()

2021-01-21 Thread Noah Misch
On Thu, Jan 21, 2021 at 05:32:56PM -0500, Tom Lane wrote: > I happened to notice that PQreset is documented thus: > > This function will close the connection to the server and attempt to > reestablish a new connection to the same server, using all the same > parameters previously used.

Re: simplifying foreign key/RI checks

2021-01-21 Thread Corey Huinker
> > > > I decided not to deviate from pk_ terminology so that the new code > doesn't look too different from the other code in the file. Although, > I guess we can at least call the main function > ri_ReferencedKeyExists() instead of ri_PrimaryKeyExists(), so I've > changed that. > I think that's

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-21 Thread Greg Nancarrow
On Fri, Jan 22, 2021 at 1:16 PM Hou, Zhijie wrote: > > > 4. > + ListCell *index_expr_item = > list_head(index_info->ii_Expressions); > ... > + index_expr = (Node *) > lfirst(index_expr_item); > +

Re: POC: postgres_fdw insert batching

2021-01-21 Thread Ian Lawrence Barwick
Hi 2021年1月21日(木) 8:00 Tomas Vondra : > OK, pushed after a little bit of additional polishing (mostly comments). > > Thanks everyone! > There's a minor typo in the doc's version of the ExecForeignBatchInsert() declaration; is: TupleTableSlot ** ExecForeignBatchInsert(EState *estate,

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-01-21 Thread Masahiko Sawada
On Fri, Dec 25, 2020 at 6:46 PM Masahiro Ikeda wrote: > > Hi, > > I rebased the patch to the master branch. Thank you for working on this. I've read the latest patch. Here are comments: --- + if (track_wal_io_timing) + { + INSTR_TIME_SET_CURRENT(dura

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-21 Thread Amit Kapila
On Fri, Jan 22, 2021 at 8:29 AM Greg Nancarrow wrote: > > On Thu, Jan 21, 2021 at 7:30 PM Amit Kapila wrote: > > > > > i.e. code-wise: > > > > > > /* > > > -* We can't support table modification in parallel-mode if > > > it's a foreign > > > -* table/partition (no FDW API

Re: adding wait_start column to pg_locks

2021-01-21 Thread torikoshia
On 2021-01-21 12:48, Fujii Masao wrote: Thanks for updating the patch! I think that this is really useful feature!! Thanks for reviewing! I have two minor comments. + role="column_definition"> + wait_start timestamptz The column name "wait_start" should be "waitstart" for the s

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-21 Thread Amit Kapila
On Thu, Jan 21, 2021 at 12:23 AM Peter Geoghegan wrote: > > On Wed, Jan 20, 2021 at 5:33 AM Amit Kapila wrote: > > > Victor independently came up with a benchmark that ran over several > > > hours, with cleanup consistently held back by ~5 minutes by a long > > > running transaction: > > > > > >

Re: New IndexAM API controlling index vacuum strategies

2021-01-21 Thread Peter Geoghegan
On Tue, Jan 19, 2021 at 2:57 PM Peter Geoghegan wrote: > Looks good. I'll give this version a review now. I will do a lot more > soon. I need to come up with a good benchmark for this, that I can > return to again and again during review as needed. I performed another benchmark, similar to the la

Re: PoC/WIP: Extended statistics on expressions

2021-01-21 Thread Justin Pryzby
On Thu, Jan 21, 2021 at 10:01:01PM -0600, Justin Pryzby wrote: > On Fri, Jan 22, 2021 at 04:49:51AM +0100, Tomas Vondra wrote: > > > > | Statistics objects: > > > > | "public"."s2" (ndistinct, dependencies, mcv) ON FROM t > > > > Umm, for me that prints: > > > "public"."s2" ON ((i + 1)),

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-21 Thread japin
On Fri, 22 Jan 2021 at 00:51, Bharath Rupireddy wrote: > On Thu, Jan 21, 2021 at 6:56 PM vignesh C wrote: >> >> Hi, >> >> Creating/altering subscription is successful when we specify a >> publication which does not exist in the publisher. I felt we should >> throw an error in this case, that w

Re: Added schema level support for publication.

2021-01-21 Thread vignesh C
Thanks Rahila for your comments. Please find my thoughts below: On Wed, Jan 20, 2021 at 6:27 PM Rahila Syed wrote: > > Hi Vignesh, > >> >> I have handled the above scenario(drop schema should automatically >> remove the schema entry from publication schema relation) & addition >> of tests in the

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-21 Thread Greg Nancarrow
On Fri, Jan 22, 2021 at 1:16 PM Hou, Zhijie wrote: > > Hi > > I took a look at v12-0001 patch, here are some comments: > > 1. > + /* > +* Setup the context used in finding the max parallel-mode hazard. > +*/ > + Assert(initial_max_parallel_hazard == 0 || > +

Re: PoC/WIP: Extended statistics on expressions

2021-01-21 Thread Justin Pryzby
On Fri, Jan 22, 2021 at 04:49:51AM +0100, Tomas Vondra wrote: > > > | Statistics objects: > > > | "public"."s2" (ndistinct, dependencies, mcv) ON FROM t > > Umm, for me that prints: > "public"."s2" ON ((i + 1)), (((i + 1) + 0)) FROM t > > which I think is OK. But maybe there's something

RE: libpq debug log

2021-01-21 Thread tsunakawa.ta...@fujitsu.com
Hello Alvaro-san, Iwata-san, First of all, thank you Alvaro-san really a lot for your great help. I'm glad you didn't lose interest and time for this patch yet. (Iwata-san is my colleague.) From: Alvaro Herrera > That's true, but it'd require that we move PQtrace() to fe-misc.c, because >

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-21 Thread Greg Nancarrow
On Thu, Jan 21, 2021 at 7:30 PM Amit Kapila wrote: > > > i.e. code-wise: > > > > /* > > -* We can't support table modification in parallel-mode if > > it's a foreign > > -* table/partition (no FDW API for supporting parallel access) or a > > +* We can't support tabl

RE: About to add WAL write/fsync statistics to pg_stat_wal view

2021-01-21 Thread kuroda.hay...@fujitsu.com
Dear Ikeda-san, This patch cannot be applied to the HEAD, but anyway I put a comment. ``` + /* +* Measure i/o timing to fsync WAL data. +* +* The wal receiver skip to collect it to avoid performance degradation of standy servers. +* If sync_method doesn't ha

Re: pglz compression performance, take two

2021-01-21 Thread Justin Pryzby
@cfbot: rebased >From 03fec5d2587cf34a1d1a75d7afdcfbad9cb7ec68 Mon Sep 17 00:00:00 2001 From: Andrey Date: Thu, 27 Jun 2019 23:18:21 +0500 Subject: [PATCH] Reorganize pglz compression code This patch accumulates several changes: 1. Convert macro-functions to regular functions for readability 2. U

Re: Single transaction in the tablesync worker?

2021-01-21 Thread Amit Kapila
On Thu, Jan 21, 2021 at 3:47 PM Amit Kapila wrote: > > On Tue, Jan 19, 2021 at 2:32 PM Peter Smith wrote: > > > > Hi Amit. > > > > PSA the v17 patch for the Tablesync Solution1. > > > > Thanks for the updated patch. Below are few comments: > One more comment: In LogicalRepSyncTableStart(), you

Re: [PoC] Non-volatile WAL buffer

2021-01-21 Thread Tomas Vondra
On 1/21/21 3:17 AM, Masahiko Sawada wrote: On Thu, Jan 7, 2021 at 2:16 AM Tomas Vondra wrote: Hi, I think I've managed to get the 0002 patch [1] rebased to master and working (with help from Masahiko Sawada). It's not clear to me how it could have worked as submitted - my theory is that an

Re: PoC/WIP: Extended statistics on expressions

2021-01-21 Thread Justin Pryzby
This already needs to be rebased on 55dc86eca. And needs to update rules.out. And doesn't address this one: On Sun, Jan 17, 2021 at 10:53:31PM -0600, Justin Pryzby wrote: > | postgres=# CREATE TABLE t(i int); > | postgres=# CREATE STATISTICS s2 ON (i+1) ,(i+1+0) FROM t; > | postgres=# \d t > |

RE: Parallel INSERT (INTO ... SELECT ...)

2021-01-21 Thread Hou, Zhijie
Hi I took a look at v12-0001 patch, here are some comments: 1. + /* +* Setup the context used in finding the max parallel-mode hazard. +*/ + Assert(initial_max_parallel_hazard == 0 || + initial_max_parallel_hazard == PROPARALLEL_SAFE || +

Re: OpenSSL connection setup debug callback issue

2021-01-21 Thread Michael Paquier
On Thu, Jan 21, 2021 at 05:01:15PM +0900, Michael Paquier wrote: > This is interesting for debugging, +1 for applying what you have > here, and this works for 1.0.1~3.0.0. Worth noting that this returns > a static string, as per ssl_stat.c. Done as of af0e79c, after an indentation. -- Michael s

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-21 Thread Greg Nancarrow
On Fri, Jan 22, 2021 at 12:08 PM Hou, Zhijie wrote: > > > > > I think that's a good idea, so I'll make that update in the next version > > of the patch. > > I do notice, however, that there seems to be quite a few places in the > > Postgres > > code where RelationGetIndexList() is being called wi

Re: Support for NSS as a libpq TLS backend

2021-01-21 Thread Jacob Champion
On Thu, 2021-01-21 at 14:21 +0900, Michael Paquier wrote: > Also, what's the minimum version of NSS that would be supported? It > would be good to define an acceptable older version, to keep that > documented and to track that perhaps with some configure checks (?), > similarly to what is done for

Re: vacuum_cost_page_miss default value and modern hardware

2021-01-21 Thread Peter Geoghegan
On Thu, Jan 14, 2021 at 8:34 PM Masahiko Sawada wrote: > +1 for this change. Lowering to 2 also looks good to me. I'm going to go ahead with committing my patch to lower the default next week. If anybody has any objections to that plan, please speak up. It doesn't really need to be said again, b

RE: Parallel INSERT (INTO ... SELECT ...)

2021-01-21 Thread Hou, Zhijie
> > > > + > > + index_oid_list = RelationGetIndexList(rel); > > ... > > > > As memtioned in the comments of RelationGetIndexList: > > * we return a copy of the list palloc'd in the caller's context. The > > caller > > * may list_free() the returned list after scanning it. > > > > Shall we li

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2021-01-21 Thread Michail Nikolaev
Hello, everyone. Oh, I just realized that it seems like I was too naive to allow standby to set LP_DEAD bits this way. There is a possible consistency problem in the case of low minRecoveryPoint value (because hint bits do not move PageLSN forward). Something like this: LSN=10 STANDBY INSERTS N

Re: PATCH: Batch/pipelining support for libpq

2021-01-21 Thread Zhihong Yu
Hi, + commandFailed(st, "SQL", "\\gset and \\aset are not allowed in a batch section"); It seems '\\gset or \\aset is not ' would correspond to the check more closely. + if (my_command->argc != 1) + syntax_error(source, lineno, my_command->first_line, my_com

Re: PATCH: Batch/pipelining support for libpq

2021-01-21 Thread Alvaro Herrera
As you can see in an XXX comment in the libpq test program, the current implementation has the behavior that PQgetResult() returns NULL after a batch is finished and has reported PGRES_BATCH_END. I don't know if there's a hard reason to do that, but I'd like to supress it because it seems weird an

Re: PATCH: Batch/pipelining support for libpq

2021-01-21 Thread Alvaro Herrera
Thanks David Johnston and Daniel Vérité, I have incorporated your changes into this patch, which is now v26. Also, it's been rebased on current sources. I've been using the new PQtrace() stuff to verify the behavior of the new feature. It's not perfect, but at least it doesn't crash immediately

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-21 Thread Sergey Shinderuk
On 22.01.2021 01:17, James Hilliard wrote: On Thu, Jan 21, 2021 at 11:38 AM Tom Lane wrote: James Hilliard writes: On Wed, Jan 20, 2021 at 4:07 PM Tom Lane wrote: I'm not sure that the case of not having the "command line tools" installed is interesting for our purposes. AFAIK you have to

Re: Avoiding smgrimmedsync() during nbtree index builds

2021-01-21 Thread Andres Freund
Hi, On 2021-01-21 23:54:04 +0200, Heikki Linnakangas wrote: > On 21/01/2021 22:36, Andres Freund wrote: > > A quick hack (probably not quite correct!) to evaluate the benefit shows > > that the attached script takes 2m17.223s with the smgrimmedsync and > > 0m22.870s passing skipFsync=false to writ

Very misleading documentation for PQreset()

2021-01-21 Thread Tom Lane
I happened to notice that PQreset is documented thus: This function will close the connection to the server and attempt to reestablish a new connection to the same server, using all the same parameters previously used. Since we invented multi-host connection parameters, a reasonable p

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-21 Thread James Hilliard
On Thu, Jan 21, 2021 at 11:38 AM Tom Lane wrote: > > James Hilliard writes: > > On Wed, Jan 20, 2021 at 4:07 PM Tom Lane wrote: > >> I'm not sure that the case of not having the "command line tools" > >> installed is interesting for our purposes. AFAIK you have to have > >> that in order to hav

Re: strange error reporting

2021-01-21 Thread Tom Lane
I wrote: > If I don't hear any other opinions, I'll change these messages to > "connection to server at socket \"%s\" failed: " > "connection to server at \"%s\" (%s), port %s failed: " Done. Also, here is a patch to remove the redundant-seeming prefixes from our reports of connection failures.

Re: Avoiding smgrimmedsync() during nbtree index builds

2021-01-21 Thread Heikki Linnakangas
On 21/01/2021 22:36, Andres Freund wrote: Hi, Every nbtree index build currently does an smgrimmedsync at the end: /* * Read tuples in correct sort order from tuplesort, and load them into * btree leaves. */ static void _bt_load(BTWriteState *wstate, BTSpool *btspool, BTSpool *btspool2) .

Re: libpq debug log

2021-01-21 Thread Alvaro Herrera
On 2021-Jan-17, tsunakawa.ta...@fujitsu.com wrote: > * I don't see the need for separate pqTraceInit() function, because it is > only called here. That's true, but it'd require that we move PQtrace() to fe-misc.c, because pqTraceInit() uses definitions that are private to that file. > (2) > +bo

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-21 Thread Justin Pryzby
On Thu, Jan 21, 2021 at 11:48:08PM +0300, Alexey Kondratov wrote: > Attached is a new patch set of first two patches, that should resolve all > the issues raised before (ACL, docs, tests) excepting TOAST. Double thanks > for suggestion to add more tests with nested partitioning. I have found and >

Re: Is Recovery actually paused?

2021-01-21 Thread Robert Haas
On Mon, Jan 18, 2021 at 9:42 PM Yugo NAGATA wrote: > If it is acceptable that pg_is_wal_replay_paused() makes users wait, > I'm ok for the current interface. I don't feel the need of > pg_is_wal_replay_paluse_requeseted(). Another idea could be that pg_is_wal_replay_paused() could be changed to t

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-21 Thread Alexey Kondratov
On 2021-01-21 17:06, Alexey Kondratov wrote: On 2021-01-21 04:41, Michael Paquier wrote: There are no tests for partitioned tables, aka we'd want to make sure that the new partitioned index is on the correct tablespace, as well as all its leaves. It may be better to have at least two levels of

Avoiding smgrimmedsync() during nbtree index builds

2021-01-21 Thread Andres Freund
Hi, Every nbtree index build currently does an smgrimmedsync at the end: /* * Read tuples in correct sort order from tuplesort, and load them into * btree leaves. */ static void _bt_load(BTWriteState *wstate, BTSpool *btspool, BTSpool *btspool2) ... /* * When we WAL-logged ind

Re: Race condition in recovery?

2021-01-21 Thread Robert Haas
On Thu, Jan 21, 2021 at 4:00 AM Dilip Kumar wrote: > 8. Node3, get it because walsender of Node2 read it from TL13 and send > it and Node2 write in the new WAL file but with TL12. > > WalSndSegmentOpen() > { > /*--- > * When reading from a historic timeline, and there is a timeline switch > *

Re: Support for NSS as a libpq TLS backend

2021-01-21 Thread Jacob Champion
On Mon, 2020-07-20 at 15:35 +0200, Daniel Gustafsson wrote: > With this, I have one failing test ("intermediate client certificate is > provided by client") which I've left failing since I believe the case should > be > supported by NSS. The issue is most likely that I havent figured out the > r

Re: [HACKERS] [PATCH] Generic type subscripting

2021-01-21 Thread Pavel Stehule
Hi > Looks good, I've applied it, thanks. > I tested last set of patches 1. There is no problem with patching and compilation 2. make check-world passed 3. build doc without problems 4. I have not any objections against implemented functionality, implementation and tests I'll mark this patch a

Re: {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2021-01-21 Thread Matthias van de Meent
On Tue, 19 Jan 2021 at 21:59, Matthias van de Meent wrote: > > On Mon, 18 Jan 2021, 21:25 Álvaro Herrera, wrote: > > > > On 2021-Jan-18, Matthias van de Meent wrote: > > > > > Example: > > > > > > 1.) RI starts > > > 2.) PHASE 2: filling the index: > > > 2.1.) scanning the heap (live tuple is cac

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-21 Thread Tom Lane
James Hilliard writes: > On Wed, Jan 20, 2021 at 4:07 PM Tom Lane wrote: >> I'm not sure that the case of not having the "command line tools" >> installed is interesting for our purposes. AFAIK you have to have >> that in order to have access to required tools like bison and gmake. >> (That remi

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-21 Thread Fujii Masao
On 2021/01/22 1:17, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 8:58 PM Fujii Masao wrote: My opinion is to check "!all", but if others prefer using such boolean flag, I'd withdraw my opinion. I'm really sorry, actually if (!all) is enough there, my earlier understanding was wrong.

Re: Add primary keys to system catalogs

2021-01-21 Thread Tom Lane
Peter Eisentraut writes: > On 2021-01-17 23:07, Tom Lane wrote: >> I've reviewed this patch. It looks pretty solid to me, with a couple >> trivial nits as mentioned below, and one bigger thing that's perhaps >> in the category of bikeshedding. Namely, do we really want to prefer >> using the OID

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-21 Thread Bharath Rupireddy
On Thu, Jan 21, 2021 at 6:56 PM vignesh C wrote: > > Hi, > > Creating/altering subscription is successful when we specify a > publication which does not exist in the publisher. I felt we should > throw an error in this case, that will help the user to check if there > is any typo in the create sub

Why does creating logical replication subscriptions require superuser?

2021-01-21 Thread Paul Martinez
Hey, all, I'm working with native logical replication, and I don't fully understand why logical replication subscribers need to be superusers, nor do I fully understand the implication of some of the comments made on this page: https://www.postgresql.org/docs/current/logical-replication-security.

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-21 Thread Bharath Rupireddy
On Thu, Jan 21, 2021 at 8:58 PM Fujii Masao wrote: > My opinion is to check "!all", but if others prefer using such boolean flag, > I'd withdraw my opinion. I'm really sorry, actually if (!all) is enough there, my earlier understanding was wrong. > + if ((all || entry->server_hashv

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-21 Thread Fujii Masao
On 2021/01/21 16:16, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 12:17 PM Fujii Masao wrote: On 2021/01/21 14:46, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 10:06 AM Fujii Masao wrote: > >> + if (entry->server_hashvalue == hashvalue && +

[PATCH] Disable bgworkers during servers start in pg_upgrade

2021-01-21 Thread Denis Laxalde
Hello, We found an issue in pg_upgrade on a cluster with a third-party background worker. The upgrade goes fine, but the new cluster is then in an inconsistent state. The background worker comes from the PoWA extension but the issue does not appear to related to this particular code. Here is a sh

postgres_fdw: IMPORT FOREIGN SCHEMA ... LIMIT TO (partition)

2021-01-21 Thread Matthias van de Meent
Hi, Recently I was trying to copy some of the data of one database to another through postgres_fdw, and found that it wouldn't import that partition through IMPORT FOREIGN SCHEMA, even when I explicitly specified the name of the table that contained the data in the LIMIT TO clause. I realised the

Re: Stronger safeguard for archive recovery not to miss data

2021-01-21 Thread Laurenz Albe
On Thu, 2021-01-21 at 13:09 +, osumi.takami...@fujitsu.com wrote: > > My vote is that we should not have a GUC for such an unlikely event, and > > that > > stopping recovery is good enough. > > OK. IIUC, my current patch for this fix doesn't need to be changed or > withdrawn. > Thank you for

Re: New IndexAM API controlling index vacuum strategies

2021-01-21 Thread Masahiko Sawada
On Wed, Jan 20, 2021 at 7:58 AM Peter Geoghegan wrote: > > On Sun, Jan 17, 2021 at 9:18 PM Masahiko Sawada wrote: > > After more thought, I think that ambulkdelete needs to be able to > > refer the answer to amvacuumstrategy. That way, the index can skip > > bulk-deletion when lazy vacuum doesn't

RE: Enhance traceability of wal_level changes for backup management

2021-01-21 Thread osumi.takami...@fujitsu.com
Hi Apologies for my delay. On Wednesday, January 6, 2021 7:03 PM I wrote: > I'll continue the discussion of [2]. > We talked about how to recognize the time or LSN when/where wal_level is > changed to 'none' there. > > You said > > The use case I imagined is that the user temporarily changes wal

Re: patch: reduce overhead of execution of CALL statement in no atomic mode from PL/pgSQL

2021-01-21 Thread Pavel Stehule
čt 21. 1. 2021 v 14:37 odesílatel Pavel Stehule napsal: > Hi > > This is a little bit of an enhanced version of the previous patch. The > worst case overhead is reduced almost to zero. The local resource owner is > created only when routine is executed in non-atomic mode, and when routine > conta

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-21 Thread Alexey Kondratov
On 2021-01-21 04:41, Michael Paquier wrote: On Wed, Jan 20, 2021 at 03:34:39PM -0300, Alvaro Herrera wrote: On 2021-Jan-20, Alexey Kondratov wrote: Ugh, forgot to attach the patches. Here they are. Yeah, looks reasonable. + + if (changed) + /* Record dependency on table

Re: patch: reduce overhead of execution of CALL statement in no atomic mode from PL/pgSQL

2021-01-21 Thread Pavel Stehule
Hi This is a little bit of an enhanced version of the previous patch. The worst case overhead is reduced almost to zero. The local resource owner is created only when routine is executed in non-atomic mode, and when routine contains a CALL statement. Regards Pavel diff --git a/doc/src/sgml/spi.s

Re: Jsonpath ** vs lax mode

2021-01-21 Thread Alvaro Herrera
On 2021-Jan-21, Alexander Korotkov wrote: > Requiring strict mode for ** is a solution, but probably too restrictive... > > What do you think about making just subsequent accessor after ** not > to unwrap arrays. That would be a bit tricky to implement, but > probably that would better satisfy t

Identify missing publications from publisher while create/alter subscription.

2021-01-21 Thread vignesh C
Hi, Creating/altering subscription is successful when we specify a publication which does not exist in the publisher. I felt we should throw an error in this case, that will help the user to check if there is any typo in the create subscription command or to create the publication before the subsc

Re: Boundary value check in lazy_tid_reaped()

2021-01-21 Thread Masahiko Sawada
On Wed, Jan 20, 2021 at 3:50 PM Peter Eisentraut wrote: > > On 2020-10-30 02:43, Masahiko Sawada wrote: > > Using the integer set is very memory efficient (5MB vs. 114MB in the > > base case) and there is no 1GB limitation. Looking at the execution > > time, I had expected that using the integer s

RE: Stronger safeguard for archive recovery not to miss data

2021-01-21 Thread osumi.takami...@fujitsu.com
Hi, Laurenz On Thursday, January 21, 2021 9:51 PM Laurenz Albe wrote: > On Thu, 2021-01-21 at 11:49 +, osumi.takami...@fujitsu.com wrote: > > Adding a condition to check if "recovery_allow_data_corruption" is > > 'on' around the end of > > CheckRequiredParameterValues() sounds safer for me

Re: patch: reduce overhead of execution of CALL statement in no atomic mode from PL/pgSQL

2021-01-21 Thread Pavel Stehule
Hi pá 15. 1. 2021 v 22:46 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > [ plpgsql-using-local-resowner-for-call-plans-20200108.patch ] > > I took a quick look through this patch, just reading it without > any testing. A few thoughts: > > * Instead of adding an argument to GetCachedPl

Re: Stronger safeguard for archive recovery not to miss data

2021-01-21 Thread Laurenz Albe
On Thu, 2021-01-21 at 11:49 +, osumi.takami...@fujitsu.com wrote: > Adding a condition to check if "recovery_allow_data_corruption" is 'on' > around the end of > CheckRequiredParameterValues() sounds safer for me too, although > implementing a new GUC parameter sounds bigger than what I expect

Re: Is Recovery actually paused?

2021-01-21 Thread Yugo NAGATA
On Tue, 19 Jan 2021 21:32:31 +0530 Dilip Kumar wrote: > On Tue, Jan 19, 2021 at 8:34 AM Dilip Kumar wrote: > > > > On Tue, 19 Jan 2021 at 8:12 AM, Yugo NAGATA wrote: > >> > >> On Sun, 17 Jan 2021 11:33:52 +0530 > >> Dilip Kumar wrote: > >> > >> > On Thu, Jan 14, 2021 at 6:49 PM Yugo NAGATA wr

Re: Additional Chapter for Tutorial - arch-dev.sgml

2021-01-21 Thread Jürgen Purtz
On 18.01.21 15:13, Heikki Linnakangas wrote: On 20/11/2020 23:52, Erik Rijkers wrote: (smallish) Changes to arch-dev.sgml This looks good to me. One little complaint: @@ -125,7 +122,7 @@ use a supervisor process (also master process) that spawns a new server process every time

Re: Columns correlation and adaptive query optimization

2021-01-21 Thread Yugo NAGATA
Hello, On Thu, 26 Mar 2020 18:49:51 +0300 Konstantin Knizhnik wrote: > Attached please find new version of the patch with more comments and > descriptions added. Adaptive query optimization is very interesting feature for me, so I looked into this patch. Here are some comments and questions.

Re: [HACKERS] [PATCH] Generic type subscripting

2021-01-21 Thread Dmitry Dolgov
> On Wed, Jan 20, 2021 at 11:37:32PM -0500, Dian M Fay wrote: > On Wed Jan 20, 2021 at 2:08 PM EST, Dmitry Dolgov wrote: > > > On Wed, Jan 20, 2021 at 11:34:16AM -0500, Dian M Fay wrote: > > > > Thanks, I need to remember to not skipp doc building for testing process > > > > even for such small cha

Re: Is Recovery actually paused?

2021-01-21 Thread Dilip Kumar
On Thu, Jan 21, 2021 at 3:29 PM Bharath Rupireddy wrote: Thanks for reviewing Bharat. > On Tue, Jan 19, 2021 at 9:32 PM Dilip Kumar wrote: > > In the last patch there were some local changes which I did not add to > > the patch and it was giving compilation warning so fixed that along > > with

RE: Stronger safeguard for archive recovery not to miss data

2021-01-21 Thread osumi.takami...@fujitsu.com
Hi On Wed, Jan 20, 2021 9:03 PM Laurenz Albe wrote: > > On Wed, 2021-01-20 at 13:10 +0900, Fujii Masao wrote: > > +errhint("Run recovery again from a > > + new base backup taken after setting wal_level higher than > > + minimal"))); > > > > Isn't it impossible to

Re: PoC/WIP: Extended statistics on expressions

2021-01-21 Thread Dean Rasheed
On Tue, 19 Jan 2021 at 01:57, Tomas Vondra wrote: > > > A slightly bigger issue that I don't like is the way it assigns > > attribute numbers for expressions starting from > > MaxHeapAttributeNumber+1, so the first expression has an attnum of > > 1601. That leads to pretty inefficient use of Bitma

Re: Printing LSN made easy

2021-01-21 Thread Ashutosh Bapat
On Thu, Jan 21, 2021 at 3:53 PM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2021-01-20 08:50, Ashutosh Bapat wrote: > > Thanks for looking into this. I would like to keep both the LSN_FORMAT > > and LSN_FORMAT_ARGS but with a note that the first can not be used in > > elog() o

Re: Printing LSN made easy

2021-01-21 Thread Peter Eisentraut
On 2021-01-20 08:50, Ashutosh Bapat wrote: Thanks for looking into this. I would like to keep both the LSN_FORMAT and LSN_FORMAT_ARGS but with a note that the first can not be used in elog() or in messages which require localization. We have many other places doing snprintf() and such stuff, wh

Re: Single transaction in the tablesync worker?

2021-01-21 Thread Amit Kapila
On Tue, Jan 19, 2021 at 2:32 PM Peter Smith wrote: > > Hi Amit. > > PSA the v17 patch for the Tablesync Solution1. > Thanks for the updated patch. Below are few comments: 1. Why are we changing the scope of PG_TRY in DropSubscription()? Also, it might be better to keep the replication slot drop p

Re: ResourceOwner refactoring

2021-01-21 Thread Heikki Linnakangas
On 21/01/2021 06:14, Michael Paquier wrote: On Thu, Jan 21, 2021 at 12:11:37AM +0200, Heikki Linnakangas wrote: Summary: In the the worst scenario, the patched version is still 24% slower than unpatched. But many other scenarios are now faster with the patch. Is there a reason explaining the s

Re: Is Recovery actually paused?

2021-01-21 Thread Bharath Rupireddy
On Tue, Jan 19, 2021 at 9:32 PM Dilip Kumar wrote: > In the last patch there were some local changes which I did not add to > the patch and it was giving compilation warning so fixed that along > with that I have addressed your this comment as well. Thanks for the patch. I took a look at the v5 p

Refactor SSL test framework to support multiple TLS libraries

2021-01-21 Thread Daniel Gustafsson
In an attempt to slice off as much non-NSS specific changes as possible from the larger libnss patch proposed in [0], the attached patch contains the ssl test harness refactoring to support multiple TLS libraries. The changes are mostly a refactoring to hide library specific setup in their own mod

Re: Add primary keys to system catalogs

2021-01-21 Thread Peter Eisentraut
On 2021-01-17 23:07, Tom Lane wrote: I've reviewed this patch. It looks pretty solid to me, with a couple trivial nits as mentioned below, and one bigger thing that's perhaps in the category of bikeshedding. Namely, do we really want to prefer using the OID indexes as the primary keys? In most

Re: Jsonpath ** vs lax mode

2021-01-21 Thread Thomas Kellerer
Alexander Korotkov schrieb am 20.01.2021 um 18:13: > We have a bug report which says that jsonpath ** operator behaves strangely > in the lax mode [1]. That report was from me ;) Thanks for looking into it. > At first sight, we may just say that lax mode just sucks and > counter-intuitive resul

Re: Jsonpath ** vs lax mode

2021-01-21 Thread Alexander Korotkov
Hi, Alvaro! Thank you for your feedback. On Wed, Jan 20, 2021 at 9:16 PM Alvaro Herrera wrote: > On 2021-Jan-20, Alexander Korotkov wrote: > > > My proposal is to make everything after the ** operator use strict mode > > (patch attached). I think this shouldn't be backpatched, just applied to >

Re: Wrong usage of RelationNeedsWAL

2021-01-21 Thread Noah Misch
On Thu, Jan 21, 2021 at 06:02:11PM +0900, Kyotaro Horiguchi wrote: > At Thu, 21 Jan 2021 00:19:58 -0800, Noah Misch wrote in > > On Thu, Jan 21, 2021 at 12:28:44AM +0900, Kyotaro Horiguchi wrote: > > > However, with the previous patch, two existing tests sto_using_cursor > > > and sto_using_selec

RE: Parallel INSERT (INTO ... SELECT ...)

2021-01-21 Thread Hou, Zhijie
> > So I think we're saying that if the target table is a foreign table or > > temporary table, it can be regarded as PARALLEL_RESTRICTED, right? > > > > Yes > > IMO, PARALLEL_RESTRICTED currently enable parallel select but disable > parallel insert. > So, the INSERT only happen in leader worker

Re: Wrong usage of RelationNeedsWAL

2021-01-21 Thread Kyotaro Horiguchi
At Thu, 21 Jan 2021 00:19:58 -0800, Noah Misch wrote in > On Thu, Jan 21, 2021 at 12:28:44AM +0900, Kyotaro Horiguchi wrote: > > At Wed, 20 Jan 2021 17:34:44 +0900 (JST), Kyotaro Horiguchi > > wrote in > > > Anyway, it seems actually dangerous that cause pruning on wal-skipped > > > relation.

Race condition in recovery?

2021-01-21 Thread Dilip Kumar
While analyzing one of the customer issues, based on the log it appeared that there is a race condition in the recovery process. The summary of the issue is, that one of the standby is promoted as the new primary (Node2) and another standby (Node3) is restarted and set the primary_info and the res

RE: Parallel INSERT (INTO ... SELECT ...)

2021-01-21 Thread Hou, Zhijie
> > > Hi > > > > > > > > I may be wrong, and if I miss sth in previous mails, please give > > > > > me some > > > > hints. > > > > > IMO, serial insertion with underlying parallel SELECT can be > > > > > considered for foreign table or temporary table, as the > > > > > insertions only > > > > happe

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-21 Thread Sergey Shinderuk
On 21.01.2021 02:07, Tom Lane wrote: I now believe what is actually happening with the short command is that it's iterating through the available SDKs (according to some not very clear search path) and picking the first one it finds that matches the host system version. That matches the ktrace e

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-21 Thread Amit Kapila
On Thu, Jan 21, 2021 at 12:44 PM Greg Nancarrow wrote: > > On Wed, Dec 23, 2020 at 1:45 PM Amit Kapila wrote: > > > > On Wed, Dec 23, 2020 at 7:52 AM Hou, Zhijie > > wrote: > > > > > > Hi > > > > > > > > I may be wrong, and if I miss sth in previous mails, please give me > > > > > some > > > >

Re: [HACKERS] make async slave to wait for lsn to be replayed

2021-01-21 Thread Kyotaro Horiguchi
Hello. At Wed, 18 Nov 2020 15:05:00 +0300, a.pervush...@postgrespro.ru wrote in > I've changed the BEGIN WAIT FOR LSN statement to core functions > pg_waitlsn, pg_waitlsn_infinite and pg_waitlsn_no_wait. > Currently the functions work inside repeatable read transactions, but > waitlsn creates a s

Re: Wrong usage of RelationNeedsWAL

2021-01-21 Thread Noah Misch
On Thu, Jan 21, 2021 at 12:28:44AM +0900, Kyotaro Horiguchi wrote: > At Wed, 20 Jan 2021 17:34:44 +0900 (JST), Kyotaro Horiguchi > wrote in > > Anyway, it seems actually dangerous that cause pruning on wal-skipped > > relation. > > > > > with your patch versions. Could you try implementing bot

Re: OpenSSL connection setup debug callback issue

2021-01-21 Thread Michael Paquier
On Thu, Dec 10, 2020 at 02:43:33PM +0100, Daniel Gustafsson wrote: > I went looking at the SSL connection state change information callback we > install when setting up connections with OpenSSL, and I wasn't getting the > state changes I expected. Turns out we install it at the tail end of setting