Re: simplifying foreign key/RI checks

2021-01-22 Thread Amit Langote
On Fri, Jan 22, 2021 at 3:22 PM Corey Huinker wrote: >> I decided not to deviate from pk_ terminology so that the new code >> doesn't look too different from the other code in the file. Although, >> I guess we can at least call the main function >> ri_ReferencedKeyExists() instead of

Re: proposal - idea - enhancing plpgsql FOREACH for JSON, jsonb and hstore

2021-01-22 Thread Julien Rouhaud
On Sat, Jan 23, 2021 at 07:46:01AM +0100, Pavel Stehule wrote: > Hi > > jsonb with subscripting support can be used as a dictionary object in > plpgsql. > > Can be nice to have support for iteration over a set of tuples (key, > value). > > Some like > > FOREACH fieldvar [ KEY keyvar] IN

proposal - idea - enhancing plpgsql FOREACH for JSON, jsonb and hstore

2021-01-22 Thread Pavel Stehule
Hi jsonb with subscripting support can be used as a dictionary object in plpgsql. Can be nice to have support for iteration over a set of tuples (key, value). Some like FOREACH fieldvar [ KEY keyvar] IN DICTIONARY sourceexpr [VALUE searchexpr] LOOP END LOOP; and for JSON arrays FOREACH var

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-22 Thread Amit Kapila
On Fri, Jan 22, 2021 at 2:59 PM Bharath Rupireddy wrote: > > On Fri, Jan 22, 2021 at 2:00 PM Amit Kapila wrote: > > Yes you are right. Looks like the above commit is causing the issue. I > reverted that commit and did not see the drop publication bug, see use > case [1]. > Okay, thanks for

Re: Single transaction in the tablesync worker?

2021-01-22 Thread Ajin Cherian
On Sat, Jan 23, 2021 at 3:16 PM Amit Kapila wrote: > > I think so. But do you have any reason to believe that it won't be > required anymore? A temporary slot will not clash with a permanent slot of the same name, regards, Ajin Cherian Fujitsu

Re: doc review for v14

2021-01-22 Thread Michael Paquier
On Fri, Jan 22, 2021 at 09:53:13PM +0900, Masahiko Sawada wrote: > It seems to me that all work has been done. Can we mark this patch > entry as "Committed"? Or waiting for something on the author? Patch 0005 posted on [1], related to some docs of replication slots, still needs a lookup. [1]:

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-22 Thread Sergey Shinderuk
On 23.01.2021 08:02, Sergey Shinderuk wrote: I checked the script [1], and it really requires the CLT. Here is the explanation [2] for this: There is actually no such requirement. However, there are formulae that will be forced to build from source if you do not have the CLT.

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-22 Thread Sergey Shinderuk
On 22.01.2021 20:12, Tom Lane wrote: [ pokes more carefully... ] Ah-hah, I see why I needed the CLT. I bet you'll find that you can't build from "git clean -dfx" state with only Xcode, because comparing the contents of /Applications/Xcode.app/Contents/Developer/usr/bin and

Re: Is Recovery actually paused?

2021-01-22 Thread Dilip Kumar
On Thu, Jan 21, 2021 at 6:20 PM Yugo NAGATA wrote: > > On Tue, 19 Jan 2021 21:32:31 +0530 > Dilip Kumar wrote: > > > On Tue, Jan 19, 2021 at 8:34 AM Dilip Kumar wrote: > > > > > > On Tue, 19 Jan 2021 at 8:12 AM, Yugo NAGATA wrote: > > >> > > >> On Sun, 17 Jan 2021 11:33:52 +0530 > > >> Dilip

Re: Refactoring HMAC in the core code

2021-01-22 Thread Michael Paquier
On Fri, Jan 08, 2021 at 04:11:53PM +0900, Michael Paquier wrote: > Please find attached a rebased version. I have simplified the > implementation to use an opaque pointer similar to the cryptohash > part, leading to a large cleanup of the allocation logic for both > implementations, with and

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-22 Thread Bharath Rupireddy
On Fri, Jan 22, 2021 at 6:43 PM Fujii Masao wrote: > >> Please review the v16 patch set further. > > > > Thanks! Will review that later. > > + /* > +* For the given server, if we closed connection or > it is still in > +* use,

Re: Race condition in recovery?

2021-01-22 Thread Dilip Kumar
On Fri, Jan 22, 2021 at 2:05 AM Robert Haas wrote: > > On Thu, Jan 21, 2021 at 4:00 AM Dilip Kumar wrote: > > 8. Node3, get it because walsender of Node2 read it from TL13 and send > > it and Node2 write in the new WAL file but with TL12. > > > > WalSndSegmentOpen() > > { > > /*--- > > *

Re: Is Recovery actually paused?

2021-01-22 Thread Dilip Kumar
On Fri, Jan 22, 2021 at 2:18 AM Robert Haas wrote: > > On Mon, Jan 18, 2021 at 9:42 PM Yugo NAGATA wrote: > > If it is acceptable that pg_is_wal_replay_paused() makes users wait, > > I'm ok for the current interface. I don't feel the need of > > pg_is_wal_replay_paluse_requeseted(). > > Another

Re: Single transaction in the tablesync worker?

2021-01-22 Thread Amit Kapila
On Sat, Jan 23, 2021 at 8:37 AM Ajin Cherian wrote: > > On Thu, Jan 21, 2021 at 9:17 PM Amit Kapila wrote: > > > 7. > > +# check for occurrence of the expected error > > +poll_output_until("replication slot \"$slotname\" already exists") > > +or die "no error stop for the pre-existing

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2021-01-22 Thread Tomas Vondra
On 1/23/21 1:10 AM, Tom Lane wrote: Pavan Deolasee writes: On Mon, Jan 18, 2021 at 3:02 AM Tomas Vondra wrote: Pushed. Thanks everyone for the effort put into this patch. The first version was sent in 2015, so it took quite a bit of time. Thanks Tomas, Anastasia and everyone else who

Re: Single transaction in the tablesync worker?

2021-01-22 Thread Ajin Cherian
On Thu, Jan 21, 2021 at 9:17 PM Amit Kapila wrote: > 7. > +# check for occurrence of the expected error > +poll_output_until("replication slot \"$slotname\" already exists") > +or die "no error stop for the pre-existing origin"; > > In this test, isn't it better to check for datasync state

Re: WIP: BRIN multi-range indexes

2021-01-22 Thread Tomas Vondra
On 1/23/21 12:27 AM, John Naylor wrote: On Thu, Jan 21, 2021 at 9:06 PM Tomas Vondra mailto:tomas.von...@enterprisedb.com>> wrote: > [wip optimizations] > The pathological case (monotonic-asc) is now 4x faster, roughly equal to > regular minmax index build. The opposite (monotonic-desc)

Re: Some more hackery around cryptohashes (some fixes + SHA1)

2021-01-22 Thread Michael Paquier
On Fri, Jan 22, 2021 at 03:50:04PM +0200, Heikki Linnakangas wrote: > Leftover reference to "contrib/pgcrypto/sha1.h" > > Other than that, looks good to me. Thanks! I have looked at that again this morning, and this was still one indentation short. I have also run more tests with different

Re: Single transaction in the tablesync worker?

2021-01-22 Thread Peter Smith
On Thu, Jan 21, 2021 at 9:17 PM Amit Kapila wrote: > > On Tue, Jan 19, 2021 at 2:32 PM Peter Smith wrote: > > > > Hi Amit. > > > > PSA the v17 patch for the Tablesync Solution1. > > > > Thanks for the updated patch. Below are few comments: > 1. Why are we changing the scope of PG_TRY in

Re: Single transaction in the tablesync worker?

2021-01-22 Thread Peter Smith
On Fri, Jan 22, 2021 at 1:43 PM Amit Kapila wrote: > > On Thu, Jan 21, 2021 at 3:47 PM Amit Kapila wrote: > > > > On Tue, Jan 19, 2021 at 2:32 PM Peter Smith wrote: > > > > > > Hi Amit. > > > > > > PSA the v17 patch for the Tablesync Solution1. > > > > > > > Thanks for the updated patch. Below

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2021-01-22 Thread Tom Lane
Pavan Deolasee writes: > On Mon, Jan 18, 2021 at 3:02 AM Tomas Vondra > wrote: >> Pushed. Thanks everyone for the effort put into this patch. The first >> version was sent in 2015, so it took quite a bit of time. > Thanks Tomas, Anastasia and everyone else who worked on the patch and > ensured

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-22 Thread Peter Geoghegan
On Thu, Jan 21, 2021 at 9:23 PM Amit Kapila wrote: > > Slowing down non-HOT updaters in these extreme cases may actually be a > > good thing, even when bottom-up deletion finally becomes ineffective. > > It can be thought of as backpressure. I am not worried about slowing > > down something that

Re: WIP: BRIN multi-range indexes

2021-01-22 Thread John Naylor
On Thu, Jan 21, 2021 at 9:06 PM Tomas Vondra wrote: > [wip optimizations] > The pathological case (monotonic-asc) is now 4x faster, roughly equal to > regular minmax index build. The opposite (monotonic-desc) is about 33% > faster, roughly in line with btree. Those numbers look good. I get

Re: Single transaction in the tablesync worker?

2021-01-22 Thread Peter Smith
Hi Amit. PSA the v18 patch for the Tablesync Solution1. Main differences from v17: + Design change to use TEMPORARY tablesync slots [ak0122] means lots of the v17 slot cleanup code became unnecessary. + Small refactor in LogicalReplicationSyncTableStart to fix a deadlock scenario. + Addressing

Re: a verbose option for autovacuum

2021-01-22 Thread Tommy Li
Hey Tom > Seems like that would very soon feel like log spam. What would be the > use case for having this on? If you want one-off results you could > run VACUUM manually. In my case I have a fairly large, fairly frequently updated table with a large number of indexes where autovacuum's

Re: a verbose option for autovacuum

2021-01-22 Thread Tom Lane
Tommy Li writes: > I was surprised to see that there's no way to get `VACUUM VERBOSE`-like > output from autovacuum. Is there any interest in enabling this? Seems like that would very soon feel like log spam. What would be the use case for having this on? If you want one-off results you could

Re: Polyphase merge is obsolete

2021-01-22 Thread Heikki Linnakangas
On 22/10/2020 14:48, Heikki Linnakangas wrote: On 11/09/2017 13:37, Tomas Vondra wrote: I planned to do some benchmarking on this patch, but apparently the patch no longer applies. Rebase please? Here's a rebase of this. Sorry to keep you waiting :-). Here's an updated version that fixes

Re: Why does creating logical replication subscriptions require superuser?

2021-01-22 Thread Paul Martinez
> We successfully created exploits against Aiven and AWS RDS services gaining > superuser with their ways of subscription creation (and reported > vulnerabilities, of course). Probably, you have this (not so scary) > vulnerability too. Can you share the rough idea of how these exploits work? What

a verbose option for autovacuum

2021-01-22 Thread Tommy Li
Hi all I was surprised to see that there's no way to get `VACUUM VERBOSE`-like output from autovacuum. Is there any interest in enabling this? Additionally, is there any interest in exposing more vacuum options to be run by autovac? Right now it runs FREEZE and ANALYZE, which leaves the VERBOSE,

GSoC 2021

2021-01-22 Thread Stephen Frost
Greetings -hackers, Google Summer of Code is back for 2021! They have changed some of how GSoC is going to work for this year, for a variety of reasons, so please be sure to read this email and posts linked for the updates if you're interested! Everyone interested in suggesting projects or

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-22 Thread James Hilliard
On Thu, Jan 21, 2021 at 11:38 AM Tom Lane wrote: > > James Hilliard writes: > > On Wed, Jan 20, 2021 at 4:07 PM Tom Lane wrote: > >> I'm not sure that the case of not having the "command line tools" > >> installed is interesting for our purposes. AFAIK you have to have > >> that in order to

[PATCH v3 1/1] Fix detection of preadv/pwritev support for OSX.

2021-01-22 Thread James Hilliard
Fixes: gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2 -I../../../../src/include -isysroot

Re: plpgsql variable assignment not supporting distinct anymore

2021-01-22 Thread Tom Lane
Pavel Stehule writes: > pá 22. 1. 2021 v 14:41 odesílatel napsal: >> ERROR: syntax error at or near "DISTINCT" >> LINE 8: _test := DISTINCT a FROM ( VALUES ( (true), ( true ) ) )... > Using DISTINCT is not a good solution. Yeah. It wouldn't be as painful to support this in the grammar as

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-22 Thread Tom Lane
Sergey Shinderuk writes: > If I run the Homebrew installer, it says that it's going to install the > Command Line Tools. I don't know why it needs them, all the tools are > there already. I thought that CLT is a lighter-weight option when you > don't want the full Xcode installation, but

Re: mkid reference

2021-01-22 Thread Tom Lane
Daniel Gustafsson writes: >> On 22 Jan 2021, at 12:56, Magnus Hagander wrote: >> And maybe even more interestnig -- is there a point to this whole >> make_diff directory at all in these days of git? Or should we just >> remove it rather than try to fix it? > There's also src/tools/make_mkid

Re: macOS SIP, next try

2021-01-22 Thread Tom Lane
BTW, a couple other things that should be noted here: * Per observation in a nearby thread, install_name_tool seems to be provided by Apple's "Command Line Tools", but not by Xcode. This is also true of bison and flex, but we don't require those in a build-from-tarball. So relying on

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-22 Thread Tom Lane
Sergey Shinderuk writes: > I rejected to install CLT and abandoned Homebrew. Then I just cloned and > built Postgres successfully. So it looks like Xcode is really enough, at > least on a recent macOS version. Hm. I seem to recall having had to install CLT as well as Xcode back in the day,

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-01-22 Thread japin
Hi, Masahiro Thanks for you update the v4 patch. Here are some comments: (1) + char*msg = NULL; + boolsync_called;/* whether to sync data to the disk. */ + instr_time start; + instr_time duration; + + /* check

Re: [PoC] Non-volatile WAL buffer

2021-01-22 Thread Konstantin Knizhnik
On 22.01.2021 5:32, Tomas Vondra wrote: On 1/21/21 3:17 AM, Masahiko Sawada wrote: On Thu, Jan 7, 2021 at 2:16 AM Tomas Vondra wrote: Hi, I think I've managed to get the 0002 patch [1] rebased to master and working (with help from Masahiko Sawada). It's not clear to me how it could

Re: CTAS command tags

2021-01-22 Thread Tom Lane
Andreas Karlsson writes: > On 1/22/21 2:19 PM, Darafei "Komяpa" Praliaskouski wrote: >> Having row count right away is very useful in CTAS in analytical and GIS >> usage scenarios. > I can see that, but would it not work if it was: > CREATE TABLE AS 1 Changing the set of command tags that have

Re: Add primary keys to system catalogs

2021-01-22 Thread Tom Lane
"Joel Jacobson" writes: > I ran this query (on a patched database) to see if there are still any > catalog tables without primary keys: > ... > pg_depend > pg_shdepend Yeah, this is noted in the patch's own regression tests. > Wouldn't it be possible to add primary keys to these two as well?

Re: CTAS command tags

2021-01-22 Thread Andreas Karlsson
On 1/22/21 2:19 PM, Darafei "Komяpa" Praliaskouski wrote: Having row count right away is very useful in CTAS in analytical and GIS usage scenarios. I can see that, but would it not work if it was: CREATE TABLE AS 1 Disclaimer: I have not looked at the code so maybe there is some good reason

Re: Commitfest 2021-01 Now in Progress

2021-01-22 Thread Masahiko Sawada
Hi all, On Fri, Jan 15, 2021 at 9:53 PM Masahiko Sawada wrote: > > Today I have gone over all the "Waiting on Author" patches and marked a few entries as "Needs Review" in cases where new versions had been posted. The current stats (and difference from the last report) is: Needs review: 149

Re: plpgsql variable assignment not supporting distinct anymore

2021-01-22 Thread easteregg
> Probably the fix is not hard, but it is almost the same situation as the > UNION case. The result of your code is not deterministic > > If there are more different ti_resource_id then some values can be randomly > ignored - when hash agg is used. > > The safe fix should be > > _resource_id :=

Re: plpgsql variable assignment not supporting distinct anymore

2021-01-22 Thread easteregg
the code provided is just a little poc to get the error ( which i have not included with my first mail sorry. ) ERROR: syntax error at or near "DISTINCT" LINE 8: _test := DISTINCT a FROM ( VALUES ( (true), ( true ) ) )... the code in production looked like this: _resource_id

Re: LogwrtResult contended spinlock

2021-01-22 Thread Masahiko Sawada
On Fri, Jan 22, 2021 at 10:39 PM Alvaro Herrera wrote: > > On 2021-Jan-22, Masahiko Sawada wrote: > > > On Wed, Nov 25, 2020 at 12:02 AM Alvaro Herrera > > wrote: > > > > Yes, please move it forward. I'll post an update sometime before the > > > next CF. > > > > Anything update on this? > >

Re: Error on failed COMMIT

2021-01-22 Thread Masahiko Sawada
On Thu, Jan 7, 2021 at 11:29 PM Dave Cramer wrote: > > I could if someone wants to commit to reviewing it. > I've updated it a number of times but it seems nobody wants to review it. Since this has a long thread, how about summarizing what consensus we reached and what discussion we still need

Re: plpgsql variable assignment not supporting distinct anymore

2021-01-22 Thread Pavel Stehule
pá 22. 1. 2021 v 15:10 odesílatel napsal: > > Probably the fix is not hard, but it is almost the same situation as the > > UNION case. The result of your code is not deterministic > > > > If there are more different ti_resource_id then some values can be > randomly > > ignored - when hash agg is

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-22 Thread Alexey Kondratov
On 2021-01-22 00:26, Justin Pryzby wrote: On Thu, Jan 21, 2021 at 11:48:08PM +0300, Alexey Kondratov wrote: Attached is a new patch set of first two patches, that should resolve all the issues raised before (ACL, docs, tests) excepting TOAST. Double thanks for suggestion to add more tests with

Re: mkid reference

2021-01-22 Thread Julien Rouhaud
Le ven. 22 janv. 2021 à 20:33, Daniel Gustafsson a écrit : > > On 22 Jan 2021, at 12:56, Magnus Hagander wrote: > > > And maybe even more interestnig -- is there a point to this whole > > make_diff directory at all in these days of git? Or should we just > > remove it rather than try to fix it?

Re: plpgsql variable assignment not supporting distinct anymore

2021-01-22 Thread Pavel Stehule
pá 22. 1. 2021 v 14:41 odesílatel napsal: > the code provided is just a little poc to get the error ( which i have not > included with my first mail sorry. ) > >ERROR: syntax error at or near "DISTINCT" >LINE 8: _test := DISTINCT a FROM ( VALUES ( (true), ( true ) ) )... > > > the

Re: Some more hackery around cryptohashes (some fixes + SHA1)

2021-01-22 Thread Heikki Linnakangas
On 07/01/2021 05:41, Michael Paquier wrote: On Mon, Dec 14, 2020 at 12:48:15PM +0900, Michael Paquier wrote: This is a nice cleanup, so I have moved ahead and applied it. A rebased version of the SHA1 business is attached. Rebased version attached to address the conflicts caused by 55fe26a.

Re: LogwrtResult contended spinlock

2021-01-22 Thread Alvaro Herrera
On 2021-Jan-22, Masahiko Sawada wrote: > On Wed, Nov 25, 2020 at 12:02 AM Alvaro Herrera > wrote: > > Yes, please move it forward. I'll post an update sometime before the > > next CF. > > Anything update on this? I'll update this one early next week. Thanks! -- Álvaro Herrera

Re: libpq debug log

2021-01-22 Thread 'Alvaro Herrera'
Hello, just two quick comments on this, On 2021-Jan-22, tsunakawa.ta...@fujitsu.com wrote: > From: Alvaro Herrera > > That's true, but it'd require that we move PQtrace() to fe-misc.c, because > > pqTraceInit() uses definitions that are private to that file. > > Ah, that was the reason for

Re: pg_rewind copies

2021-01-22 Thread Heikki Linnakangas
On 16/12/2020 00:08, Cary Huang wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:not tested Hello The patch seems

Re: LogwrtResult contended spinlock

2021-01-22 Thread Masahiko Sawada
Hi Alvaro, On Wed, Nov 25, 2020 at 12:02 AM Alvaro Herrera wrote: > > On 2020-Nov-24, Anastasia Lubennikova wrote: > > > On 04.09.2020 20:13, Andres Freund wrote: > > > > Re general routine: On second thought, it might actually be worth having > > > it. Even just for LSNs - there's plenty places

Re: CTAS command tags

2021-01-22 Thread Komяpa
Having row count right away is very useful in CTAS in analytical and GIS usage scenarios. пт, 22 сту 2021, 16:14 карыстальнік Vik Fearing напісаў: > I was recently surprised by the following inconsistencies in returned > command tags for CTAS: > > > postgres=# create table a as select 123; >

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2021-01-22 Thread Masahiko Sawada
Hi Andy, On Mon, Dec 7, 2020 at 9:15 PM Andy Fan wrote: > > > > On Mon, Dec 7, 2020 at 4:16 PM Jesper Pedersen > wrote: >> >> Hi, >> >> On 12/5/20 10:38 PM, Andy Fan wrote: >> > Currently the UniqueKey is defined as a List of Expr, rather than >> > EquivalenceClasses. >> > A complete

CTAS command tags

2021-01-22 Thread Vik Fearing
I was recently surprised by the following inconsistencies in returned command tags for CTAS: postgres=# create table a as select 123; SELECT 1 postgres=# create table b as select 123 with data; SELECT 1 postgres=# create table c as select 123 with no data; CREATE TABLE AS Shouldn't we make

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-22 Thread Fujii Masao
On 2021/01/22 3:29, Fujii Masao wrote: On 2021/01/22 1:17, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 8:58 PM Fujii Masao wrote: My opinion is to check "!all", but if others prefer using such boolean flag, I'd withdraw my opinion. I'm really sorry, actually if (!all) is enough

Re: PoC Refactor AM analyse API

2021-01-22 Thread Heikki Linnakangas
On 30/12/2020 11:12, Denis Smirnov wrote: But why do you pass int natts and VacAttrStats **stats to acquire_sample_rows()? Is it of any use? It seems to break abstraction too. Yes, it is really a kluge that should be discussed. The main problem is that we don’t pass projection information to

Re: PoC/WIP: Extended statistics on expressions

2021-01-22 Thread Tomas Vondra
On 1/22/21 5:01 AM, Justin Pryzby wrote: On Fri, Jan 22, 2021 at 04:49:51AM +0100, Tomas Vondra wrote: | Statistics objects: | "public"."s2" (ndistinct, dependencies, mcv) ON FROM t Umm, for me that prints: "public"."s2" ON ((i + 1)), (((i + 1) + 0)) FROM t which I think is

Re: PoC/WIP: Extended statistics on expressions

2021-01-22 Thread Tomas Vondra
On 1/22/21 10:00 AM, Dean Rasheed wrote: On Fri, 22 Jan 2021 at 04:46, Justin Pryzby wrote: I think you'll maybe have to do something better - this seems a bit too weird: | postgres=# CREATE STATISTICS s2 ON (i+1) ,i FROM t; | postgres=# \d t | ... | "public"."s2" (ndistinct,

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-01-22 Thread Masahiro Ikeda
On 2021-01-22 14:50, Masahiko Sawada wrote: On Fri, Dec 25, 2020 at 6:46 PM Masahiro Ikeda wrote: Hi, I rebased the patch to the master branch. Thank you for working on this. I've read the latest patch. Here are comments: --- + if (track_wal_io_timing) + { +

Re: Is it useful to record whether plans are generic or custom?

2021-01-22 Thread Chengxi Sun
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: not tested Documentation:not tested Hi Atsushi, I just run a few test on your latest patch. It works

Re: doc review for v14

2021-01-22 Thread Masahiko Sawada
On Wed, Jan 6, 2021 at 10:37 AM Michael Paquier wrote: > > On Sun, Jan 03, 2021 at 09:05:09PM +0900, Michael Paquier wrote: > > So let's use this version and call it a day for this part. > > This has been done as of b49154b. It seems to me that all work has been done. Can we mark this patch

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-22 Thread vignesh C
On Fri, Jan 22, 2021 at 10:14 AM japin wrote: > > > On Fri, 22 Jan 2021 at 00:51, Bharath Rupireddy > wrote: > > On Thu, Jan 21, 2021 at 6:56 PM vignesh C wrote: > >> > >> Hi, > >> > >> Creating/altering subscription is successful when we specify a > >> publication which does not exist in the

Re: Parallel Inserts in CREATE TABLE AS

2021-01-22 Thread Bharath Rupireddy
On Fri, Jan 22, 2021 at 5:16 PM Tang, Haiying wrote: > > Hi Bharath, > > I'm trying to take some performance measurements on you patch v23. > But when I started, I found an issue about the tuples unbalance distribution among workers(99% tuples read by one worker) under specified case which lead

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-22 Thread vignesh C
On Fri, Jan 22, 2021 at 12:14 PM Bharath Rupireddy wrote: > > On Fri, Jan 22, 2021 at 10:14 AM japin wrote: > > > 2) Can't we know whether the publications exist on the publisher with > > > the existing (or modifying it a bit if required) query in > > > fetch_table_list(), so that we can avoid

Re: mkid reference

2021-01-22 Thread Daniel Gustafsson
> On 22 Jan 2021, at 12:56, Magnus Hagander wrote: > And maybe even more interestnig -- is there a point to this whole > make_diff directory at all in these days of git? Or should we just > remove it rather than try to fix it? There's also src/tools/make_mkid which use this mkid tool. +1 for

Re: Online checksums patch - once again

2021-01-22 Thread Heikki Linnakangas
On 22/01/2021 13:55, Heikki Linnakangas wrote: I read through the latest patch, v31-0001-Support-checksum-enable-disable-in-a-running-clu.patch. Some comments below: One more thing: In SetRelationNumChecks(), you should use SearchSysCacheCopy1() to get a modifiable copy of the tuple.

RE: About to add WAL write/fsync statistics to pg_stat_wal view

2021-01-22 Thread Masahiro Ikeda
On 2021-01-22 11:54, kuroda.hay...@fujitsu.com wrote: Dear Ikeda-san, This patch cannot be applied to the HEAD, but anyway I put a comment. ``` + /* +* Measure i/o timing to fsync WAL data. +* +* The wal receiver skip to collect it to avoid performance degradation

mkid reference

2021-01-22 Thread Magnus Hagander
Hi! in src/tools/make_diff/ there is a reference: "If I use mkid (from ftp.postgreSQL.org), I can do:" There is no such thing on our download site, and I can't find what it even was at one point. Was this part of some other package, since removed? And maybe even more interestnig -- is there a

Re: Online checksums patch - once again

2021-01-22 Thread Heikki Linnakangas
I read through the latest patch, v31-0001-Support-checksum-enable-disable-in-a-running-clu.patch. Some comments below: On 19/01/2021 14:32, Daniel Gustafsson wrote: + /* +* Hold interrupts for the duration of xlogging to avoid the state of data +* checksums changing

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-22 Thread Greg Nancarrow
On Fri, Jan 22, 2021 at 7:52 PM tsunakawa.ta...@fujitsu.com wrote: > > > (1) > -* (Note that we do allow CREATE TABLE AS, SELECT INTO, and CREATE > -* MATERIALIZED VIEW to use parallel plans, but as of now, only the > leader > -* backend writes into a completely new

RE: Parallel Inserts in CREATE TABLE AS

2021-01-22 Thread Tang, Haiying
Hi Bharath, I'm trying to take some performance measurements on you patch v23. But when I started, I found an issue about the tuples unbalance distribution among workers(99% tuples read by one worker) under specified case which lead the "parallel select" part makes no performance gain. Then I

Re: [PATCH] Identify LWLocks in tracepoints

2021-01-22 Thread Peter Eisentraut
On 2021-01-14 09:39, Craig Ringer wrote: On Thu, 14 Jan 2021 at 15:56, Peter Eisentraut > wrote: On 2020-12-19 06:00, Craig Ringer wrote: > Patch 1 fixes a bogus tracepoint where an lwlock__acquire event would be > fired from

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-22 Thread Bharath Rupireddy
On Fri, Jan 22, 2021 at 2:59 PM Bharath Rupireddy wrote: > > BTW, have we tried to check if this problem exists in back-branches? > > It seems to me the problem has been recently introduced by commit > > 69bd60672a. I am telling this by looking at code and have yet not > > performed any testing

plpgsql variable assignment not supporting distinct anymore

2021-01-22 Thread easteregg
hi, no noticed after the assignment with union ( https://www.postgresql.org/message-id/flat/20210105201257.f0d76aff%40mail.verfriemelt.org ), that the assignment with distinct is broken aswell. DO $$ DECLARE _test bool; BEGIN _test := DISTINCT a FROM ( VALUES ( (true), ( true

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-22 Thread Bharath Rupireddy
On Fri, Jan 22, 2021 at 2:00 PM Amit Kapila wrote: > Thanks for the patch. Few comments: > + > +# Test replication with multiple publications for subscription > + > > While checking the existing tests in 001_rep_changes.pl, I came across > the below test which has multiple publications, won't

Re: Additional Chapter for Tutorial - arch-dev.sgml

2021-01-22 Thread Heikki Linnakangas
On 21/01/2021 14:38, Jürgen Purtz wrote: This supervisor process is called postmaster and listens at a specified TCP/IP port for incoming connections. Whenever he detects a request for a connection, he spawns a new backend process. It sounds weird to refer to a process with "he". I left out

Re: adding wait_start column to pg_locks

2021-01-22 Thread Fujii Masao
On 2021/01/22 14:37, torikoshia wrote: On 2021-01-21 12:48, Fujii Masao wrote: Thanks for updating the patch! I think that this is really useful feature!! Thanks for reviewing! I have two minor comments. +  +   wait_start timestamptz The column name "wait_start" should be

Re: PoC/WIP: Extended statistics on expressions

2021-01-22 Thread Dean Rasheed
On Fri, 22 Jan 2021 at 04:46, Justin Pryzby wrote: > > I think you'll maybe have to do something better - this seems a bit too weird: > > | postgres=# CREATE STATISTICS s2 ON (i+1) ,i FROM t; > | postgres=# \d t > | ... > | "public"."s2" (ndistinct, dependencies, mcv) ON i FROM t > I guess

RE: Parallel INSERT (INTO ... SELECT ...)

2021-01-22 Thread tsunakawa.ta...@fujitsu.com
Hello Greg-san, Initially, some miner comments: (1) -* (Note that we do allow CREATE TABLE AS, SELECT INTO, and CREATE -* MATERIALIZED VIEW to use parallel plans, but as of now, only the leader -* backend writes into a completely new table. In the future, we can -

Re: Add primary keys to system catalogs

2021-01-22 Thread Joel Jacobson
Many thanks for excellent work! I've tested the patch successfully. I ran this query (on a patched database) to see if there are still any catalog tables without primary keys: SELECT table_name FROM information_schema.tables WHERE table_schema = 'pg_catalog' AND table_type = 'BASE TABLE'

Re: plpgsql variable assignment not supporting distinct anymore

2021-01-22 Thread Pavel Stehule
pá 22. 1. 2021 v 9:21 odesílatel napsal: > hi, > > no noticed after the assignment with union ( > https://www.postgresql.org/message-id/flat/20210105201257.f0d76aff%40mail.verfriemelt.org > ), that the assignment with distinct is broken aswell. > > > > DO $$ > DECLARE > _test bool; >

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-22 Thread Bharath Rupireddy
On Fri, Jan 22, 2021 at 2:00 PM Amit Kapila wrote: > > On Sat, Jan 16, 2021 at 6:08 PM Bharath Rupireddy > wrote: > > > > On Sat, Jan 16, 2021 at 12:21 PM Bharath Rupireddy > > wrote: > > > > In the test, can we have multiple publications for the subscription > > > > because that is how we

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-22 Thread Amit Kapila
On Sat, Jan 16, 2021 at 6:08 PM Bharath Rupireddy wrote: > > On Sat, Jan 16, 2021 at 12:21 PM Bharath Rupireddy > wrote: > > > In the test, can we have multiple publications for the subscription > > > because that is how we discovered that the originally proposed patch > > > was not correct?

Re: Why does creating logical replication subscriptions require superuser?

2021-01-22 Thread Andrey Borodin
[offlist] > 22 янв. 2021 г., в 13:16, Thomas Kellerer написал(а): > > Andrey Borodin schrieb am 22.01.2021 um 08:32: > >> Replication is running under superuser and e.g. one can add system catalog >> to subscription. >> Or exploit this fact other way. Having superuser you can just COPY FROM

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-22 Thread Greg Nancarrow
On Fri, Jan 22, 2021 at 6:21 PM Hou, Zhijie wrote: > Hi greg, > > Thanks for debugging this. > > May be I missed something. I am not sure about the case when rel->rd_index > was NULL. > Because, In function BuildIndexInfo, it seems does not have NULL-check for > index->rd_index. > Like the

Re: Why does creating logical replication subscriptions require superuser?

2021-01-22 Thread Thomas Kellerer
Andrey Borodin schrieb am 22.01.2021 um 08:32: > Replication is running under superuser and e.g. one can add system catalog to > subscription. > Or exploit this fact other way. Having superuser you can just COPY FROM > PROGRAM anything. It was my understanding that the replication process

Re: Preventing hangups in bgworker start/stop during DB shutdown

2021-01-22 Thread Craig Ringer
On Fri, 25 Dec 2020 at 06:07, Tom Lane wrote: > I wrote: > > Bharath Rupireddy writes: > >> 4) IIUC, in the patch we mark slot->terminate = true only for > >> BGW_NEVER_RESTART kind bg workers, what happens if a bg worker has > >> bgw_restart_time seconds and don't we hit the hanging issue(that

Re: Implementing Incremental View Maintenance

2021-01-22 Thread Yugo NAGATA
Hi, Attached is a revised patch (v22) rebased for the latest master head. Regards, Yugo Nagata -- Yugo NAGATA IVM_patches_v22.tar.gz Description: application/gzip