Re: Perform streaming logical transactions by background workers and parallel apply

2022-05-01 Thread Masahiko Sawada
On Fri, Apr 8, 2022 at 6:14 PM houzj.f...@fujitsu.com wrote: > > On Wednesday, April 6, 2022 1:20 PM Amit Kapila > wrote: > > > In this email, I would like to discuss allowing streaming logical > > transactions (large in-progress transactions) by background workers > > and parallel apply in gene

Re: bogus: logical replication rows/cols combinations

2022-05-01 Thread Amit Kapila
On Mon, May 2, 2022 at 11:01 AM Amit Kapila wrote: > > On Mon, May 2, 2022 at 3:27 AM Tomas Vondra > wrote: > > > > On 4/30/22 12:11, Amit Kapila wrote: > > > On Sat, Apr 30, 2022 at 3:01 PM Alvaro Herrera > > > wrote: > > >> > > >> My proposal is that if users want to define multiple publicati

Re: bogus: logical replication rows/cols combinations

2022-05-01 Thread Amit Kapila
On Mon, May 2, 2022 at 3:27 AM Tomas Vondra wrote: > > On 4/30/22 12:11, Amit Kapila wrote: > > On Sat, Apr 30, 2022 at 3:01 PM Alvaro Herrera > > wrote: > >> > >> My proposal is that if users want to define multiple publications, and > >> their definitions conflict in a way that would behave ri

Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set

2022-05-01 Thread Noah Misch
On Fri, Apr 01, 2022 at 10:16:48AM +0900, Michael Paquier wrote: > On Thu, Mar 31, 2022 at 09:49:50AM -0400, Tom Lane wrote: > > Well, let's go ahead with it and see what happens. If it's too > > much of a mess we can always revert. > > Okay, done after an extra round of self-review. commit 322b

Re: Add index scan progress to pg_stat_progress_vacuum

2022-05-01 Thread Masahiko Sawada
On Thu, Apr 14, 2022 at 10:32 AM Imseih (AWS), Sami wrote: > > Taking the above feedback, I modified the patches > and I believe this approach should be acceptable. > > For now, I also reduced the scope to only exposing > Indexes_total and indexes_completed in > pg_stat_progress_vacuum. I will cre

Re: Logical replication timeout problem

2022-05-01 Thread Masahiko Sawada
On Mon, May 2, 2022 at 11:32 AM Amit Kapila wrote: > > On Mon, May 2, 2022 at 7:33 AM Masahiko Sawada wrote: > > On Thu, Apr 28, 2022 at 7:01 PM houzj.f...@fujitsu.com > > wrote: > > > > > > Hi Sawada-san, Wang > > > > > > I was looking at the patch and noticed that we moved some logic from > >

Re: Logical replication timeout problem

2022-05-01 Thread Amit Kapila
On Mon, May 2, 2022 at 7:33 AM Masahiko Sawada wrote: > On Thu, Apr 28, 2022 at 7:01 PM houzj.f...@fujitsu.com > wrote: > > > > Hi Sawada-san, Wang > > > > I was looking at the patch and noticed that we moved some logic from > > update_replication_progress() to OutputPluginUpdateProgress() like >

Re: ERROR: type of parameter 1 (fruit2) does not match that when preparing the plan (fruit1)

2022-05-01 Thread David G. Johnston
Moving discussion to -hackers On Sun, May 1, 2022 at 12:46 PM Tom Lane wrote: > "David G. Johnston" writes: > > On Sun, May 1, 2022 at 10:08 AM Tom Lane wrote: > >> Maybe we could improve this situation by treating a "record" parameter > >> as polymorphic, though that might cause some odd inco

Re: Logical replication timeout problem

2022-05-01 Thread Masahiko Sawada
On Thu, Apr 28, 2022 at 7:01 PM houzj.f...@fujitsu.com wrote: > > On Wednesday, April 20, 2022 3:21 PM Masahiko Sawada > wrote: > > > > BTW the changes in > > REL_14_v1-0001-Fix-the-logical-replication-timeout-during-large-.patch, > > adding end_xact to LogicalDecodingContext, seems good to me a

Re: Handle infinite recursion in logical replication setup

2022-05-01 Thread Peter Smith
Thanks for updating the patches for all my prior feedback. For v12* I have only minor feedback for the docs. == V12-0001 no comments == V12-0002 1. Commit message In another thread using the terminology "multi-master" seems to be causing some problems. Maybe you should choose differen

Re: Tree-walker callbacks vs -Wdeprecated-non-prototype

2022-05-01 Thread Tom Lane
Thomas Munro writes: > As visible on seawasp (and noticed here in passing, while hacking on > the opaque pointer changes for bleeding edge LLVM), Clang 15 now warns > by default about our use of tree walkers functions with no function > prototype, because the next revision of C (C23?) will apparen

Re: Libpq single-row mode slowness

2022-05-01 Thread Daniele Varrazzo
On Sun, 1 May 2022 at 23:12, Tom Lane wrote: > The usual expectation is that you call PQconsumeInput to get rid of > a read-ready condition on the socket. If you don't have a poll() or > select() or the like in the loop, you might be wasting a lot of > pointless recvfrom calls. You definitely d

Tree-walker callbacks vs -Wdeprecated-non-prototype

2022-05-01 Thread Thomas Munro
Hi, As visible on seawasp (and noticed here in passing, while hacking on the opaque pointer changes for bleeding edge LLVM), Clang 15 now warns by default about our use of tree walkers functions with no function prototype, because the next revision of C (C23?) will apparently be harmonising with C

Re: Accessing git.postgresql.org fails

2022-05-01 Thread Tatsuo Ishii
> Tatsuo Ishii writes: >> This is ok: >> git clone ssh://g...@gitmaster.postgresql.org/postgresql.git > > That's the thing to use if you're a committer. > >> But this fails: >> $ git clone ssh://g...@git.postgresql.org/postgresql.git > > Per [1], the recommended git URL for non-committers is >

Re: bogus: logical replication rows/cols combinations

2022-05-01 Thread Tomas Vondra
On 4/30/22 12:11, Amit Kapila wrote: > On Sat, Apr 30, 2022 at 3:01 PM Alvaro Herrera > wrote: >> >> On 2022-Apr-30, Amit Kapila wrote: >> >>> On Sat, Apr 30, 2022 at 2:02 AM Tomas Vondra >>> wrote: >> That seems to deal with a circular replication, i.e. two logical replication links -

Re: bogus: logical replication rows/cols combinations

2022-05-01 Thread Tomas Vondra
On 4/29/22 07:05, Amit Kapila wrote: > On Thu, Apr 28, 2022 at 5:56 PM Peter Eisentraut > wrote: >> >> On 27.04.22 12:33, Amit Kapila wrote: >>> Currently, when the subscription has multiple publications, we combine >>> the objects, and actions of those publications. It happens for >>> 'publish

Re: Libpq single-row mode slowness

2022-05-01 Thread Tom Lane
Daniele Varrazzo writes: > The operations we perform, for every row, are PQconsumeInput, > PQisBusy, PQgetResult. Every PQconsumeInput results in a recvfrom() > syscall, of which the first one returns the whole recordset, the > following ones return EAGAIN. There are two extra cycles: one to get >

Re: bogus: logical replication rows/cols combinations

2022-05-01 Thread Tomas Vondra
On 4/30/22 11:28, Alvaro Herrera wrote: > On 2022-Apr-28, Tomas Vondra wrote: > >> Attached is a patch doing the same thing in tablesync. The overall idea >> is to generate copy statement with CASE expressions, applying filters to >> individual columns. For Alvaro's example, this generates some

Libpq single-row mode slowness

2022-05-01 Thread Daniele Varrazzo
Hello, a Psycopg 3 user has tested what boils down pretty much to a "generate_series(100K)" and has reported a 100x difference between fetching it normally and in single-row mode. I have repeated the test myself and I have found a 50x difference (the OP doesn't specify which platform is using, min

Re: Missing can't-assign-to-constant checks in plpgsql

2022-05-01 Thread Pavel Stehule
Hi >> Regardless of which way we handle that point, I'm inclined to >> change this only in HEAD. Probably people wouldn't thank us >> for making the back branches more strict. >> > > +1 > > I can implement these checks in plpgsql_check. So possible issues can be > detected and fixed on older ver

Re: testclient.exe installed under MSVC

2022-05-01 Thread Justin Pryzby
On Sun, May 01, 2022 at 10:23:18PM +0900, Michael Paquier wrote: > On Sun, May 01, 2022 at 01:07:06AM -0700, Noah Misch wrote: > > My annual audit for executables missing Windows icons turned up these: > > > > pginstall/bin/testclient.exe > > pginstall/bin/uri-regress.exe > > > >

Re: Accessing git.postgresql.org fails

2022-05-01 Thread Magnus Hagander
On Sun, May 1, 2022 at 4:52 PM Tom Lane wrote: > Tatsuo Ishii writes: > > This is ok: > > git clone ssh://g...@gitmaster.postgresql.org/postgresql.git > > That's the thing to use if you're a committer. > > > But this fails: > > $ git clone ssh://g...@git.postgresql.org/postgresql.git > > Per [1]

Re: Accessing git.postgresql.org fails

2022-05-01 Thread Tom Lane
Tatsuo Ishii writes: > This is ok: > git clone ssh://g...@gitmaster.postgresql.org/postgresql.git That's the thing to use if you're a committer. > But this fails: > $ git clone ssh://g...@git.postgresql.org/postgresql.git Per [1], the recommended git URL for non-committers is https://git.postg

Re: testclient.exe installed under MSVC

2022-05-01 Thread Michael Paquier
On Sun, May 01, 2022 at 01:07:06AM -0700, Noah Misch wrote: > My annual audit for executables missing Windows icons turned up these: > > pginstall/bin/testclient.exe > pginstall/bin/uri-regress.exe > > I was going to add the icons, but I felt the testclient.exe name is too > gener

Re: avoid multiple hard links to same WAL file after a crash

2022-05-01 Thread Michael Paquier
On Tue, Apr 12, 2022 at 09:27:42AM -0700, Nathan Bossart wrote: > On Tue, Apr 12, 2022 at 03:46:31PM +0900, Kyotaro Horiguchi wrote: >> At Mon, 11 Apr 2022 09:52:57 -0700, Nathan Bossart >> wrote in >>> I traced this back a while ago. I believe the link() was first added in >>> November 2000 as

testclient.exe installed under MSVC

2022-05-01 Thread Noah Misch
My annual audit for executables missing Windows icons turned up these: pginstall/bin/testclient.exe pginstall/bin/uri-regress.exe I was going to add the icons, but I felt the testclient.exe name is too generic-sounding to be installed. testclient originated in commit ebc8b7d. I

Accessing git.postgresql.org fails

2022-05-01 Thread Tatsuo Ishii
This is ok: git clone ssh://g...@gitmaster.postgresql.org/postgresql.git But this fails: $ git clone ssh://g...@git.postgresql.org/postgresql.git Cloning into 'postgresql'... Permission denied on repository for user ishii fatal: Could not r