Re: Non-superuser subscription owners

2023-04-23 Thread Amit Kapila
On Fri, Apr 21, 2023 at 6:21 PM Robert Haas wrote: > > On Fri, Apr 21, 2023 at 8:19 AM Amit Kapila wrote: > > LGTM. Let's see if Robert or others have any comments, otherwise, I'll > > push this early next week. > > LGTM too. > Pushed. -- With Regards, Amit Kapila.

Re: Add two missing tests in 035_standby_logical_decoding.pl

2023-04-23 Thread Drouvot, Bertrand
Hi, On 4/24/23 6:04 AM, vignesh C wrote: On Wed, 12 Apr 2023 at 21:45, Drouvot, Bertrand wrote: hi hackers, In the logical decoding on standby thread [1], Andres proposed 2 new tests (that I did not find the time to complete before the finish line): - Test that we can subscribe to the

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-04-23 Thread Masahiko Sawada
On Wed, Apr 19, 2023 at 4:02 PM John Naylor wrote: > > On Mon, Apr 17, 2023 at 8:49 PM Masahiko Sawada wrote: > > > > - With lazy expansion and single-value leaves, the root of a radix tree > > > can point to a single leaf. That might get rid of the need to track > > > TBMStatus, since setting

Re: Perform streaming logical transactions by background workers and parallel apply

2023-04-23 Thread Amit Kapila
On Mon, Apr 24, 2023 at 7:26 AM Masahiko Sawada wrote: > > While looking at the worker.c, I realized that we have the following > code in handle_streamed_transaction(): > > default: > Assert(false); > return false; / silence compiler warning / > > I think

Re: pg_stat_io not tracking smgrwriteback() is confusing

2023-04-23 Thread Amit Kapila
On Sun, Apr 23, 2023 at 12:55 AM Jonathan S. Katz wrote: > > On 4/19/23 1:23 PM, Andres Freund wrote: > > Hi, > > > > I noticed that the numbers in pg_stat_io dont't quite add up to what I > > expected in write heavy workloads. Particularly for checkpointer, the > > numbers > > for "write" in

Re: Add two missing tests in 035_standby_logical_decoding.pl

2023-04-23 Thread vignesh C
On Wed, 12 Apr 2023 at 21:45, Drouvot, Bertrand wrote: > > hi hackers, > > In the logical decoding on standby thread [1], Andres proposed 2 new tests > (that I did > not find the time to complete before the finish line): > > - Test that we can subscribe to the standby (with the publication

Re: Perform streaming logical transactions by background workers and parallel apply

2023-04-23 Thread Kyotaro Horiguchi
At Mon, 24 Apr 2023 08:59:07 +0530, Amit Kapila wrote in > > Sorry for posting multiple times in a row, but I'm a bit unceratin > > whether we should use FATAL or ERROR for this situation. The stream is > > not provided by user, and the session or process cannot continue. > > > > I think ERROR

Re: Support logical replication of DDLs

2023-04-23 Thread Amit Kapila
On Mon, Apr 24, 2023 at 8:18 AM shveta malik wrote: > > On Thu, Apr 20, 2023 at 3:40 PM Amit Kapila wrote: > > > > On Thu, Apr 20, 2023 at 9:11 AM shveta malik wrote: > > > > > > > > > Few comments for ddl_deparse.c in patch dated April17: > > > > > > > > 6) There are plenty of places where we

Re: Perform streaming logical transactions by background workers and parallel apply

2023-04-23 Thread Amit Kapila
On Mon, Apr 24, 2023 at 8:40 AM Kyotaro Horiguchi wrote: > > At Mon, 24 Apr 2023 11:50:37 +0900 (JST), Kyotaro Horiguchi > wrote in > > In my opinion, it is fine to replace the Assert with an ERROR. > > Sorry for posting multiple times in a row, but I'm a bit unceratin > whether we should use

Re: Autogenerate some wait events code and documentation

2023-04-23 Thread Michael Paquier
On Sat, Apr 22, 2023 at 03:36:05PM +0200, Drouvot, Bertrand wrote: > On 4/20/23 3:09 AM, Michael Paquier wrote: >> It is clear that the format of the file is: >> - category >> - C symbol in enums. >> - Format in the system views. >> - Description in the docs. >> Or perhaps it would be better to

Re: Perform streaming logical transactions by background workers and parallel apply

2023-04-23 Thread Kyotaro Horiguchi
At Mon, 24 Apr 2023 11:50:37 +0900 (JST), Kyotaro Horiguchi wrote in > In my opinion, it is fine to replace the Assert with an ERROR. Sorry for posting multiple times in a row, but I'm a bit unceratin whether we should use FATAL or ERROR for this situation. The stream is not provided by user,

Re: Logical replication failed with SSL SYSCALL error

2023-04-23 Thread shaurya jain
Hi Vignesh, That's really prompt and solves our problem. Thank you buddy. Please go through my inline comments:- On Thu, Apr 20, 2023 at 11:49 AM vignesh C wrote: > On Wed, 19 Apr 2023 at 17:26, shaurya jain <12345shau...@gmail.com> wrote: > > > > Hi Team, > > > > Could you please help me

Re: Perform streaming logical transactions by background workers and parallel apply

2023-04-23 Thread Kyotaro Horiguchi
At Mon, 24 Apr 2023 11:50:37 +0900 (JST), Kyotaro Horiguchi wrote in > I concur that returning false is problematic. > > For assertion builds, Assert typically provides more detailed > information than elog. However, in this case, it wouldn't matter much > since the worker would repeatedly

Re: Perform streaming logical transactions by background workers and parallel apply

2023-04-23 Thread Kyotaro Horiguchi
At Mon, 24 Apr 2023 10:55:44 +0900, Masahiko Sawada wrote in > While looking at the worker.c, I realized that we have the following > code in handle_streamed_transaction(): > > default: > Assert(false); > return false; / silence compiler warning / > > I

Re: Support logical replication of DDLs

2023-04-23 Thread shveta malik
On Thu, Apr 20, 2023 at 3:40 PM Amit Kapila wrote: > > On Thu, Apr 20, 2023 at 9:11 AM shveta malik wrote: > > > > > > Few comments for ddl_deparse.c in patch dated April17: > > > > > 6) There are plenty of places where we use 'append_not_present' > > without using 'append_null_object'. > > Do

Re: Fix documentation for max_wal_size and min_wal_size

2023-04-23 Thread Kyotaro Horiguchi
At Mon, 24 Apr 2023 10:57:56 +0900 (JST), Kyotaro Horiguchi wrote in > So, to clarify the situation, I would phrase it like this: > > The default size is 80MB. Note that if you have changed the WAL > segment size from the default of 16MB with the initdb option > --wal-segsize, the tool should

Re: Perform streaming logical transactions by background workers and parallel apply

2023-04-23 Thread Masahiko Sawada
On Mon, Jan 9, 2023 at 5:51 PM Amit Kapila wrote: > > On Sun, Jan 8, 2023 at 11:32 AM houzj.f...@fujitsu.com > wrote: > > > > On Sunday, January 8, 2023 11:59 AM houzj.f...@fujitsu.com > > wrote: > > > Attach the updated patch set. > > > > Sorry, the commit message of 0001 was accidentally

Re: duplicate function declaration in multirangetypes_selfuncs.c

2023-04-23 Thread Alexander Korotkov
On Sun, Apr 23, 2023 at 3:36 PM Daniel Gustafsson wrote: > > On 23 Apr 2023, at 13:59, Alexander Korotkov wrote: > > > > On Fri, Apr 21, 2023 at 2:21 PM Pavel Borisov > > wrote: > >>> On Fri, 21 Apr 2023 at 15:14, Daniel Gustafsson wrote: > >>> > On 21 Apr 2023, at 12:58, Anton Voloshin

Re: xmlserialize bug - extra empty row at the end

2023-04-23 Thread Tom Lane
Jim Jones writes: > If we agree to remove it, the change wouldn't be substantial :) I guess > we could just pchomp it in the end of the function, as suggested by Tom. > Attached a draft patch. I wouldn't actually *use* pchomp here, because that induces an unnecessary copy of the result string.

Re: buffer refcount leak in foreign batch insert code

2023-04-23 Thread Michael Paquier
On Fri, Apr 21, 2023 at 07:16:03PM +0900, Michael Paquier wrote: > On Fri, Apr 21, 2023 at 01:07:03PM +0300, Alexander Pyhalov wrote: >> We've found that in cases like the one attached, when we insert into foreign >> partition with batch_size set, buffer refcount leak is detected. >> >> The above

Re: run pgindent on a regular basis / scripted manner

2023-04-23 Thread Magnus Hagander
On Sat, Apr 22, 2023 at 9:59 PM Noah Misch wrote: > > (Given that another commentator is "absolutely against" a hook, this message > is mostly for readers considering this for other projects.) > > On Sat, Apr 22, 2023 at 03:23:59PM +0200, Magnus Hagander wrote: > > On Tue, Feb 7, 2023 at 5:43 AM

Re: xmlserialize bug - extra empty row at the end

2023-04-23 Thread Jim Jones
My inclination would be, if we're just calling to a long-standardized library routine, to just accept its output as is. If a program is saving the output to a text file, that would be the expected behaviour. If not, then we need to document that the output of our function is the output of

Re: run pgindent on a regular basis / scripted manner

2023-04-23 Thread Magnus Hagander
On Sat, Apr 22, 2023 at 4:12 PM Andrew Dunstan wrote: > > > On 2023-04-22 Sa 08:47, Magnus Hagander wrote: > > On Sat, Apr 22, 2023 at 1:42 PM Andrew Dunstan wrote: > > On 2023-04-22 Sa 04:50, Michael Paquier wrote: > > On Fri, Apr 21, 2023 at 09:58:17AM +0200, Jelte Fennema wrote: > > For 2 the

Re: Missing update of all_hasnulls in BRIN opclasses

2023-04-23 Thread Tomas Vondra
Hi, here's an updated version of the patch, including a backport version. I ended up making the code yet a bit closer to master by introducing add_values_to_range(). The current pre-14 code has the loop adding data to the BRIN tuple in two places, but with the "fixed" logic handling NULLs and the

Re: xmlserialize bug - extra empty row at the end

2023-04-23 Thread Isaac Morland
On Sun, 23 Apr 2023 at 12:28, Pavel Stehule wrote: > > > Dne ne 23. 4. 2023 18:03 uživatel Isaac Morland > napsal: > >> On Sun, 23 Apr 2023 at 10:52, Tom Lane wrote: >> >>> Isaac Morland writes: >>> >> >> >>> > I might go so >>> > far as to change the psql display routines to not leave a

Re: xmlserialize bug - extra empty row at the end

2023-04-23 Thread Pavel Stehule
Dne ne 23. 4. 2023 18:03 uživatel Isaac Morland napsal: > On Sun, 23 Apr 2023 at 10:52, Tom Lane wrote: > >> Isaac Morland writes: >> > > >> > I might go so >> > far as to change the psql display routines to not leave a blank line >> after >> > the content in the event it ends with a newline.

Re: xmlserialize bug - extra empty row at the end

2023-04-23 Thread Isaac Morland
On Sun, 23 Apr 2023 at 10:52, Tom Lane wrote: > Isaac Morland writes: > > > I might go so > > far as to change the psql display routines to not leave a blank line > after > > the content in the event it ends with a newline. > > psql has *no* business changing what it displays: if what came

Re: run pgindent on a regular basis / scripted manner

2023-04-23 Thread Jelte Fennema
On Sun, 23 Apr 2023 at 17:16, Tom Lane wrote: > I think we could go ahead and commit the perltidyrc and README changes > now. But the ensuing reformatting should happen as part of the mass > pgindent run, probably next month sometime. I think it's better to make the changes close together, not

Re: run pgindent on a regular basis / scripted manner

2023-04-23 Thread Tom Lane
Andrew Dunstan writes: > On 2023-04-22 Sa 15:58, Tom Lane wrote: >> Cool, let's use perltidy 20230309 then. > OK, so when would we do this? The change to 20230309 + valign changes is > fairly large: I think we could go ahead and commit the perltidyrc and README changes now. But the ensuing

Re: run pgindent on a regular basis / scripted manner

2023-04-23 Thread Andrew Dunstan
On 2023-04-22 Sa 16:15, Tom Lane wrote: Noah Misch writes: - skip if the break-glass "pgindent: no" appears in a commit message There are two things that bother me about putting this functionality into a server hook, beyond the possible speed issue: * The risk of failure. I don't have a

Re: run pgindent on a regular basis / scripted manner

2023-04-23 Thread Andrew Dunstan
On 2023-04-22 Sa 15:58, Tom Lane wrote: Andrew Dunstan writes: On 2023-04-22 Sa 11:37, Tom Lane wrote: * I see that there's now a 20230309 release, should we consider that instead? A test I just ran gave identical results to those from 20221112 Cool, let's use perltidy 20230309 then.

Re: xmlserialize bug - extra empty row at the end

2023-04-23 Thread Tom Lane
Isaac Morland writes: > That being said, this is a database column result and I agree it would look > more elegant if the blank line in the display were not there. Yeah, that would basically be the argument for editorializing on libxml2's result. It's a weak argument, but not entirely without

Re: xmlserialize bug - extra empty row at the end

2023-04-23 Thread Pavel Stehule
ne 23. 4. 2023 v 16:48 odesílatel Tom Lane napsal: > Jim Jones writes: > > On 23.04.23 07:31, Pavel Stehule wrote: > >> Looks so there is an extra empty row. > > > Good catch! It looks like it comes directly from libxml2. > > Is it really a bug? If libxml2 itself is putting in that newline, >

Re: xmlserialize bug - extra empty row at the end

2023-04-23 Thread Pavel Stehule
ne 23. 4. 2023 v 14:42 odesílatel Isaac Morland napsal: > On Sun, 23 Apr 2023 at 01:31, Pavel Stehule > wrote: > >> Hi >> >> maybe I found a bug in xmlserialize >> >> SELECT xmlserialize(DOCUMENT '42' >> AS varchar INDENT); >> >> (2023-04-23 07:27:53) postgres=# SELECT xmlserialize(DOCUMENT >>

Re: xmlserialize bug - extra empty row at the end

2023-04-23 Thread Tom Lane
Jim Jones writes: > On 23.04.23 07:31, Pavel Stehule wrote: >> Looks so there is an extra empty row. > Good catch! It looks like it comes directly from libxml2. Is it really a bug? If libxml2 itself is putting in that newline, I'm not sure we should take it on ourselves to strip it. > I'll do

Index merge join

2023-04-23 Thread Marek Mosiewicz
Hi, Best way to restrict query on many columns is to have index on these columns. BUT with faster and faster IOPS is it not already feasable to make merge join on indexes ? Then it could be possible to have just index on single columns for more rare queries. Index which have column and

Re: xmlserialize bug - extra empty row at the end

2023-04-23 Thread Isaac Morland
On Sun, 23 Apr 2023 at 01:31, Pavel Stehule wrote: > Hi > > maybe I found a bug in xmlserialize > > SELECT xmlserialize(DOCUMENT '42' > AS varchar INDENT); > > (2023-04-23 07:27:53) postgres=# SELECT xmlserialize(DOCUMENT > '42' AS varchar INDENT); > ┌─┐ > │

Re: duplicate function declaration in multirangetypes_selfuncs.c

2023-04-23 Thread Daniel Gustafsson
> On 23 Apr 2023, at 13:59, Alexander Korotkov wrote: > > On Fri, Apr 21, 2023 at 2:21 PM Pavel Borisov wrote: >>> On Fri, 21 Apr 2023 at 15:14, Daniel Gustafsson wrote: >>> On 21 Apr 2023, at 12:58, Anton Voloshin wrote: On 21/04/2023 13:45, Pavel Borisov wrote: > The

Re: xmlserialize bug - extra empty row at the end

2023-04-23 Thread Dmitry Dolgov
> On Sun, Apr 23, 2023 at 02:02:17PM +0200, Jim Jones wrote: > On 23.04.23 07:31, Pavel Stehule wrote: > > Hi > > > > maybe I found a bug in xmlserialize > > > > SELECT xmlserialize(DOCUMENT '42' > > AS varchar INDENT); > > > > (2023-04-23 07:27:53) postgres=# SELECT xmlserialize(DOCUMENT > > '42'

Re: xmlserialize bug - extra empty row at the end

2023-04-23 Thread Jim Jones
On 23.04.23 07:31, Pavel Stehule wrote: Hi maybe I found a bug in xmlserialize SELECT xmlserialize(DOCUMENT 'x="y">42' AS varchar INDENT); (2023-04-23 07:27:53) postgres=# SELECT xmlserialize(DOCUMENT '42' AS varchar INDENT); ┌─┐ │      xmlserialize       │

Re: duplicate function declaration in multirangetypes_selfuncs.c

2023-04-23 Thread Alexander Korotkov
On Fri, Apr 21, 2023 at 2:21 PM Pavel Borisov wrote: > On Fri, 21 Apr 2023 at 15:14, Daniel Gustafsson wrote: > > > > > On 21 Apr 2023, at 12:58, Anton Voloshin > > > wrote: > > > > > > On 21/04/2023 13:45, Pavel Borisov wrote: > > >> The patch is attached. Anyone to commit? > > > > > >

Re: Improve list manipulation in several places

2023-04-23 Thread tender wang
Richard Guo 于2023年4月21日周五 15:35写道: > There was discussion in [1] about improvements to list manipulation in > several places. But since the discussion is not related to the topic in > that thread, fork a new thread here and attach a patch to show my > thoughts. > > Some are just cosmetic

Re: Improving worst-case merge join performance with often-null foreign key

2023-04-23 Thread Richard Guo
On Sat, Apr 22, 2023 at 11:21 PM Tom Lane wrote: > Steinar Kaldager writes: > > First-time potential contributor here. We recently had an incident due > > to a sudden 1000x slowdown of a Postgres query (from ~10ms to ~10s) > > due to a join with a foreign key that was often null. We found that

Re: Improve list manipulation in several places

2023-04-23 Thread Richard Guo
On Fri, Apr 21, 2023 at 7:16 PM Ranier Vilela wrote: > +lcons_copy(void *datum, const List *list) > +lappend_copy(const List *list, void *datum) > list param pointer can be const here not? > Correct. Good point. V2 patch does that. Thanks Richard

Re: Improve list manipulation in several places

2023-04-23 Thread Richard Guo
On Sat, Apr 22, 2023 at 12:55 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 21.04.23 09:34, Richard Guo wrote: > > There was discussion in [1] about improvements to list manipulation in > > several places. But since the discussion is not related to the topic in > > that