Re: Add connection active, idle time to pg_stat_activity

2023-06-13 Thread Sergey Dudoladov
Hello hackers, Andrey and Nik, thank you for selecting this patch for review in Postgres Hacking 101: I've modified the patch based both on your email and the video. 1. Session statistics is now collected only for client backends. PG internal processes like wal sender seem to stop sending

Re: Let's make PostgreSQL multi-threaded

2023-06-13 Thread Konstantin Knizhnik
On 13.06.2023 11:46 AM, Kyotaro Horiguchi wrote: So we can assume that catalog  and relation cache should always fit in memory memory (otherwise significant rewriting of all Postgtres code working with relations will be needed). I'm not sure that is ture.. But likely to be? Sorry, looks

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-06-13 Thread Peter Smith
Here are some review comments for the patch v2-0001. == Commit message 1. General Better to use consistent terms in this message. Either "relations" or "tables" -- not a mixture of both. ~~~ 2. Before this commit, tablesync workers were capable of syncing only one relation. For each table,

Re: remap the .text segment into huge pages at run time

2023-06-13 Thread John Naylor
On Sat, Nov 5, 2022 at 3:27 PM Andres Freund wrote: >/* > * Make huge pages out of it. Requires at least linux 6.1. We could > * fall back to MADV_HUGEPAGE if it fails, but it doesn't do all that > * much in older kernels. > */ > #define MADV_COLLAPSE

Re: Views no longer in rangeTabls?

2023-06-13 Thread Amit Langote
On Wed, Jun 14, 2023 at 12:08 Amit Langote wrote: > On Tue, Jun 13, 2023 at 9:40 PM David Steele wrote: > > On 6/13/23 11:38, Amit Langote wrote: > > > On Tue, Jun 13, 2023 at 6:33 PM Alvaro Herrera < > alvhe...@alvh.no-ip.org> wrote: > > >> Note that you changed one comment from "permission

Re: make_ctags: use -I option to ignore pg_node_attr macro

2023-06-13 Thread Masahiko Sawada
On Wed, Jun 14, 2023 at 11:16 AM Tatsuo Ishii wrote: > > > Hi Sawada-san, > > > >> In my Mac environment where non-Exuberant ctags and emacs 28.2 are > >> installed, the generated etags file cannot be loaded by emacs due to > >> file format error. The generated TAGS file is: > >> > >> % head -10

Re: Replace (GUC_UNIT_MEMORY | GUC_UNIT_TIME) with GUC_UNIT in guc.c

2023-06-13 Thread Masahiko Sawada
On Wed, Jun 14, 2023 at 12:33 PM Japin Li wrote: > > > Hi, hackers > > We use (GUC_UNIT_MEMORY | GUC_UNIT_TIME) instead of GUC_UNIT even though we > already define it in guc.h. > > Maybe using GUC_UNIT is better? Here is a patch to fix it. Yeah, it seems more consistent with other places in

Re: Fix a typo in md.c

2023-06-13 Thread Masahiko Sawada
On Wed, Jun 14, 2023 at 9:19 AM Michael Paquier wrote: > > On Wed, Jun 14, 2023 at 06:33:18AM +0900, Masahiko Sawada wrote: > > I've attached the patch for $subject. In the following comment, > > LGTM. Thanks, pushed. Regards, -- Masahiko Sawada Amazon Web Services: https://aws.amazon.com

Re: Do we want a hashset type?

2023-06-13 Thread Tom Dunstan
On Mon, 12 Jun 2023 at 22:37, Tomas Vondra wrote: > Perhaps. So you're proposing to have this as a regular built-in type? > It's hard for me to judge how popular this feature would be, but I guess > people often use arrays while they actually want set semantics ... > Perspective from a

Re: [DOC] Update ALTER SUBSCRIPTION documentation v3

2023-06-13 Thread Amit Kapila
On Wed, Jun 14, 2023 at 9:25 AM Peter Smith wrote: > > On Wed, Jun 14, 2023 at 1:10 PM Amit Kapila wrote: > > > > On Wed, May 17, 2023 at 11:57 AM Peter Smith wrote: > > > > > > On Wed, May 17, 2023 at 2:53 PM Robert Sjöblom > > > wrote: > > > > > > > > Attached is the revised version. > > > >

Re: old_snapshot_threshold bottleneck on replica

2023-06-13 Thread Thomas Munro
On Tue, Feb 14, 2023 at 9:45 AM Andres Freund wrote: > On 2023-01-24 10:46:28 -0500, Robert Haas wrote: > > On Mon, Jan 23, 2023 at 9:40 AM Maxim Orlov wrote: > > > One of our customers stumble onto a significant performance degradation > > > while running multiple OLAP-like queries on a

Re: [DOC] Update ALTER SUBSCRIPTION documentation v3

2023-06-13 Thread Peter Smith
On Wed, Jun 14, 2023 at 1:10 PM Amit Kapila wrote: > > On Wed, May 17, 2023 at 11:57 AM Peter Smith wrote: > > > > On Wed, May 17, 2023 at 2:53 PM Robert Sjöblom > > wrote: > > > > > > Attached is the revised version. > > > > > > > v4 looks good to me. > > > > The latest version looks good to

RE: Non-superuser subscription owners

2023-06-13 Thread Zhijie Hou (Fujitsu)
On Wednesday, June 14, 2023 10:11 AM Amit Kapila wrote: > > On Tue, Jun 13, 2023 at 2:25 PM Amit Kapila wrote: > > > > On Fri, May 12, 2023 at 3:28 PM Zhijie Hou (Fujitsu) > > wrote: > > > > > > > > > I can reproduce this via gdb following similar steps in [1]. > > > > > > I think we need to

Replace (GUC_UNIT_MEMORY | GUC_UNIT_TIME) with GUC_UNIT in guc.c

2023-06-13 Thread Japin Li
Hi, hackers We use (GUC_UNIT_MEMORY | GUC_UNIT_TIME) instead of GUC_UNIT even though we already define it in guc.h. Maybe using GUC_UNIT is better? Here is a patch to fix it. diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c index a9033b7a54..5308896c87 100644 ---

RE: TRAP: FailedAssertion("prev_first_lsn < cur_txn->first_lsn", File: "reorderbuffer.c", Line: 927, PID: 568639)

2023-06-13 Thread Zhijie Hou (Fujitsu)
On Tuesday, June 13, 2023 12:19 PM Zhijie Hou (Fujitsu) wrote: > > On Tuesday, June 13, 2023 12:04 PM Amit Kapila > wrote: > > > > On Wed, Jun 7, 2023 at 6:02 PM Tomas Vondra > > wrote: > > > > > > > > > Well, I think the issue is pretty clear - we end up with an initial > > > snapshot that's

Re: Document that server will start even if it's unable to open some TCP/IP ports

2023-06-13 Thread Tom Lane
Kyotaro Horiguchi writes: > If I had to say, I would feel it rather surprising if server > successfully starts even when any explicitly-specified port can't be > opened (which is the current case). There is certainly an argument that such a condition indicates that something's very broken in our

Re: [DOC] Update ALTER SUBSCRIPTION documentation v3

2023-06-13 Thread Amit Kapila
On Wed, May 17, 2023 at 11:57 AM Peter Smith wrote: > > On Wed, May 17, 2023 at 2:53 PM Robert Sjöblom > wrote: > > > > Attached is the revised version. > > > > v4 looks good to me. > The latest version looks good to me as well. I think we should backpatch this change as this is a user-facing

Re: Views no longer in rangeTabls?

2023-06-13 Thread Amit Langote
On Tue, Jun 13, 2023 at 9:40 PM David Steele wrote: > On 6/13/23 11:38, Amit Langote wrote: > > On Tue, Jun 13, 2023 at 6:33 PM Alvaro Herrera > > wrote: > >> Note that you changed one comment from "permission checks" to > >> "permission hecks". > > > > Oops, thanks for pointing that out. > > >

Re: Document that server will start even if it's unable to open some TCP/IP ports

2023-06-13 Thread Kyotaro Horiguchi
At Tue, 13 Jun 2023 14:38:14 -0700, Nathan Bossart wrote in > On Tue, Jun 13, 2023 at 04:28:31PM -0400, Tom Lane wrote: > > Nathan Bossart writes: > >> Before we spend too much time trying to document the current behavior, I > >> think we should see if we can change it to something less

Re: [PATCH] Add loongarch native checksum implementation.

2023-06-13 Thread YANG Xudong
Attached a new patch with fixes based on the comment below. On 2023/6/13 18:26, John Naylor wrote: On Thu, Jun 8, 2023 at 12:24 PM YANG Xudong > wrote: > > This patch tries to add loongarch native crc32 check with crcc.* > instructions to postgresql. > >

Re: Add wait event for log emission?

2023-06-13 Thread Kyotaro Horiguchi
At Tue, 13 Jun 2023 17:18:58 -0700, Andres Freund wrote in > Hi, > > On 2023-06-13 20:55:19 +0200, Magnus Hagander wrote: > > On Tue, Jun 13, 2023 at 6:59 PM Andres Freund wrote: > > > I think we should add a few wait events for log emission. I think it'd be > > > good > > > to have one wait

Re: make_ctags: use -I option to ignore pg_node_attr macro

2023-06-13 Thread Tatsuo Ishii
> Hi Sawada-san, > >> In my Mac environment where non-Exuberant ctags and emacs 28.2 are >> installed, the generated etags file cannot be loaded by emacs due to >> file format error. The generated TAGS file is: >> >> % head -10 TAGS >> >> ) / >>

Re: Non-superuser subscription owners

2023-06-13 Thread Amit Kapila
On Tue, Jun 13, 2023 at 2:25 PM Amit Kapila wrote: > > On Fri, May 12, 2023 at 3:28 PM Zhijie Hou (Fujitsu) > wrote: > > > > > > I can reproduce this via gdb following similar steps in [1]. > > > > I think we need to move this call into a transaction as well and here is an > > attempt > > to do

Re: Avoid unncessary always true test (src/backend/storage/buffer/bufmgr.c)

2023-06-13 Thread Kyotaro Horiguchi
At Wed, 14 Jun 2023 10:01:59 +0900 (JST), Kyotaro Horiguchi wrote in > If we decide to remove it, the preceding blank line seems to be a > separator from the previous function call. So, we might want to Mmm. that is a bit short. Anyway I meant that the blank will become useless after removing

Re: [PATCH] Using named captures in Catalog::ParseHeader()

2023-06-13 Thread Michael Paquier
On Thu, Jun 01, 2023 at 01:12:22PM +0100, Dagfinn Ilmari Mannsåker wrote: > While I was rewriting the regexes I noticed that they were inconsistent > about whether they accepted whitespace in the parameter lists, so I took > the liberty to make them consistently allow whitespace after the opening

Re: Avoid unncessary always true test (src/backend/storage/buffer/bufmgr.c)

2023-06-13 Thread Kyotaro Horiguchi
At Tue, 13 Jun 2023 08:21:20 -0300, Ranier Vilela wrote in > Em ter., 13 de jun. de 2023 às 02:51, Gurjeet Singh > escreveu: > > > Please see attached v2 of the patch; it includes both occurrences of > > the spurious checks identified in this thread. > > > +1 > LGTM. >

Re: Inconsistent results with libc sorting on Windows

2023-06-13 Thread Thomas Munro
Trying to follow along here... you're doing the moral equivalent of strxfrm(), so sort keys have the transitive property but direct string comparisons don't? Or is this because LCIDs reach a different algorithm somehow (or otherwise why do you need to use LCIDs for this, when there is a non-LCID

Re: DSA failed to allocate memory

2023-06-13 Thread Thomas Munro
On Mon, Feb 20, 2023 at 5:52 PM Thomas Munro wrote: > I'm wondering about this bit in rebin_segment(): > > + if (segment_map->header == NULL) > + return; > > Why would we be rebinning an uninitialised/unused segment? Answering my own question: because destroy_superblock() can

Re: Fix a typo in md.c

2023-06-13 Thread Michael Paquier
On Wed, Jun 14, 2023 at 06:33:18AM +0900, Masahiko Sawada wrote: > I've attached the patch for $subject. In the following comment, LGTM. -- Michael signature.asc Description: PGP signature

Re: Add wait event for log emission?

2023-06-13 Thread Andres Freund
Hi, On 2023-06-13 20:55:19 +0200, Magnus Hagander wrote: > On Tue, Jun 13, 2023 at 6:59 PM Andres Freund wrote: > > > > Hi, > > > > I just helped somebody debug a postgres performance problem that turned out > > to > > be not actually be postgres' fault. It turned out to be because postgres' >

Re: add non-option reordering to in-tree getopt_long

2023-06-13 Thread Noah Misch
On Wed, Jun 14, 2023 at 09:03:22AM +0900, Michael Paquier wrote: > On Wed, Jun 14, 2023 at 08:52:27AM +0900, Michael Paquier wrote: > > On Tue, Jun 13, 2023 at 03:36:57PM -0700, Nathan Bossart wrote: > >> Windows seems to allow rearranging argv, based upon cfbot's results. I do > >> not know

Re: Improve logging when using Huge Pages

2023-06-13 Thread Michael Paquier
On Tue, Jun 13, 2023 at 02:50:30PM +0900, Michael Paquier wrote: > + Assert(strcmp("unknown", GetConfigOption("huge_pages_status", false, > false)) != 0); > > Not sure that there is anything to gain with this assertion in > CreateSharedMemoryAndSemaphores(), because this is pretty much

Re: add non-option reordering to in-tree getopt_long

2023-06-13 Thread Michael Paquier
On Wed, Jun 14, 2023 at 08:52:27AM +0900, Michael Paquier wrote: > On Tue, Jun 13, 2023 at 03:36:57PM -0700, Nathan Bossart wrote: >> Windows seems to allow rearranging argv, based upon cfbot's results. I do >> not know about AIX. In any case, C99 explicitly mentions that argv should >> be

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-06-13 Thread Nathan Bossart
On Wed, Jun 14, 2023 at 08:16:15AM +0900, Michael Paquier wrote: > While on it, this buzzes me: > static bool > -vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params) > +vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool > skip_privs) > > VacuumParams has been

Re: add non-option reordering to in-tree getopt_long

2023-06-13 Thread Michael Paquier
On Tue, Jun 13, 2023 at 03:36:57PM -0700, Nathan Bossart wrote: > Windows seems to allow rearranging argv, based upon cfbot's results. I do > not know about AIX. In any case, C99 explicitly mentions that argv should > be modifiable. Few people have AIX machines around these days, but looking

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-06-13 Thread Michael Paquier
On Tue, Jun 13, 2023 at 02:12:46PM -0700, Nathan Bossart wrote: > I've been reviewing ff9618e lately, and I'm wondering whether it has the > same problem that 19de0ab solved. Specifically, ff9618e introduces > has_partition_ancestor_privs(), which is used to check whether a user has > MAINTAIN on

Re: add non-option reordering to in-tree getopt_long

2023-06-13 Thread Nathan Bossart
On Tue, Jun 13, 2023 at 04:02:01PM +0900, Kyotaro Horiguchi wrote: > Hmm. from the initial mail, I got the impression that AIX and Windows > allow this, so I thought that we can do that for them. While there > could be other platforms that allow it, perhaps we don't need to go as > far as

Re: ERROR: wrong varnullingrels (b 3) (expected (b)) for Var 2/1

2023-06-13 Thread Tom Lane
I wrote: > Richard Guo writes: >> Oh, wait ... It occurred to me that we may have this same issue with >> Memoize cache keys. > Good catch --- I'll take a closer look tomorrow. Pushed after a little more fiddling with the comments. regards, tom lane

Re: Support logical replication of DDLs

2023-06-13 Thread Michael Paquier
On Tue, Jun 13, 2023 at 06:49:42PM +0530, Amit Kapila wrote: > We have to choose one of the approaches between 0001 and 0008. I feel > we don't need an intermediate ObjTree representation as that adds > overhead and an additional layer which is not required. As mentioned > in my previous email I

Re: Document that server will start even if it's unable to open some TCP/IP ports

2023-06-13 Thread Nathan Bossart
On Tue, Jun 13, 2023 at 04:28:31PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> Before we spend too much time trying to document the current behavior, I >> think we should see if we can change it to something less surprising (i.e., >> failing to start if the server fails for any address).

Fix a typo in md.c

2023-06-13 Thread Masahiko Sawada
Hi, I've attached the patch for $subject. In the following comment, /* * If available and useful, use posix_fallocate() (via FileAllocate()) * to extend the relation. That's often more efficient than using * write(), as it commonly won't cause the kernel to

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-06-13 Thread Nathan Bossart
I've been reviewing ff9618e lately, and I'm wondering whether it has the same problem that 19de0ab solved. Specifically, ff9618e introduces has_partition_ancestor_privs(), which is used to check whether a user has MAINTAIN on any partition ancestors. This involves syscache lookups, and presently

Re: Meson build updates

2023-06-13 Thread Tristan Partin
Forgot that I had gotten a review from a Meson maintainer. The last two patches in this set are new. One is just a simple spelling correction. -- Tristan Partin Neon (https://neon.tech) From c20ece79c625c8ca33fee26791ab40c311a5c97e Mon Sep 17 00:00:00 2001 From: Tristan Partin Date: Tue, 16 May

Re: Document that server will start even if it's unable to open some TCP/IP ports

2023-06-13 Thread Tom Lane
Nathan Bossart writes: > Before we spend too much time trying to document the current behavior, I > think we should see if we can change it to something less surprising (i.e., > failing to start if the server fails for any address). The original > objections around kernel support for IPv6 might

Re: Do we want a hashset type?

2023-06-13 Thread Joel Jacobson
On Tue, Jun 13, 2023, at 20:50, Joel Jacobson wrote: > hashset is now using hash_bytes_uint32() from hashfn.h I spotted a problem in the ordering logic of the comparison functions. The issue was with handling hashsets containing empty positions, causing non-lexicographic ordering. The updated

Re: Meson build updates

2023-06-13 Thread Tristan Partin
Wow. I didn't attach them... -- Tristan Partin Neon (https://neon.tech) From c20ece79c625c8ca33fee26791ab40c311a5c97e Mon Sep 17 00:00:00 2001 From: Tristan Partin Date: Tue, 16 May 2023 07:55:03 -0500 Subject: [PATCH v4 01/14] Remove triple-quoted strings Triple-quoted strings are for

Re: Meson build updates

2023-06-13 Thread Tristan Partin
On Mon Jun 12, 2023 at 4:43 PM CDT, Tristan Partin wrote: > On Mon Jun 12, 2023 at 4:24 PM CDT, Peter Eisentraut wrote: > > On 12.06.23 20:48, Tristan Partin wrote: > > > Attached you will find a v3 with the offending commits removed. I did > > > leave the overrides in since you didn't mention it

Re: Meson build updates

2023-06-13 Thread Tristan Partin
On Fri Jun 9, 2023 at 1:36 PM CDT, Andres Freund wrote: > On 2023-06-09 13:15:27 -0500, Tristan Partin wrote: > > On Fri Jun 9, 2023 at 11:41 AM CDT, Andres Freund wrote: > > > > From 189d3ac3d5593ce3e475813e4830a29bb4e96f70 Mon Sep 17 00:00:00 2001 > > > > From: Tristan Partin > > > > Date: Wed,

Re: Document that server will start even if it's unable to open some TCP/IP ports

2023-06-13 Thread Nathan Bossart
On Mon, Jun 12, 2023 at 11:57:45PM -0700, Gurjeet Singh wrote: > Perhaps we can prefix that statement with "If listen_addresses is not > empty", like so: Before we spend too much time trying to document the current behavior, I think we should see if we can change it to something less surprising

Re: Add wait event for log emission?

2023-06-13 Thread Magnus Hagander
On Tue, Jun 13, 2023 at 6:59 PM Andres Freund wrote: > > Hi, > > I just helped somebody debug a postgres performance problem that turned out to > be not actually be postgres' fault. It turned out to be because postgres' > stdout/stderr were piped to a program, and that program was slow. Whenever

Re: Do we want a hashset type?

2023-06-13 Thread Joel Jacobson
On Mon, Jun 12, 2023, at 22:36, Joel Jacobson wrote: > On Mon, Jun 12, 2023, at 21:58, Tomas Vondra wrote: >> My suggestion is to be lazy, just use the lookup3 we have in hashfn.c >> (through hash_bytes or something), and at the same time make it possible >> to switch to a different function in

Re: [PATCH] Slight improvement of worker_spi.c example

2023-06-13 Thread Nathan Bossart
On Tue, Jun 13, 2023 at 07:58:02PM +0800, Julien Rouhaud wrote: > That being said, I still don't understand why you focus on this tiny and not > really important detail while the module itself is actually broken (for > dynamic > bgworker without s_p_l) and also has some broken behaviors with

Re: Add wait event for log emission?

2023-06-13 Thread Nathan Bossart
On Tue, Jun 13, 2023 at 09:58:54AM -0700, Andres Freund wrote: > I think we should add a few wait events for log emission. I think it'd be good > to have one wait event for each log destination. +1 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Use COPY for populating all pgbench tables

2023-06-13 Thread Tristan Partin
On Thu Jun 8, 2023 at 11:38 AM CDT, Tristan Partin wrote: > On Thu Jun 8, 2023 at 12:33 AM CDT, David Rowley wrote: > > On Thu, 8 Jun 2023 at 07:16, Tristan Partin wrote: > > > > > > master: > > > > > > 5000 of 5000 tuples (100%) done (elapsed 260.93 s, remaining 0.00 > > > s)) > > >

Re: logical decoding and replication of sequences, take 2

2023-06-13 Thread Tomas Vondra
On 5/18/23 16:23, Ashutosh Bapat wrote: > Hi, > Sorry for jumping late in this thread. > > I started experimenting with the functionality. Maybe something that > was already discussed earlier. Given that the thread is being > discussed for so long and has gone several changes, revalidating the >

Add wait event for log emission?

2023-06-13 Thread Andres Freund
Hi, I just helped somebody debug a postgres performance problem that turned out to be not actually be postgres' fault. It turned out to be because postgres' stdout/stderr were piped to a program, and that program was slow. Whenever the pipe buffer filled up, postgres stopped making progress.

Support TZ format code in to_timestamp()

2023-06-13 Thread Tom Lane
It's annoyed me for some time that to_timestamp() doesn't implement the TZ format code that to_char() has. I finally got motivated to do something about that after the complaint at [1] that jsonpath's datetime() method can't read typical JSON.stringify() output like "2023-05-22T03:09:37.825Z".

Re: RFC: Logging plan of the running query

2023-06-13 Thread James Coleman
On Tue, Jun 13, 2023 at 11:22 AM torikoshia wrote: > > On 2023-06-13 00:52, James Coleman wrote: > >> > >> > I've attached v27. The important change here in 0001 is that it > >> > guarantees the interrupt handler is re-entrant, since that was a bug > >> > exposed by my testing. I've also included

Re: Use COPY for populating all pgbench tables

2023-06-13 Thread Tristan Partin
I think I am partial to number 2. Removing a context switch always leads to more productivity. -- Tristan Partin Neon (https://neon.tech)

Re: Allow pg_archivecleanup to remove backup history files

2023-06-13 Thread torikoshia
On 2023-06-12 16:33, Michael Paquier wrote: On Fri, Jun 09, 2023 at 12:32:15AM +0900, Fujii Masao wrote: Thanks for reviewing! printf(_(" -n, --dry-run dry run, show the names of the files that would be removed\n")); + printf(_(" -b, --clean-backup-history clean up files

Re: RFC: Logging plan of the running query

2023-06-13 Thread torikoshia
On 2023-06-13 00:52, James Coleman wrote: > I've attached v27. The important change here in 0001 is that it > guarantees the interrupt handler is re-entrant, since that was a bug > exposed by my testing. I've also included 0002 which is only meant for > testing (it attempts to log in the plan

Re: [PATCH] Using named captures in Catalog::ParseHeader()

2023-06-13 Thread Dagfinn Ilmari Mannsåker
Dagfinn Ilmari Mannsåker writes: > However, now that we require Perl 5.14, we can use the named capture > feature (introduced in Perl 5.10) to make that a lot clearer, as in the > attached patch. Added to the open commitfest: https://commitfest.postgresql.org/43/4361/ - ilmari

Re: Support logical replication of DDLs

2023-06-13 Thread Amit Kapila
On Tue, Jun 13, 2023 at 1:21 PM Michael Paquier wrote: > > On Mon, Jun 12, 2023 at 01:47:02AM +, Wei Wang (Fujitsu) wrote: > > # load test cases from the regression tests > > -my @regress_tests = split /\s+/, $ENV{REGRESS}; > > +#my @regress_tests = split /\s+/, $ENV{REGRESS}; > > +my

OK to build LLVM (*.bc) with CLANG but rest of postgresql with CC (other compiler)?

2023-06-13 Thread Palle Girgensohn
Hi! TL;DR: CLANG is used to compile *.bc files during postgresql build. Is it OK to have a different compiler for the rest of the build? gcc, or even another version of clang? -- Slightly longer version: I'm packaging postgresql for FreeBSD and as you probably know, in that OS clang is the

Re: Views no longer in rangeTabls?

2023-06-13 Thread David Steele
On 6/13/23 11:38, Amit Langote wrote: On Tue, Jun 13, 2023 at 6:33 PM Alvaro Herrera wrote: Note that you changed one comment from "permission checks" to "permission hecks". Oops, thanks for pointing that out. Fixed in the attached. I have done another (more careful) review of the

Re: [PATCH] Slight improvement of worker_spi.c example

2023-06-13 Thread Julien Rouhaud
On Tue, Jun 13, 2023 at 12:34:09PM +0300, Aleksander Alekseev wrote: > > > I agree that the current code > > could lead folks to think that PushActiveSnapshot must go after > > SPI_connect, but wouldn't the reverse ordering just give folks the opposite > > impression? > > This is the exact reason

RE: extended statistics n-distinct on multiple columns not used when join two tables

2023-06-13 Thread James Pang (chaolpan)
Thanks for your information, yes, with multiple columns equal join and correlation , looks like extended statistics could help reduce “significantly rows estimation”. Hopefully it’s in future version. James From: Pavel Stehule Sent: Tuesday, June 13, 2023 7:29 PM To: David Rowley Cc:

Re: extended statistics n-distinct on multiple columns not used when join two tables

2023-06-13 Thread Pavel Stehule
út 13. 6. 2023 v 13:26 odesílatel David Rowley napsal: > (moving to -hackers) > > On Tue, 13 Jun 2023 at 21:30, Pavel Stehule > wrote: > > út 13. 6. 2023 v 11:21 odesílatel James Pang (chaolpan) < > chaol...@cisco.com> napsal: > >> When join two table on multiple columns equaljoin, rows

Re: extended statistics n-distinct on multiple columns not used when join two tables

2023-06-13 Thread David Rowley
(moving to -hackers) On Tue, 13 Jun 2023 at 21:30, Pavel Stehule wrote: > út 13. 6. 2023 v 11:21 odesílatel James Pang (chaolpan) > napsal: >> When join two table on multiple columns equaljoin, rows estimation >> always use selectivity = multiplied by distinct multiple individual

Re: Avoid unncessary always true test (src/backend/storage/buffer/bufmgr.c)

2023-06-13 Thread Ranier Vilela
Em ter., 13 de jun. de 2023 às 02:51, Gurjeet Singh escreveu: > Please see attached v2 of the patch; it includes both occurrences of > the spurious checks identified in this thread. > +1 LGTM. regards, Ranier Vilela

Re: [BUG] pg_dump does not properly deal with BEGIN ATOMIC function

2023-06-13 Thread Morris de Oryx
Thanks or the confirmation, and here's hoping no ICBMs!

Re: Making empty Bitmapsets always be NULL

2023-06-13 Thread David Rowley
On Tue, 13 Jun 2023 at 00:32, Yuya Watari wrote: > In March, I reported that David's patch caused a degradation in > planning performance. I have investigated this issue further and found > some bugs in the patch. Due to these bugs, Bitmapset operations in the > original patch computed incorrect

Re: [PATCH] Using named captures in Catalog::ParseHeader()

2023-06-13 Thread John Naylor
On Thu, Jun 1, 2023 at 7:12 PM Dagfinn Ilmari Mannsåker wrote: > > Hi Hackers, > > Peter's patch set for autogenerating syscache info > (https://postgr.es/m/75ae5875-3abc-dafc-8aec-73247ed41cde%40eisentraut.org ) > touched on one of my least favourite parts of Catalog.pm: the >

Re: [BUG] pg_dump does not properly deal with BEGIN ATOMIC function

2023-06-13 Thread Alvaro Herrera
On 2023-Jun-13, Morris de Oryx wrote: > Quick follow-up: I've heard back from AWS regarding applying Tom Lane's > patch. Nope. RDS releases numbered versions, nothing else. Sounds like a reasonable policy to me. > As Postgres is now at 15.8/15.3 in the wild and on 15.7/15.3 on RDS, > I'm

Re: [PATCH] Add loongarch native checksum implementation.

2023-06-13 Thread John Naylor
On Thu, Jun 8, 2023 at 12:24 PM YANG Xudong wrote: > > This patch tries to add loongarch native crc32 check with crcc.* > instructions to postgresql. > > The patch is tested on my Loongson 3A5000 machine with Loong Arch Linux > and GCC 13.1.0 / clang 16.0.0 with > > - default ./configure > -

Re: [BUG] pg_dump does not properly deal with BEGIN ATOMIC function

2023-06-13 Thread Morris de Oryx
Quick follow-up: I've heard back from AWS regarding applying Tom Lane's patch. Nope. RDS releases numbered versions, nothing else. As Postgres is now at 15.8/15.3 in the wild and on 15.7/15.3 on RDS, I'm guessing that the patch won't be available until 14.9/15.4. Am I right in thinking that this

RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-06-13 Thread Hayato Kuroda (Fujitsu)
Dear Melih, Thank you for making the patch! I'm also interested in the patchset. Here are the comments for 0001. Some codes are not suit for our coding conventions, but followings do not contain them because patches seems in the early statge. Moreover, 0003 needs rebase. 01. general Why do

Re: Let's make PostgreSQL multi-threaded

2023-06-13 Thread Andreas Karlsson
On 6/13/23 10:20, Konstantin Knizhnik wrote: The fact that it is flushed out upon reconnection can not help much: what if backends are not going to disconnect? This is why many connection pools have a maximum connection lifetime which can be configured. So in practice flushing all caches on

Re: Views no longer in rangeTabls?

2023-06-13 Thread Amit Langote
On Tue, Jun 13, 2023 at 6:33 PM Alvaro Herrera wrote: > Note that you changed one comment from "permission checks" to > "permission hecks". Oops, thanks for pointing that out. Fixed in the attached. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com

Re: [PATCH] Slight improvement of worker_spi.c example

2023-06-13 Thread Aleksander Alekseev
Hi, > On Sat, Jun 03, 2023 at 06:35:00PM -0400, Michael Paquier wrote: > > It does not have to be complicated, but I definitely agree that we'd > > better spend some efforts in improving it as a whole especially > > knowing that this is mentioned on the docs as an example that one > > could rely

Re: Views no longer in rangeTabls?

2023-06-13 Thread Alvaro Herrera
Note that you changed one comment from "permission checks" to "permission hecks". -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "No nos atrevemos a muchas cosas porque son difíciles, pero son difíciles porque no nos atrevemos a hacerlas" (Séneca)

OK to build LLVM (*.bc) with CLANG but rest of postgresql with CC (other compiler)?

2023-06-13 Thread Palle Girgensohn
Hi! TL;DR: CLANG is used to compile *.bc files during postgresql build. Is it OK to have a different compiler for the rest of the build? gcc, or even another version of clang? -- Slightly longer version: I'm packaging postgresql for FreeBSD and as you probably know, in that OS clang is the

Re: Non-superuser subscription owners

2023-06-13 Thread Amit Kapila
On Tue, Jun 13, 2023 at 2:25 PM Amit Kapila wrote: > > On Fri, May 12, 2023 at 3:28 PM Zhijie Hou (Fujitsu) > wrote: > > > > > > I can reproduce this via gdb following similar steps in [1]. > > > > I think we need to move this call into a transaction as well and here is an > > attempt > > to do

Re: Views no longer in rangeTabls?

2023-06-13 Thread David Steele
On 6/13/23 10:27, Amit Langote wrote: On Tue, Jun 13, 2023 at 4:44 PM David Steele wrote: I decided to go with the following because I think it is easier to read: /* We only care about tables/views and can ignore subqueries, etc. */ if (!(rte->rtekind == RTE_RELATION || (rte->rtekind

Re: Non-superuser subscription owners

2023-06-13 Thread Amit Kapila
On Fri, May 12, 2023 at 3:28 PM Zhijie Hou (Fujitsu) wrote: > > > I can reproduce this via gdb following similar steps in [1]. > > I think we need to move this call into a transaction as well and here is an > attempt > to do that. > I am able to reproduce this issue following the steps

Re: Let's make PostgreSQL multi-threaded

2023-06-13 Thread Kyotaro Horiguchi
At Tue, 13 Jun 2023 11:20:56 +0300, Konstantin Knizhnik wrote in > > > On 13.06.2023 10:55 AM, Kyotaro Horiguchi wrote: > > At Tue, 13 Jun 2023 09:55:36 +0300, Konstantin Knizhnik > > wrote in > >> Postgres backend is "thick" not because of large number of local > >> variables. > >> It is

Re: Skip collecting decoded changes of already-aborted transactions

2023-06-13 Thread Masahiko Sawada
On Sun, Jun 11, 2023 at 5:31 AM Andres Freund wrote: > > Hi, > > On 2023-06-09 14:16:44 +0900, Masahiko Sawada wrote: > > In logical decoding, we don't need to collect decoded changes of > > aborted transactions. While streaming changes, we can detect > > concurrent abort of the (sub)transaction

Re: Views no longer in rangeTabls?

2023-06-13 Thread Amit Langote
On Tue, Jun 13, 2023 at 4:44 PM David Steele wrote: > On 6/13/23 06:09, Amit Langote wrote: > > On Sat, Jun 10, 2023 at 10:27 PM Tom Lane wrote: > >> Julien Rouhaud writes: > >>> On Sat, Jun 10, 2023 at 08:56:47AM -0400, Tom Lane wrote: > - rte->relkind = 0; > >> > >>> and also handle

Re: Let's make PostgreSQL multi-threaded

2023-06-13 Thread Konstantin Knizhnik
On 13.06.2023 10:55 AM, Kyotaro Horiguchi wrote: At Tue, 13 Jun 2023 09:55:36 +0300, Konstantin Knizhnik wrote in Postgres backend is "thick" not because of large number of local variables. It is because of local caches: catalog cache, relation cache, prepared statements cache,... If they

Re: Doc limitation update proposal: include out-of-line OID usage per TOAST-ed columns

2023-06-13 Thread John Naylor
On Thu, Apr 27, 2023 at 7:36 PM Jakub Wartak wrote: > > Spurious whitespace. > > Hopefully fixed, I've tried to align with the other entries tags. Hope springs eternal. ;-) --- a/doc/src/sgml/limits.sgml +++ b/doc/src/sgml/limits.sgml @@ -10,6 +10,7 @@ hard limits are reached. + @@

Re: Let's make PostgreSQL multi-threaded

2023-06-13 Thread Kyotaro Horiguchi
At Tue, 13 Jun 2023 09:55:36 +0300, Konstantin Knizhnik wrote in > Postgres backend is "thick" not because of large number of local > variables. > It is because of local caches: catalog cache, relation cache, prepared > statements cache,... > If they are not rewritten, then backend still may

Re: Support logical replication of DDLs

2023-06-13 Thread Michael Paquier
On Mon, Jun 12, 2023 at 01:47:02AM +, Wei Wang (Fujitsu) wrote: > # load test cases from the regression tests > -my @regress_tests = split /\s+/, $ENV{REGRESS}; > +#my @regress_tests = split /\s+/, $ENV{REGRESS}; > +my @regress_tests = ("create_type", "create_schema", "create_rule", >

Re: Views no longer in rangeTabls?

2023-06-13 Thread David Steele
On 6/13/23 06:09, Amit Langote wrote: On Sat, Jun 10, 2023 at 10:27 PM Tom Lane wrote: Julien Rouhaud writes: On Sat, Jun 10, 2023 at 08:56:47AM -0400, Tom Lane wrote: - rte->relkind = 0; and also handle that field in (read|out)funcs.c Oh, right. Ugh, that means a catversion bump.

Re: Avoid unncessary always true test (src/backend/storage/buffer/bufmgr.c)

2023-06-13 Thread Kyotaro Horiguchi
At Tue, 13 Jun 2023 15:11:26 +0900, Michael Paquier wrote in > On Mon, Jun 12, 2023 at 10:51:24PM -0700, Gurjeet Singh wrote: > > To me, it looks like these checks are a result of code being > > copy-pasted from somewhere else, where this check might have been > > necessary. The checks are sure

Re: add non-option reordering to in-tree getopt_long

2023-06-13 Thread Kyotaro Horiguchi
At Mon, 12 Jun 2023 22:13:43 -0700, Nathan Bossart wrote in > On Tue, Jun 13, 2023 at 12:00:01PM +0900, Kyotaro Horiguchi wrote: > > POSIXLY_CORRECT appears to be intended for debugging or feature > > validation. If we know we can always rearrange argv on those > > platforms, we don't need it.

Re: Document that server will start even if it's unable to open some TCP/IP ports

2023-06-13 Thread Gurjeet Singh
On Mon, Jun 12, 2023 at 10:59 PM Nathan Bossart wrote: > On Sat, May 27, 2023 at 03:17:21PM -0700, Gurjeet Singh wrote: > > If listen_addresses is empty, then server won't try to open any TCP/IP > > ports. The patch does not change any language related to that. > > Your proposed change notes that

Re: Let's make PostgreSQL multi-threaded

2023-06-13 Thread Konstantin Knizhnik
On 12.06.2023 3:23 PM, Pavel Borisov wrote: Is the following true or not? 1. If we switch processes to threads but leave the amount of session local variables unchanged, there would be hardly any performance gain. 2. If we move some backend's local variables into shared memory then the

Re: Improve logging when using Huge Pages

2023-06-13 Thread Michael Paquier
On Mon, Jun 12, 2023 at 11:11:14PM -0700, Nathan Bossart wrote: > On Tue, Jun 13, 2023 at 02:50:30PM +0900, Michael Paquier wrote: >> Don't we need to update save_backend_variables() and add an entry >> in BackendParameters to make other processes launched by EXEC_BACKEND >> inherit the status

Re: pg_get_indexdef() modification to use TxnSnapshot

2023-06-13 Thread Masahiko Sawada
On Fri, May 26, 2023 at 6:55 PM shveta malik wrote: > > I have attempted to convert pg_get_indexdef() to use > systable_beginscan() based on transaction-snapshot rather than using > SearchSysCache(). The latter does not have any old info and thus > provides only the latest info as per the

Re: query_id, pg_stat_activity, extended query protocol

2023-06-13 Thread kaido vaikla
Thnx. br Kaido On Tue, 13 Jun 2023 at 03:16, Michael Paquier wrote: > On Mon, Jun 12, 2023 at 09:03:24PM +0300, kaido vaikla wrote: > > I have noticed, if query comes from PostgreSQL JDBC Driver, then query_id > > is not present in pg_stat_activity. Erik Wienhold figured out that > reason > >

  1   2   >