Re: Collation version tracking for macOS

2022-06-10 Thread Peter Geoghegan
On Fri, Jun 10, 2022 at 9:08 PM Thomas Munro wrote: > They're still useful for non-ICU collations (for example FreeBSD and > Windows can tell you about version changes based on open standards), > and they're *maybe* still useful for ICU, considering that there > are minor version upgrades, though

Re: Collation version tracking for macOS

2022-06-10 Thread Peter Geoghegan
On Fri, Jun 10, 2022 at 8:47 PM Thomas Munro wrote: > I'm also suspicious that there are more subtle hazards like pathkeys > lurking in the shadows. We go to great effort to recognise matching > and non-matching collations by OID alone, which is why my first > attempt was "distinct [OIDs]", so

Re: Collation version tracking for macOS

2022-06-10 Thread Thomas Munro
On Sat, Jun 11, 2022 at 3:36 PM Peter Geoghegan wrote: > Do we even need to store a version for indexes most of the time if > we're versioning ICU itself, as part of the "time travelling > collations" design? For that matter, do we even need to version > collations directly anymore? They're

Re: Collation version tracking for macOS

2022-06-10 Thread Thomas Munro
On Sat, Jun 11, 2022 at 2:29 PM Peter Geoghegan wrote: > The special REINDEX (or whatever) won't work as an atomic > operation...but that doesn't mean that the system as a whole will have > a mix of old and new physical collations forever, or even for very > long. So while everything still has to

Re: Collation version tracking for macOS

2022-06-10 Thread Peter Geoghegan
On Thu, Jun 9, 2022 at 9:31 PM Thomas Munro wrote: > Perhaps that could be modeled with a pg_depend row pointing to a > pg_icu_library row, which you'd probably need anyway, to prevent a > registered ICU library that is needed for a live index from being > dropped. (That's assuming that the

Re: Collation version tracking for macOS

2022-06-10 Thread Peter Geoghegan
On Fri, Jun 10, 2022 at 6:48 PM Thomas Munro wrote: > Executive summary of experiments so far: the "distinct collations" > concept is quite simple and robust, but exposes all the versions to > users and probably makes it really hard to upgrade (details not worked > out), while the "time

Re: Collation version tracking for macOS

2022-06-10 Thread Thomas Munro
On Fri, Jun 10, 2022 at 4:30 PM Thomas Munro wrote: > I'm not sold on any particular plan, but working through some examples > helped me see your idea better... I may try to code that up in a > minimal way so we can kick the tyres... I did a bit of hacking on that idea. The goal was to stamp

Re: Replica Identity check of partition table on subscriber

2022-06-10 Thread Amit Kapila
On Fri, Jun 10, 2022 at 2:26 PM Amit Langote wrote: > > +logicalrep_partmap_invalidate > > I wonder why not call this logicalrep_partmap_update() to go with > logicalrep_relmap_update()? It seems confusing to have > logicalrep_partmap_invalidate() right next to >

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2022-06-10 Thread David Zhang
Hi, > > MaxAllocSize is pretty easy: > > SELECT pg_logical_emit_message(false, long, long) FROM repeat(repeat(' ', 1024), 1024*1023) as l(long); > > > > on a standby: > > > > 2022-03-11 16:41:59.336 PST [3639744][startup][1/0:0] LOG:  record length 2145386550 at 0/360 too long > > Thanks

Re: replacing role-level NOINHERIT with a grant-level option

2022-06-10 Thread Stephen Frost
Greetings, On Fri, Jun 10, 2022 at 16:36 Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 02.06.22 18:26, Robert Haas wrote: > > On Mon, Feb 7, 2022 at 11:13 AM Joe Conway wrote: > >>> It seems to me that the INHERIT role flag isn't very well-considered. > >>> Inheritance, or

Re: [v15 beta] pg_upgrade failed if earlier executed with -c switch

2022-06-10 Thread Andrew Dunstan
On 2022-06-08 We 20:53, Michael Paquier wrote: > On Wed, Jun 08, 2022 at 04:13:37PM -0500, Justin Pryzby wrote: >> On Wed, Jun 08, 2022 at 10:55:29AM +0900, Michael Paquier wrote: >>> And applied, to take care of this open item. >> Shouldn't this wait for the buildfarm to be updated again ? >

Re: replacing role-level NOINHERIT with a grant-level option

2022-06-10 Thread Peter Eisentraut
On 02.06.22 18:26, Robert Haas wrote: On Mon, Feb 7, 2022 at 11:13 AM Joe Conway wrote: It seems to me that the INHERIT role flag isn't very well-considered. Inheritance, or the lack of it, ought to be decided separately for each inherited role. However, that would be a major architectural

Re: Removing "plpythonu" in PG 15

2022-06-10 Thread Bruce Momjian
On Fri, Jun 10, 2022 at 02:53:22PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > I see we removed "plpythonu" when we removed "plpython2u" in PG 15. Is > > there a good reason for that? > > There was extensive discussion of that in the relevant threads, > but basically (1) risk of

Re: Removing "plpythonu" in PG 15

2022-06-10 Thread Tom Lane
Bruce Momjian writes: > I see we removed "plpythonu" when we removed "plpython2u" in PG 15. Is > there a good reason for that? There was extensive discussion of that in the relevant threads, but basically (1) risk of confusion and (2) most python installations have removed "python", not

Re: better page-level checksums

2022-06-10 Thread Robert Haas
On Fri, Jun 10, 2022 at 12:08 PM Stephen Frost wrote: > So, it's not quite as simple as use X or use Y, we need to be > considering the use case too. In particular, the amount of data that's > being hash'd is relevant when it comes to making a decision about what > hash or checksum to use. When

Removing "plpythonu" in PG 15

2022-06-10 Thread Bruce Momjian
I see we removed "plpythonu" when we removed "plpython2u" in PG 15. Is there a good reason for that? We don't have the software version number in other server-side language names, as far as I know. We added "plpython2u" when we were adding "plpython3u", but now that we have removed

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-06-10 Thread Jacob Champion
On Thu, Jun 9, 2022 at 6:23 AM Robert Haas wrote: > On Wed, Jun 8, 2022 at 7:53 PM Jacob Champion wrote: > > But I don't have any better ideas for how to achieve both. I'm fine > > with your suggestion of ClientConnectionInfo, if that sounds good to > > others; the doc comment can clarify why it

Support load balancing in libpq

2022-06-10 Thread Jelte Fennema
Load balancing connections across multiple read replicas is a pretty common way of scaling out read queries. There are two main ways of doing so, both with their own advantages and disadvantages: 1. Load balancing at the client level 2. Load balancing by connecting to an intermediary load balancer

Re: Error from the foreign RDBMS on a foreign table I have no privilege on

2022-06-10 Thread Phil Florent
Hi, Thanks for your explanations. Test case had no real-world logic anyway. It was just an oversight in a one-time use legacy migration script. Regards, Phil From: Laurenz Albe Sent: Friday, June 10, 2022 11:17:07 AM To: Etsuro Fujita Cc: Kyotaro Horiguchi ;

Re: better page-level checksums

2022-06-10 Thread Stephen Frost
Greetings, * Fabien COELHO (coe...@cri.ensmp.fr) wrote: > >I think for this purpose we should limit ourselves to algorithms > >whose output size is, at minimum, 64 bits, and ideally, a multiple of > >64 bits. I'm sure there are plenty of options other than the ones that > >btrfs uses; I mentioned

Re: better page-level checksums

2022-06-10 Thread Stephen Frost
Greetings, * Andrey Borodin (x4m@double.cloud) wrote: > On Fri, Jun 10, 2022 at 5:00 AM Matthias van de Meent < > boekewurm+postg...@gmail.com> wrote: > > Can't we add some extra fork that stores this extra per-page > > information, and contains this extra metadata > > +1 for this approach. I

Re: Sharing DSA pointer between parallel workers after they've been created

2022-06-10 Thread Robert Haas
On Thu, Jun 9, 2022 at 2:36 PM Ma, Marcus wrote: > I’m currently working on a parallelization optimization of the Sequential > Scan in the codebase, and I need to share information between the workers as > they scan a relation. I’ve done a decent amount of testing, and I know that > the

Re: better page-level checksums

2022-06-10 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Jun 10, 2022 at 9:36 AM Peter Eisentraut > wrote: > > I think there ought to be a bit more principled analysis here than just > > "let's add a lot more bits". There is probably some kind of information > > to be had about how

Re: better page-level checksums

2022-06-10 Thread Robert Haas
On Fri, Jun 10, 2022 at 9:36 AM Peter Eisentraut wrote: > I think there ought to be a bit more principled analysis here than just > "let's add a lot more bits". There is probably some kind of information > to be had about how many CRC bits are useful for a given block size, say. > > And then

Re: better page-level checksums

2022-06-10 Thread Robert Haas
On Thu, Jun 9, 2022 at 8:00 PM Matthias van de Meent wrote: > Why so? We already dole out per-page space in 4-byte increments > through pd_linp, and I see no reason why we can't reserve some line > pointers for per-page metadata if we decide that we need extra > per-page ~overhead~ metadata.

Re: better page-level checksums

2022-06-10 Thread Peter Eisentraut
On 10.06.22 15:16, Robert Haas wrote: I'm not perfectly attached to the idea of using SHA here, but it seems to me that's pretty much the standard thing these days. Stephen Frost and David Steele pushed hard for SHA checksums in backup manifests, and actually wanted it to be the default. That

Re: Allow foreign keys to reference a superset of unique columns

2022-06-10 Thread Kaiting Chen
On Fri, Jun 10, 2022 at 12:14 AM Peter Eisentraut wrote: > > On 10.06.22 05:47, David Rowley wrote: > >> I think this should be referring to constraint name, not an index name. > > Can you explain why you think that? > > If you wanted to specify this feature in the SQL standard (I'm not >

Re: better page-level checksums

2022-06-10 Thread Robert Haas
On Thu, Jun 9, 2022 at 5:34 PM Peter Geoghegan wrote: > Why not? The only problems that it won't solve are all related to > crypto. Which is perfectly fine, but it seems like there is a > terminology issue here. ISTM that you're really talking about adding a > cryptographic hash function, not a

Re: Count output lines automatically in psql/help.c

2022-06-10 Thread Tom Lane
Peter Eisentraut writes: > I wonder whether this mix of HELP0 and HELP is necessary. The original > code didn't care about calling fprintf even if there are no > substitutions. I think this could lead to misalignment errors. I > vaguely recall we once had mixes of fprintf and fputs and got

Re: [PATCH] psql: \dn+ to show size of each schema (and \dA+ for AMs)

2022-06-10 Thread Justin Pryzby
Rebased >From 4d88986706e334e2dccc6f576139342c102033f8 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 13 Jul 2021 21:25:48 -0500 Subject: [PATCH 1/4] Add pg_am_size(), pg_namespace_size() .. See also: 358a897fa, 528ac10c7 --- src/backend/utils/adt/dbsize.c | 130

Re: Count output lines automatically in psql/help.c

2022-06-10 Thread Peter Eisentraut
On 03.06.22 22:51, Tom Lane wrote: + HELP0(" -c, --command=COMMANDrun only single command (SQL or internal) and exit\n"); + HELP(" -d, --dbname=DBNAME database name to connect to (default: \"%s\")\n", +env); I wonder whether this mix of HELP0 and HELP

Re: A proposal to force-drop replication slots to make disabling async/sync standbys or logical replication faster in production environments

2022-06-10 Thread Bharath Rupireddy
On Thu, Jun 9, 2022 at 12:11 PM Amit Kapila wrote: > > On Thu, Jun 9, 2022 at 11:07 AM Bharath Rupireddy > wrote: > > > > Currently postgres doesn't allow dropping a replication slot that's active > > [1]. This can make certain operations more time-consuming or stuck in > > production

Re: A proposal to force-drop replication slots to make disabling async/sync standbys or logical replication faster in production environments

2022-06-10 Thread Bharath Rupireddy
On Thu, Jun 9, 2022 at 11:24 AM Tom Lane wrote: > > Bharath Rupireddy writes: > > How about we provide a function to force-drop a replication slot? > > Isn't this akin to filing off the safety interlock on the loaded revolver > you keep in your hip pocket? IMO the entire point of replication

Re: Multi-Master Logical Replication

2022-06-10 Thread Amit Kapila
On Fri, Jun 10, 2022 at 12:40 PM Bharath Rupireddy wrote: > > On Fri, Jun 10, 2022 at 9:54 AM Amit Kapila wrote: > > > > > > 1. Are you proposing to use logical replication subscribers to be in > > > sync quorum? In other words, in an N-masters node, M (M >= N)-node > > > configuration, will

Re: Error from the foreign RDBMS on a foreign table I have no privilege on

2022-06-10 Thread Laurenz Albe
On Fri, 2022-06-10 at 17:17 +0900, Etsuro Fujita wrote: > > I am not sure if it worth adding to the documentation.  I would never have > >thought > > of the problem if Phil hadn't brought it up.  On the other hand, I was > > surprised > > to learn that permissions aren't checked until the

Re: Replica Identity check of partition table on subscriber

2022-06-10 Thread Amit Langote
Hello, On Wed, Jun 8, 2022 at 5:47 PM shiy.f...@fujitsu.com wrote: > Hi hackers, > > I saw a problem in logical replication, when the target table on subscriber > is a > partitioned table, it only checks whether the Replica Identity of partitioned > table is consistent with the publisher, and

RE: pgcon unconference / impact of block size on performance

2022-06-10 Thread Jakub Wartak
> On 6/9/22 13:23, Jakub Wartak wrote: > >>> The really > >> puzzling thing is why is the filesystem so much slower for > >> smaller pages. I mean, why would writing 1K be 1/3 of writing 4K? > >> Why would a filesystem have such effect? > > > > Ha! I don't care at this

RE: Replica Identity check of partition table on subscriber

2022-06-10 Thread shiy.f...@fujitsu.com
On Thu, June 9, 2022 7:02 PM Amit Kapila wrote: > > > I think one approach to fix it is to check the target partition in this > > case, > > instead of the partitioned table. > > > > This approach sounds reasonable to me. One minor point: > +/* > + * Check that replica identity matches. > + * >

Re: Handle infinite recursion in logical replication setup

2022-06-10 Thread Peter Smith
Below are some review comments for the patch v18-0004 == 1. Commit message Document the steps for the following: a) Create a two-node bidirectional replication when there is no data in both the nodes. b) Adding a new node when there is no data in any of the nodes. c) Adding a new node when

Re: better page-level checksums

2022-06-10 Thread Andrey Borodin
On Fri, Jun 10, 2022 at 5:00 AM Matthias van de Meent < boekewurm+postg...@gmail.com> wrote: > > Can't we add some extra fork that stores this extra per-page > information, and contains this extra metadata > +1 for this approach. I had observed some painful corruption cases where block storage

Re: Error from the foreign RDBMS on a foreign table I have no privilege on

2022-06-10 Thread Etsuro Fujita
On Fri, Jun 10, 2022 at 1:26 AM Laurenz Albe wrote: > On Thu, 2022-06-09 at 21:55 +0900, Etsuro Fujita wrote: > > However, I'm not 100% sure we really need to document > > something about this, because 1) this doesn't cause any actual > > problems, as you described, and 2) this is a

Re: Multi-Master Logical Replication

2022-06-10 Thread Bharath Rupireddy
On Fri, Jun 10, 2022 at 9:54 AM Amit Kapila wrote: > > On Thu, Jun 9, 2022 at 6:04 PM Bharath Rupireddy > wrote: > > > > On Thu, Apr 28, 2022 at 5:20 AM Peter Smith wrote: > > > > > > MULTI-MASTER LOGICAL REPLICATION > > > > > > 1.0 BACKGROUND > > > > > > Let’s assume that a user wishes to set

Re: better page-level checksums

2022-06-10 Thread Fabien COELHO
Hello Robert, I think for this purpose we should limit ourselves to algorithms whose output size is, at minimum, 64 bits, and ideally, a multiple of 64 bits. I'm sure there are plenty of options other than the ones that btrfs uses; I mentioned them only as a way of jump-starting the

Re: Using PQexecQuery in pipeline mode produces unexpected Close messages

2022-06-10 Thread Kyotaro Horiguchi
At Fri, 10 Jun 2022 15:25:44 +0900 (JST), Kyotaro Horiguchi wrote in > The current implement of PQsendQueryInternal looks like the result of > a misunderstanding of the doc. In the regression tests, that path is > excercised only for an error case, where no CloseComplete comes. > > The

Re: Support logical replication of DDLs

2022-06-10 Thread Amit Kapila
On Thu, Jun 9, 2022 at 5:14 PM houzj.f...@fujitsu.com wrote: > > Hi, > > I did some research for one potential problem[1] mentioned earlier which is > related > to the function execution when replicating DDL. > > [1]> 4. Statements that have nondeterministic side effects (e.g., as caused > > by

Re: Using PQexecQuery in pipeline mode produces unexpected Close messages

2022-06-10 Thread Kyotaro Horiguchi
(Moved to -hackers) At Wed, 8 Jun 2022 17:08:47 +0200, Alvaro Herrera wrote in > What that Close message is doing is closing the unnamed portal, which > is otherwise closed implicitly when the next one is opened. That's how > single-query mode works: if you run a single portal, it'll be kept