RE: Initial Schema Sync for Logical Replication

2023-04-18 Thread Kumar, Sachin
> From: Masahiko Sawada > > > While writing a PoC patch, I found some difficulties in this idea. > > > First, I tried to add schemaname+relname to pg_subscription_rel but > > > I could not define the primary key of pg_subscription_rel. The > > > primary key on (srsubid, srrelid) doesn't work since

Re: Enhanced rmgr desc routines test !has_image, not has_data

2023-04-18 Thread Michael Paquier
On Tue, Apr 18, 2023 at 02:36:40PM -0700, Peter Geoghegan wrote: > This isn't just a theoretical issue. Currently, we won't display > detailed descriptions of block data whenever wal_consistency_checking > happens to be in use. At least for those records with relevant block > data available to summ

Re: [PATCH] Allow Postgres to pick an unused port to listen

2023-04-18 Thread Yurii Rashkovskii
Stephen, > You could just drop another file into the data directory that just > contains > > the port number ($PGDATA/port). However, if we ever do multiple ports, > that > > would still require a change in the format of that file, so I don't know > if > > that's actually better than a). > I fin

Re: [PATCH] Allow Postgres to pick an unused port to listen

2023-04-18 Thread Stephen Frost
Greetings, * Peter Eisentraut (peter.eisentr...@enterprisedb.com) wrote: > On 13.04.23 04:45, Yurii Rashkovskii wrote: > > But getting your agreement is important to get this in; I am willing to > > play along and resolve both (1) and (2) in one go. As for the > > implementation approach for (2),

Re: [PATCH] Allow Postgres to pick an unused port to listen

2023-04-18 Thread Peter Eisentraut
On 13.04.23 04:45, Yurii Rashkovskii wrote: But getting your agreement is important to get this in; I am willing to play along and resolve both (1) and (2) in one go. As for the implementation approach for (2), which of the following options would you prefer? a) Document postmaster.pid as it

Re: Use INT_MAX for wal size related gucs's max value

2023-04-18 Thread Junwang Zhao
These gucs are always used with ConvertToXSegs, to calculate the count of wal segments(see the following code snip), and wal_segment_size can be configured by initdb as a value of a power of 2 between 1 and 1024 (megabytes), so I think INT_MAX should be safe here. /* * Convert values of GUCs measu

Re: Use INT_MAX for wal size related gucs's max value

2023-04-18 Thread Tom Lane
Junwang Zhao writes: > The wal size related gucs use the MB unit, so we should just use > INT_MAX instead of MAX_KILOBYTES as the max value. The point of MAX_KILOBYTES is to avoid overflow when the value is multiplied by 1kB. It does seem like that might not be appropriate for these values, but

Use INT_MAX for wal size related gucs's max value

2023-04-18 Thread Junwang Zhao
The wal size related gucs use the MB unit, so we should just use INT_MAX instead of MAX_KILOBYTES as the max value. -- Regards Junwang Zhao 0001-use-INT_MAX-for-wal-size-related-max-value.patch Description: Binary data

Re: Allowing parallel-safe initplans

2023-04-18 Thread Richard Guo
On Tue, Apr 18, 2023 at 9:33 PM Tom Lane wrote: > Richard Guo writes: > > It seems that in this case the top_plan does not have any extParam, so > > the Gather node that is added atop the top_plan does not have a chance > > to get its initParam filled in set_param_references(). > > Oh, so maybe

Re: check_strxfrm_bug()

2023-04-18 Thread Jonathan S. Katz
On 4/18/23 9:19 PM, Thomas Munro wrote: On Tue, Apr 18, 2023 at 11:52 AM Michael Paquier wrote: On Mon, Apr 17, 2023 at 03:40:14PM -0700, Peter Geoghegan wrote: +1 for getting rid of TRUST_STRXFRM. +1 The situation is not improving fast, and requires hard work to follow on each OS. Clearly

Re: pg_collation.collversion for C.UTF-8

2023-04-18 Thread Thomas Munro
On Wed, Apr 19, 2023 at 1:30 PM Jeff Davis wrote: > On Wed, 2023-04-19 at 07:48 +1200, Thomas Munro wrote: > > Many OSes have a locale with this name. I don't know this history, > > who did it first etc, but now I am wondering if they all took the > > "obvious" interpretation, that it should be c

Re: pg_collation.collversion for C.UTF-8

2023-04-18 Thread Jeff Davis
On Wed, 2023-04-19 at 07:48 +1200, Thomas Munro wrote: > Many OSes have a locale with this name.  I don't know this history, > who did it first etc, but now I am wondering if they all took the > "obvious" interpretation, that it should be code-point based, > extrapolating from "C" (really memcmp or

Re: check_strxfrm_bug()

2023-04-18 Thread Thomas Munro
On Tue, Apr 18, 2023 at 11:52 AM Michael Paquier wrote: > On Mon, Apr 17, 2023 at 03:40:14PM -0700, Peter Geoghegan wrote: > > +1 for getting rid of TRUST_STRXFRM. +1 The situation is not improving fast, and requires hard work to follow on each OS. Clearly, mainstreaming ICU is the way to go.

Re: Should we put command options in alphabetical order in the doc?

2023-04-18 Thread Peter Geoghegan
On Tue, Apr 18, 2023 at 4:30 PM Peter Geoghegan wrote: > > I'd be interested to know if you could tell me if SKIP_LOCKED has more > > importance than INDEX_CLEANUP, for example. If you can, it would seem > > like trying to say apples are more important than oranges, or > > vice-versa. > > I don't

Re: allow_in_place_tablespaces vs. pg_basebackup

2023-04-18 Thread Michael Paquier
On Tue, Apr 18, 2023 at 11:35:41AM -0400, Robert Haas wrote: > On Mon, Apr 17, 2023 at 1:30 AM Michael Paquier wrote: >> FWIW, doing that now rather than the beginning of July is OK for me >> for this stuff. > > OK, committed. Thanks! -- Michael signature.asc Description: PGP signature

Re: Should we put command options in alphabetical order in the doc?

2023-04-18 Thread Peter Geoghegan
On Tue, Apr 18, 2023 at 4:18 PM David Rowley wrote: > "Importance order" just seems horribly subjective to me. Alphabetical order seems objectively bad. At least to me. > I'd be interested to know if you could tell me if SKIP_LOCKED has more > importance than INDEX_CLEANUP, for example. If you c

Re: Should we put command options in alphabetical order in the doc?

2023-04-18 Thread David Rowley
On Tue, 18 Apr 2023 at 18:53, Peter Geoghegan wrote: > Take the VACUUM command. Right now FULL, FREEZE, and VERBOSE all come > first. Those options are approximately the most important options -- > especially VERBOSE. But your patch places VERBOSE dead last. hmm, how can we verify that the option

Enhanced rmgr desc routines test !has_image, not has_data

2023-04-18 Thread Peter Geoghegan
Recent commits that enhanced rmgr desc routines (commits 7d8219a4 and 1c453cfd) dealt with records that lack relevant block data (and so lack anything to give a more detailed summary of) by testing !DecodedBkpBlock.has_image -- that is the gating condition that determines if we want to (say) output

Re: Request for comment on setting binary format output per session

2023-04-18 Thread Greg Stark
On Mon, 17 Apr 2023 at 16:22, Tom Lane wrote: > > I tend to agree with the proposition that we aren't going to add new > message types very often, as long as we're careful to make them general > purpose. Don't forget that adding a new message type isn't just a matter > of writing some spec text -

Re: pg_collation.collversion for C.UTF-8

2023-04-18 Thread Thomas Munro
On Wed, Apr 19, 2023 at 12:36 AM Daniel Verite wrote: > This seems to be based on the idea that C.* collations provide an > immutable sort like "C", but it appears that it's not the case. Hmm. It seems I added that exemption initially for FreeBSD only in ca051d8b101, and then merged the cases fo

Re: Direct I/O

2023-04-18 Thread Greg Stark
On Mon, 17 Apr 2023 at 17:45, Thomas Munro wrote: > > Reasons: (1) There will always be a > few file systems that refuse O_DIRECT (Linux tmpfs is one such, as we > learned in this thread; if fails with EINVAL at open() time), and So why wouldn't we just automatically turn it off (globally or for

Re: Fix typos and inconsistencies for v16

2023-04-18 Thread Tom Lane
Justin Pryzby writes: > On Tue, Apr 18, 2023 at 02:06:43PM +1200, David Rowley wrote: >> On Tue, 18 Apr 2023 at 10:10, Justin Pryzby wrote: >>> and s/evade/avoid/ >> I didn't touch this. You'll need to provide more justification for why >> you think it's more correct than what's there. > I'd

Re: Fix typos and inconsistencies for v16

2023-04-18 Thread Alexander Lakhin
Hi Justin and David, 18.04.2023 01:10, Justin Pryzby wrote: Well done. Thank you for reviewing! On Mon, Apr 17, 2023 at 09:00:00PM +0300, Alexander Lakhin wrote: Hello hackers, Please consider fixing the following unique words/identifiers introduced in v16: Note that your patches are over

Re: Fix typos and inconsistencies for v16

2023-04-18 Thread Justin Pryzby
On Tue, Apr 18, 2023 at 02:06:43PM +1200, David Rowley wrote: > On Tue, 18 Apr 2023 at 10:10, Justin Pryzby wrote: > > > - * USER SET values are appliciable only for PGC_USERSET > > > parameters. We > > > + * USER SET values are applicable only for PGC_USERSET > > > par

Re: [PATCH] Compression dictionaries for JSONB

2023-04-18 Thread Aleksander Alekseev
Matthias, Nikita, Many thanks for the feedback! > Any type with typlen < 0 should work, right? Right. > The use of dictionaries should be dependent on only the use of a > compression method that supports pre-computed compression > dictionaries. I think storage=MAIN + compression dictionaries sh

Re: constants for tar header offsets

2023-04-18 Thread Dagfinn Ilmari Mannsåker
Robert Haas writes: > On Tue, Apr 18, 2023 at 12:06 PM Tom Lane wrote: >> Hmm, you're right: I checked the POSIX.1-2018 spec as well, and >> it agrees that the prefix field is 155 bytes long. Perhaps just >> add another comment line indicating that 12 bytes remain unassigned? > > OK. Here's v2,

Re: constants for tar header offsets

2023-04-18 Thread Tom Lane
Robert Haas writes: > OK. Here's v2, with that change and a few others. LGTM. regards, tom lane

Re: Request for comment on setting binary format output per session

2023-04-18 Thread Dave Cramer
On Tue, 18 Apr 2023 at 12:24, Robert Haas wrote: > On Tue, Apr 18, 2023 at 11:51 AM Tom Lane wrote: > > Robert Haas writes: > > > One thing I think we should do in this area is introduce #defines for > > > all the message type codes and use those instead of having hard-coded > > > constants eve

Re: constants for tar header offsets

2023-04-18 Thread Robert Haas
On Tue, Apr 18, 2023 at 12:06 PM Tom Lane wrote: > Hmm, you're right: I checked the POSIX.1-2018 spec as well, and > it agrees that the prefix field is 155 bytes long. Perhaps just > add another comment line indicating that 12 bytes remain unassigned? OK. Here's v2, with that change and a few ot

Re: Request for comment on setting binary format output per session

2023-04-18 Thread Robert Haas
On Tue, Apr 18, 2023 at 11:51 AM Tom Lane wrote: > Robert Haas writes: > > One thing I think we should do in this area is introduce #defines for > > all the message type codes and use those instead of having hard-coded > > constants everywhere. > > +1, but I wonder where we should put those exact

Re: [PATCH] Compression dictionaries for JSONB

2023-04-18 Thread Nikita Malakhov
Hi, I don't think it's a good idea to interfere with the storage strategies. Dictionary should be a kind of storage option, like a compression, but not the strategy declining all others. >> While thinking about how a user interface could look like it occured >> to me that what we are discussing c

Re: constants for tar header offsets

2023-04-18 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 18, 2023 at 11:38 AM Tom Lane wrote: >> 2. The header size is defined as 512 bytes, but this doesn't sum to 512: >> + TAR_OFFSET_PREFIX = 345 /* 155 byte string */ > I think that what happened is that whoever designed the original tar > format deci

Re: constants for tar header offsets

2023-04-18 Thread Robert Haas
On Tue, Apr 18, 2023 at 11:38 AM Tom Lane wrote: > Robert Haas writes: > > We have a few different places in the code where we generate or modify > > tar headers or just read data out of them. The code in question uses > > one of my less-favorite programming things: magic numbers. The offsets > >

Re: Request for comment on setting binary format output per session

2023-04-18 Thread Tom Lane
Robert Haas writes: > One thing I think we should do in this area is introduce #defines for > all the message type codes and use those instead of having hard-coded > constants everywhere. +1, but I wonder where we should put those exactly. My first thought was postgres_ext.h, but the charter for

Re: Request for comment on setting binary format output per session

2023-04-18 Thread Robert Haas
On Mon, Apr 17, 2023 at 4:22 PM Tom Lane wrote: > The fact that we've gotten away without adding *any* new message types > for about twenty years suggests to me that the growth rate isn't such > that we need sub-message-types yet. I'd keep the structure the same > until such time as we can't choo

Re: [PATCH] Compression dictionaries for JSONB

2023-04-18 Thread Matthias van de Meent
On Tue, 18 Apr 2023 at 17:28, Aleksander Alekseev wrote: > > Hi Andres, > > > As I said, I don't think we should extend dictionaries. For this to work > > we'll > > likely need a new / extended compressed toast datum header of some form, > > with > > a reference to the dictionary. That'd likely

Re: constants for tar header offsets

2023-04-18 Thread Tom Lane
Robert Haas writes: > We have a few different places in the code where we generate or modify > tar headers or just read data out of them. The code in question uses > one of my less-favorite programming things: magic numbers. The offsets > of the various fields within the tar header are just hard-c

Re: allow_in_place_tablespaces vs. pg_basebackup

2023-04-18 Thread Robert Haas
On Mon, Apr 17, 2023 at 1:30 AM Michael Paquier wrote: > FWIW, doing that now rather than the beginning of July is OK for me > for this stuff. OK, committed. -- Robert Haas EDB: http://www.enterprisedb.com

constants for tar header offsets

2023-04-18 Thread Robert Haas
Hi, We have a few different places in the code where we generate or modify tar headers or just read data out of them. The code in question uses one of my less-favorite programming things: magic numbers. The offsets of the various fields within the tar header are just hard-coded in each relevant pl

Re: Temporary tables versus wraparound... again

2023-04-18 Thread Greg Stark
Hm, in an optimized build using kernel perf I see this. But I don't know how to find what the call sites are for LWLockAcquire/Release. If it's the locks on pgproc that would be kind of bad. I wonder if I should be gathering horizons once in the PrecommitActions and then just using those for every

Re: [PATCH] Compression dictionaries for JSONB

2023-04-18 Thread Aleksander Alekseev
Hi Andres, > As I said, I don't think we should extend dictionaries. For this to work we'll > likely need a new / extended compressed toast datum header of some form, with > a reference to the dictionary. That'd likely be needed even with updatable > dictionaries, as we IIRC don't know which colum

Re: Allowing parallel-safe initplans

2023-04-18 Thread Tom Lane
Richard Guo writes: > On Mon, Apr 17, 2023 at 11:04 PM Tom Lane wrote: >> I wondered about that too, but how come neither of us saw non-cosmetic >> failures (ie, actual query output changes not just EXPLAIN changes) >> when we tried this? > Sorry I forgot to mention that I did see query output c

pg_collation.collversion for C.UTF-8

2023-04-18 Thread Daniel Verite
Hi, get_collation_actual_version() in pg_locale.c currently excludes C.UTF-8 (and more generally C.*) from versioning, which makes pg_collation.collversion being empty for these collations. char * get_collation_actual_version(char collprovider, const char *collcollate) { if (collpr

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-04-18 Thread Daniel Gustafsson
> On 17 Apr 2023, at 18:20, Jacob Champion wrote: > Note that it won't fix the > (completely different) misleading error message for OpenSSL 3.0, but > since that's an *actively* unhelpful error message coming back from > OpenSSL, I don't think we want to override it. Agreed, the best we can do

Re: Adding argument names to aggregate functions

2023-04-18 Thread Jim Jones
On 18.04.23 12:27, Dagfinn Ilmari Mannsåker wrote: Link to the actual job: https://cirrus-ci.com/task/5881376021413888 The failure was: [09:54:38.727] 216/262 postgresql:recovery / recovery/031_recovery_conflict ERROR 198.73s exit status 60 Looking at its log: https://api.cirrus-ci.com/v1/ar

Re: Adding argument names to aggregate functions

2023-04-18 Thread Dagfinn Ilmari Mannsåker
Jim Jones writes: > On 18.04.23 10:58, I wrote: >> On 14.04.23 12:03, Dagfinn Ilmari Mannsåker wrote: >>> Thanks for the heads-up, here's a rebased patch. I've also formatted >>> the lines to match what reformat_dat_file.pl wants.  It also wanted to >>> reformat a bunch of other entries, but I le

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-04-18 Thread David Rowley
On Tue, 18 Apr 2023 at 19:29, Miroslav Bendik wrote: > here is an updated patch with proposed changes. Here's a quick review: 1. I don't think this is required. match_pathkeys_to_index() sets these to NIL and they're set accordingly by the other code paths. - List*orderbyclauses; - List

Re: Adding argument names to aggregate functions

2023-04-18 Thread Jim Jones
On 18.04.23 10:58, I wrote: On 14.04.23 12:03, Dagfinn Ilmari Mannsåker wrote: Thanks for the heads-up, here's a rebased patch. I've also formatted the lines to match what reformat_dat_file.pl wants.  It also wanted to reformat a bunch of other entries, but I left those alone. - ilmari The pa

Tab completion for GRANT MAINTAIN

2023-04-18 Thread Ken Kato
Hi hackers, I found that GRANT MAINTAIN is not tab-completed with ON, so here is a patch. Best wishes, -- Ken Kato Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATIONdiff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c index 5825b2

Support "Right Semi Join" plan shapes

2023-04-18 Thread Richard Guo
In thread [1] which discussed 'Right Anti Join', Tom once mentioned 'Right Semi Join'. After a preliminary investigation I think it is beneficial and can be implemented with very short change. With 'Right Semi Join', what we want to do is to just have the first match for each inner tuple. For Ha

Re: Adding argument names to aggregate functions

2023-04-18 Thread Jim Jones
On 14.04.23 12:03, Dagfinn Ilmari Mannsåker wrote: Thanks for the heads-up, here's a rebased patch. I've also formatted the lines to match what reformat_dat_file.pl wants. It also wanted to reformat a bunch of other entries, but I left those alone. - ilmari The patch applies cleanly now and \

Re: Fix documentation for max_wal_size and min_wal_size

2023-04-18 Thread sirisha chamarthi
Hi On Mon, Apr 17, 2023 at 9:38 PM Michael Paquier wrote: > On Mon, Apr 17, 2023 at 07:57:58PM -0700, sirisha chamarthi wrote: > > On Fri, Apr 14, 2023 at 1:01 AM Kyotaro Horiguchi < > horikyota@gmail.com> > > wrote: > >> So, I personally think it should be written like this: "The default >

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-04-18 Thread Miroslav Bendik
po 17. 4. 2023 o 15:26 Richard Guo napísal(a): > > > On Sun, Apr 16, 2023 at 1:20 AM Miroslav Bendik > wrote: >> >> Postgres allows incremental sort only for ordered indexes. Function >> build_index_paths dont build partial order paths for access methods >> with order support. My patch adds supp

Re: Allowing parallel-safe initplans

2023-04-18 Thread Richard Guo
On Mon, Apr 17, 2023 at 11:04 PM Tom Lane wrote: > Richard Guo writes: > > So now it seems that the breakage of regression tests is more severe > > than being cosmetic. I wonder if we need to update the comments to > > indicate the potential wrong results issue if we move the initPlans to > > t