Re: [PATCH] json_lex_string: don't overread on bad UTF8

2024-04-30 Thread Michael Paquier
On Tue, Apr 30, 2024 at 10:39:04AM -0700, Jacob Champion wrote: > When json_lex_string() hits certain types of invalid input, it calls > pg_encoding_mblen_bounded(), which assumes that its input is > null-terminated and calls strnlen(). But the JSON lexer is constructed > with an explicit string le

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-04-30 Thread Michael Paquier
On Sat, Apr 27, 2024 at 08:33:55PM +0200, Daniel Gustafsson wrote: > > On 27 Apr 2024, at 20:32, Daniel Gustafsson wrote: > > > That's a good point, there is potential for more code removal here. The > > attached 0001 takes a stab at it while it's fresh in mind, I'll revisit > > before > > the

Re: partitioning and identity column

2024-04-30 Thread Michael Paquier
On Tue, Apr 30, 2024 at 04:29:11PM +0530, Ashutosh Bapat wrote: > PFA patch which fixes all the three problems. Please note that this was not tracked as an open item, so I have added one referring to the failures reported by Alexander. -- Michael signature.asc Description: PGP signature

Re: Fix parallel vacuum buffer usage reporting

2024-04-30 Thread Alena Rybakina
Hi! On 30.04.2024 05:18, Masahiko Sawada wrote: On Fri, Apr 26, 2024 at 9:12 PM Alena Rybakina wrote: Hi! The same script was run, but using vacuum verbose analyze, and I saw the difference again in the fifth step: with your patch: buffer usage: 32312 hits, 607 misses, 1566 dirtied master: b

Re: pg_sequence_last_value() for unlogged sequences on standbys

2024-04-30 Thread Michael Paquier
On Tue, Apr 30, 2024 at 09:05:31PM -0500, Nathan Bossart wrote: > This ended up being easier than I expected. While unlogged sequences are > only supported on v15 and above, temporary sequences have been around since > v7.2, so this will probably need to be back-patched to all supported > versions

Re: Fix parallel vacuum buffer usage reporting

2024-04-30 Thread Masahiko Sawada
On Tue, Apr 30, 2024 at 3:34 PM Anthonin Bonnefoy wrote: > > I've done some additional tests to validate the reported numbers. Using > pg_statio, it's possible to get the minimum number of block hits (Full script > attached). > > -- Save block hits before vacuum > SELECT pg_stat_force_next_flush

Re: Streaming I/O, vectored I/O (WIP)

2024-04-30 Thread Thomas Munro
On Wed, May 1, 2024 at 2:51 PM David Rowley wrote: > On Wed, 24 Apr 2024 at 14:32, David Rowley wrote: > > I've attached a patch with a few typo fixes and what looks like an > > incorrect type for max_ios. It's an int16 and I think it needs to be > > an int. Doing "max_ios = Min(max_ios, PG_INT16

Re: query_id, pg_stat_activity, extended query protocol

2024-04-30 Thread Imseih (AWS), Sami
Here is a new rev of the patch which deals with the scenario mentioned by Andrei [1] in which the queryId may change due to a cached query invalidation. [1] https://www.postgresql.org/message-id/724348C9-8023-41BC-895E-80634E79A538%40amazon.com Regards, Sami 0001-v2-Fix-Extended-Query-Proto

Re: add tab-complete for memory, serialize option and other minor issues.

2024-04-30 Thread Michael Paquier
On Sat, Apr 27, 2024 at 11:15:47AM -0400, Tom Lane wrote: > https://www.postgresql.org/message-id/3870833.1712696581%40sss.pgh.pa.us > > Post-feature-freeze is no time to be messing with behavior as basic > as WORD_BREAKS, though. Indeed. By the way, that psql completion patch has fallen through

Re: Streaming I/O, vectored I/O (WIP)

2024-04-30 Thread David Rowley
On Wed, 24 Apr 2024 at 14:32, David Rowley wrote: > I've attached a patch with a few typo fixes and what looks like an > incorrect type for max_ios. It's an int16 and I think it needs to be > an int. Doing "max_ios = Min(max_ios, PG_INT16_MAX);" doesn't do > anything when max_ios is int16. No fee

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2024-04-30 Thread Masahiko Sawada
On Wed, May 1, 2024 at 2:29 AM Tom Lane wrote: > > Alexander Korotkov writes: > > I agree that storing char signedness might seem weird. But it appears > > that we already store indexes that depend on char signedness. So, > > it's effectively property of bits-on-disk even though it affects > >

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2024-04-30 Thread Alexander Korotkov
Hi Noah, On Wed, May 1, 2024 at 5:24 AM Noah Misch wrote: > On Thu, Apr 25, 2024 at 04:59:54PM +0400, Pavel Borisov wrote: > > 0001: Optimize speed by avoiding heap visibility checking for different > > non-deduplicated index tuples as proposed by Noah Misch > > > > Speed measurements on my lapto

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2024-04-30 Thread Noah Misch
On Thu, Apr 25, 2024 at 04:59:54PM +0400, Pavel Borisov wrote: > 0001: Optimize speed by avoiding heap visibility checking for different > non-deduplicated index tuples as proposed by Noah Misch > > Speed measurements on my laptop using the exact method recommended by Noah > upthread: > Current ma

Re: pg_sequence_last_value() for unlogged sequences on standbys

2024-04-30 Thread Nathan Bossart
On Tue, Apr 30, 2024 at 08:13:17PM -0500, Nathan Bossart wrote: > Good point. I'll work on a patch along these lines, then. This ended up being easier than I expected. While unlogged sequences are only supported on v15 and above, temporary sequences have been around since v7.2, so this will prob

Re: pg_sequence_last_value() for unlogged sequences on standbys

2024-04-30 Thread Nathan Bossart
On Tue, Apr 30, 2024 at 09:06:04PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> If you create an unlogged sequence on a primary, pg_sequence_last_value() >> for that sequence on a standby will error like so: >> postgres=# select pg_sequence_last_value('test'::regclass); >> ERROR:

Re: pg_sequence_last_value() for unlogged sequences on standbys

2024-04-30 Thread Tom Lane
Nathan Bossart writes: > If you create an unlogged sequence on a primary, pg_sequence_last_value() > for that sequence on a standby will error like so: > postgres=# select pg_sequence_last_value('test'::regclass); > ERROR: could not open file "base/5/16388": No such file or directory

pg_sequence_last_value() for unlogged sequences on standbys

2024-04-30 Thread Nathan Bossart
If you create an unlogged sequence on a primary, pg_sequence_last_value() for that sequence on a standby will error like so: postgres=# select pg_sequence_last_value('test'::regclass); ERROR: could not open file "base/5/16388": No such file or directory This function is used by t

Re: TLS certificate alternate trust paths issue in libpq - certificate chain validation failing

2024-04-30 Thread Jacob Champion
On Tue, Apr 30, 2024 at 2:41 PM Thomas Spear wrote: > The full details can be found at github.com/pgjdbc/pgjdbc/discussions/3236 - > in summary, both jdbc-postgres and the psql cli seem to be affected by an > issue validating the certificate chain up to a publicly trusted root > certificate tha

TLS certificate alternate trust paths issue in libpq - certificate chain validation failing

2024-04-30 Thread Thomas Spear
Hello, I've recently joined the list on a tip from one of the maintainers of jdbc-postgres as I wanted to discuss an issue we've run into and find out if the fix we've worked out is the right thing to do, or if there is actually a bug that needs to be fixed. The full details can be found at github

pg_parse_json() should not leak token copies on failure

2024-04-30 Thread Jacob Champion
Hi, When a client of our JSON parser registers semantic action callbacks, the parser will allocate copies of the lexed tokens to pass into those callbacks. The intent is for the callbacks to always take ownership of the copied memory, and if they don't want it then they can pfree() it. However, i

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-30 Thread Dmitry Koval
Hi! 30.04.2024 6:00, Alexander Lakhin пишет: Maybe I'm doing something wrong, but the following script: CREATE TABLE t (i int, PRIMARY KEY(i)) PARTITION BY RANGE (i); CREATE TABLE tp_0 PARTITION OF t FOR VALUES FROM (0) TO (1); CREATE TABLE tp_1 PARTITION OF t FOR VALUES FROM (1) TO (2); CREATE

Re: Build with meson + clang + sanitizer resulted in undefined reference

2024-04-30 Thread Dmitry Dolgov
> On Thu, Apr 25, 2024 at 06:38:58PM +0300, Maxim Orlov wrote: > > And then upon build I've got overwhelmed by thousands of undefined > reference errors. > > fe-auth-scram.c:(.text+0x17a): undefined reference to > `__ubsan_handle_builtin_unreachable' > /usr/bin/ld: fe-auth-scram.c:(.text+0x189): un

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-30 Thread Justin Pryzby
On Thu, Apr 11, 2024 at 08:00:00PM +0300, Alexander Lakhin wrote: > 11.04.2024 16:27, Dmitry Koval wrote: > > > > Added correction (and test), see > > v3-0001-Fix-for-SPLIT-MERGE-partitions-of-temporary-table.patch. > > Thank you for the correction, but may be an attempt to merge into implicit >

Re: tablecmds.c/MergeAttributes() cleanup

2024-04-30 Thread Robert Haas
On Tue, Apr 30, 2024 at 2:19 AM Ashutosh Bapat wrote: > On Mon, Apr 29, 2024 at 6:46 PM Robert Haas wrote: >> On Sat, Apr 20, 2024 at 12:17 AM Ashutosh Bapat >> wrote: >> > Yes please. Probably this issue surfaced again after we reverted >> > compression and storage fix? Please If that's the c

Re: Support tid range scan in parallel?

2024-04-30 Thread Cary Huang
Hi David Thank you for your reply. > From a CPU point of view, I'd hard to imagine that a SELECT * query > without any other items in the WHERE clause other than the TID range > quals would run faster with multiple workers than with 1. The problem > is the overhead of pushing tuples to the main

Re: pg17 issues with not-null contraints

2024-04-30 Thread Justin Pryzby
On Tue, Apr 30, 2024 at 01:52:02PM -0400, Robert Haas wrote: > On Thu, Apr 18, 2024 at 12:52 PM Justin Pryzby wrote: > > I'm not totally clear on what's intended in v17 - maybe it'd be dead > > code, and maybe it shouldn't even be applied to master branch. But I do > > think it's worth patching e

Control flow in logical replication walsender

2024-04-30 Thread Christophe Pettus
Hi, I wanted to check my understanding of how control flows in a walsender doing logical replication. My understanding is that the (single) thread in each walsender process, in the simplest case, loops on: 1. Pull a record out of the WAL. 2. Pass it to the recorder buffer code, which, 3. Sor

Re: pg17 issues with not-null contraints

2024-04-30 Thread Robert Haas
On Thu, Apr 18, 2024 at 12:52 PM Justin Pryzby wrote: > I'm not totally clear on what's intended in v17 - maybe it'd be dead > code, and maybe it shouldn't even be applied to master branch. But I do > think it's worth patching earlier versions (even though it'll be less > useful than having done

[PATCH] json_lex_string: don't overread on bad UTF8

2024-04-30 Thread Jacob Champion
Hi all, When json_lex_string() hits certain types of invalid input, it calls pg_encoding_mblen_bounded(), which assumes that its input is null-terminated and calls strnlen(). But the JSON lexer is constructed with an explicit string length, and we don't ensure that the string is null-terminated in

Re: A problem about partitionwise join

2024-04-30 Thread Robert Haas
On Wed, Feb 21, 2024 at 6:25 AM Richard Guo wrote: > This patch was returned due to 'lack of interest'. However, upon > verification, it appears that the reported issue still exists, and the > proposed fix in the thread remains valid. Hence, resurrect this patch > after rebasing it on master. I

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2024-04-30 Thread Tom Lane
Alexander Korotkov writes: > I agree that storing char signedness might seem weird. But it appears > that we already store indexes that depend on char signedness. So, > it's effectively property of bits-on-disk even though it affects > indirectly. Then I see two options to make the picture cons

Re: TerminateOtherDBBackends code comments inconsistency.

2024-04-30 Thread Noah Misch
On Tue, Apr 30, 2024 at 09:10:52AM +0530, Amit Kapila wrote: > On Tue, Apr 30, 2024 at 2:58 AM Noah Misch wrote: > > On Mon, Apr 29, 2024 at 10:18:35AM +0530, Amit Kapila wrote: > > > On Mon, Apr 22, 2024 at 9:56 PM Noah Misch wrote: > > > > > 3a9b18b309 didn't change the docs of pg_terminate_bac

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2024-04-30 Thread Alexander Korotkov
On Tue, Apr 30, 2024 at 7:54 PM Tom Lane wrote: > Alexander Korotkov writes: > > Given this, should we try to do better with binary compatibility > > checks using ControlFileData? AFAICS they are supposed to check if > > the database cluster is binary compatible with the running > > architecture

Re: Avoid orphaned objects dependencies, take 3

2024-04-30 Thread Alexander Lakhin
Hi Bertrand, 25.04.2024 10:20, Bertrand Drouvot wrote: postgres=# CREATE FUNCTION f() RETURNS int LANGUAGE SQL RETURN f() + 1; ERROR: cache lookup failed for function 16400 This stuff does appear before we get a chance to call the new depLockAndCheckObject() function. I think this is what To

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2024-04-30 Thread Tom Lane
Alexander Korotkov writes: > Given this, should we try to do better with binary compatibility > checks using ControlFileData? AFAICS they are supposed to check if > the database cluster is binary compatible with the running > architecture. But it obviously allows incompatibilities. Perhaps. pg

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2024-04-30 Thread Alexander Korotkov
On Tue, Apr 23, 2024 at 5:57 PM Tom Lane wrote: > "Guo, Adam" writes: > > I would like to report an issue with the pg_trgm extension on > > cross-architecture replication scenarios. When an x86_64 standby > > server is replicating from an aarch64 primary server or vice versa, > > the gist_trgm_op

Re: SQL:2011 application time

2024-04-30 Thread Paul Jungwirth
On 4/30/24 09:24, Robert Haas wrote: Peter, could you have a look at http://postgr.es/m/47550967-260b-4180-9791-b224859fe...@illuminatedcomputing.com and express an opinion about whether each of those proposals are (a) good or bad ideas and (b) whether they need to be fixed for the current releas

Re: SQL:2011 application time

2024-04-30 Thread Robert Haas
On Fri, Apr 26, 2024 at 3:41 PM Paul Jungwirth wrote: > On 4/26/24 12:25, Robert Haas wrote: > > I think this thread should be added to the open items list. > > Thanks! I sent a request to pgsql-www to get edit permission. I didn't > realize there was a wiki page > tracking things like this. I ag

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2024-04-30 Thread Tom Lane
Masahiko Sawada writes: > On Tue, Apr 30, 2024 at 12:37 PM Tom Lane wrote: >> I think this will break existing indexes that are working fine. >> Yeah, it would have been better to avoid the difference, but >> it's too late now. > True. So it will be a PG18 item. How will it be any better in v18

Re: Support LIKE with nondeterministic collations

2024-04-30 Thread Daniel Verite
Peter Eisentraut wrote: > This patch adds support for using LIKE with nondeterministic > collations. So you can do things such as > > col LIKE 'foo%' COLLATE case_insensitive Nice! > The pattern is partitioned into substrings at wildcard characters > (so 'foo%bar' is partitioned i

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-30 Thread Daniel Gustafsson
> On 26 Apr 2024, at 15:04, Melanie Plageman wrote: > If this seems correct to you, are you okay with the rest of the fix > and test? We could close this open item once the patch is acceptable. From reading the discussion and the patch this seems like the right fix to me. Does the test added her

Re: CREATE TABLE/ProcessUtility hook behavior change

2024-04-30 Thread jian he
On Tue, Apr 30, 2024 at 4:35 PM David Steele wrote: > > On 4/30/24 12:57, jian he wrote: > > On Tue, Apr 30, 2024 at 10:26 AM David Steele wrote: > >> > >> Since bb766cde cannot be readily applied to older commits in master I'm > >> unable to continue bisecting to find the ALTER TABLE behavioral

Re: partitioning and identity column

2024-04-30 Thread Ashutosh Bapat
On Sun, Apr 28, 2024 at 12:29 PM Alexander Lakhin wrote: > 27.04.2024 18:00, Alexander Lakhin wrote: > > > > Please look also at another script, which produces the same error: > > I've discovered yet another problematic case: > CREATE TABLE tbl1 (a int GENERATED ALWAYS AS IDENTITY, b text) >

Re: pg_input_error_info doc 2 exampled crammed together

2024-04-30 Thread Michael Paquier
On Sun, Apr 28, 2024 at 10:07:49PM -0700, David G. Johnston wrote: > Agreed. The column names are self-explanatory if you’ve seen errors > before. The values are immaterial. Plus we don’t generally use > psql-specific features in our examples. Okay, I've just cleaned up that a bit with f6ab942f

Re: Removing unneeded self joins

2024-04-30 Thread Alexander Korotkov
On Tue, Apr 30, 2024 at 9:00 AM Alexander Lakhin wrote: > 23.10.2023 12:47, Alexander Korotkov wrote: > > I think this patch makes substantial improvement to query planning. > > It has received plenty of reviews. The code is currently in quite > > good shape. I didn't manage to find the cases wh

Re: Direct SSL connection with ALPN and HBA rules

2024-04-30 Thread Daniel Gustafsson
> On 29 Apr 2024, at 21:06, Heikki Linnakangas wrote: > Oh I was not aware sslrootcert=system works like that. That's a bit > surprising, none of the other ssl-related settings imply or require that SSL > is actually used. Did we intend to set a precedence for new settings with > that? It was

Re: using extended statistics to improve join estimates

2024-04-30 Thread Andy Fan
Hello Justin, Thanks for showing interest on this! > On Sun, Apr 28, 2024 at 10:07:01AM +0800, Andy Fan wrote: >> 's/estimiatedcluases/estimatedclauses/' typo error in the >> commit message is not fixed since I have to regenerate all the commits > > Maybe you know this, but some of these patche

Re: [PATCH] Fix bug when calling strncmp in check_authmethod_valid

2024-04-30 Thread Daniel Gustafsson
> On 30 Apr 2024, at 04:41, Jingxian Li wrote: > Attached is a patch that fixes bug when calling strncmp function, in > which case the third argument (authmethod - strchr(authmethod, ' ')) > may be negative, which is not as expected.. The calculation is indeed incorrect, but the lack of compla

Re: Removing unneeded self joins

2024-04-30 Thread Alexander Korotkov
Hi, Alexander! On Tue, Apr 30, 2024 at 9:00 AM Alexander Lakhin wrote: > 23.10.2023 12:47, Alexander Korotkov wrote: > > I think this patch makes substantial improvement to query planning. > > It has received plenty of reviews. The code is currently in quite > > good shape. I didn't manage to f

Re: speed up a logical replica setup

2024-04-30 Thread Amit Kapila
On Tue, Apr 30, 2024 at 12:04 PM Amit Kapila wrote: > > On Mon, Apr 29, 2024 at 5:28 PM Amit Kapila wrote: > > > > On Mon, Apr 29, 2024 at 5:23 PM Euler Taveira wrote: > > > > I was trying to test this utility when 'sync_replication_slots' is on > and it gets in an ERROR loop [1] and never finis

Re: CREATE TABLE/ProcessUtility hook behavior change

2024-04-30 Thread David Steele
On 4/30/24 12:57, jian he wrote: On Tue, Apr 30, 2024 at 10:26 AM David Steele wrote: Since bb766cde cannot be readily applied to older commits in master I'm unable to continue bisecting to find the ALTER TABLE behavioral change. This seems to leave me with manual code inspection and there ha

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2024-04-30 Thread Masahiko Sawada
On Tue, Apr 30, 2024 at 12:37 PM Tom Lane wrote: > > Masahiko Sawada writes: > > On Tue, Apr 23, 2024 at 11:57 PM Tom Lane wrote: > >> Reject as not a bug. Discourage people from thinking that physical > >> replication will work across architectures. > > > While cross-arch physical replication