Re: Identify huge pages accessibility using madvise

2024-09-25 Thread Dmitry Dolgov
> On Thu, Sep 26, 2024 at 07:57:12AM GMT, Gabriele Bartolini wrote: > Hi Dmitry, > > I've been attempting to replicate this issue directly in Kubernetes, but I > haven't been successful so far. I've been using EKS nodes, and it seems > that they all run cgroup v2 now. Do you have anything that coul

Re: long-standing data loss bug in initial sync of logical replication

2024-09-25 Thread Shlok Kyal
> In the v7 patch, I am looping through the reorder buffer of the > current committed transaction and storing all invalidation messages in > a list. Then I am distributing those invalidations. > But I found that for a transaction we already store all the > invalidation messages (see [1]). So we don

Re: Using per-transaction memory contexts for storing decoded tuples

2024-09-25 Thread Masahiko Sawada
On Sun, Sep 22, 2024 at 9:29 PM Amit Kapila wrote: > > On Sun, Sep 22, 2024 at 11:27 AM David Rowley wrote: > > > > On Fri, 20 Sept 2024 at 17:46, Amit Kapila wrote: > > > > > > On Fri, Sep 20, 2024 at 5:13 AM David Rowley wrote: > > > > In general, it's a bit annoying to have to code around th

Modify comment in /postgres/src/bin/pg_walsummary/nls.mk

2024-09-25 Thread btnakamurakoukil
Hi hackers, I noticed incorrect comment in /postgres/src/bin/pg_walsummary/nls.mk. The part that should be "pg_walsummary" is "pg_combinebackup", patch attached. Regards, Koki Nakamura From 70959d10410224afe2a30d74f1b0b17221d05c57 Mon Sep 17 00:00:00 2001 From: KokiNaka Date: Thu, 26 Sep 202

Re: Identify huge pages accessibility using madvise

2024-09-25 Thread Gabriele Bartolini
Hi Dmitry, I've been attempting to replicate this issue directly in Kubernetes, but I haven't been successful so far. I've been using EKS nodes, and it seems that they all run cgroup v2 now. Do you have anything that could help me get started on this more quickly? Thanks, Gabriele On Sat, 13 Apr

Re: meson and check-tests

2024-09-25 Thread Ashutosh Bapat
On Wed, Sep 25, 2024 at 8:24 PM Nazir Bilal Yavuz wrote: > > Hi, > > Thanks for looking into this! > > On Wed, 25 Sept 2024 at 13:27, Ashutosh Bapat > wrote: > > > > On Mon, Sep 23, 2024 at 2:16 PM Nazir Bilal Yavuz > > wrote: > > > > > > On Sat, 21 Sept 2024 at 09:01, jian he > > > wrote: >

Re: Logical Replication of sequences

2024-09-25 Thread shveta malik
On Fri, Sep 20, 2024 at 9:36 AM vignesh C wrote: > > On Wed, 21 Aug 2024 at 11:54, vignesh C wrote: > > > > On Wed, 21 Aug 2024 at 08:33, Peter Smith wrote: > > > > > > Hi Vignesh, Here are my only review comments for the latest patch set. > > > > Thanks, these issues have been addressed in the

Re: CREATE INDEX regression in 17 RC1 or expected behavior?

2024-09-25 Thread Yugo NAGATA
On Thu, 26 Sep 2024 13:27:54 +0930 Tom Dunstan wrote: > On Thu, 26 Sept 2024 at 13:21, Yugo Nagata wrote: > > > By the way, this is not mentioned in CREATE MATERIALIZED VIEW > > documentation, although > > we can find in REFRESH MATERIALIZED VIEW doc. So, I sent the doc patch in > > [1], > > an

Re: First draft of PG 17 release notes

2024-09-25 Thread Yugo Nagata
On Thu, 9 May 2024 00:03:50 -0400 Bruce Momjian wrote: > I have committed the first draft of the PG 17 release notes; you can > see the results here: I propose to improve the following description in "Migration to Version 17" section by adding CREATE INDEX and CREATE MATERIALIZED VIEW into the

Re: installcheck-world concurrency issues

2024-09-25 Thread Michael Paquier
On Tue, Sep 24, 2024 at 11:24:46AM +0900, Michael Paquier wrote: > Corey (added in CC.), has noticed that the issue fixed by c3315a7 in > 16~ for advisory locks is not complicated to reach, leading to > failures in some of our automated internal stuff. A cherry-pick of > c3315a7 works cleanly acro

Re: CREATE INDEX regression in 17 RC1 or expected behavior?

2024-09-25 Thread Yugo Nagata
On Wed, 25 Sep 2024 22:16:06 -0500 Nathan Bossart wrote: > On Thu, Sep 26, 2024 at 12:22:32PM +0930, Tom Dunstan wrote: > > Reporting in case this is unexpected. At the very least if a function used > > in an index must now always find other functions using an explicit path, it > > seems like thi

Re: Add contrib/pg_logicalsnapinspect

2024-09-25 Thread shveta malik
On Wed, Sep 25, 2024 at 11:01 PM Bertrand Drouvot wrote: > > Hi, > > On Wed, Sep 25, 2024 at 11:23:17AM +0530, shveta malik wrote: > > + OUT catchange_xip xid[] > > > > One question, what is xid datatype, is it too int8? Sorry, could not > > find the correct doc. > > I think that we can get the an

Re: CREATE INDEX regression in 17 RC1 or expected behavior?

2024-09-25 Thread Nathan Bossart
On Thu, Sep 26, 2024 at 12:22:32PM +0930, Tom Dunstan wrote: > Reporting in case this is unexpected. At the very least if a function used > in an index must now always find other functions using an explicit path, it > seems like this should be documented and noted in the release notes. The first c

CREATE INDEX regression in 17 RC1 or expected behavior?

2024-09-25 Thread Tom Dunstan
Hi all While testing out 17 RC1 I found that a construct that previously worked has now stopped working: CREATE OR REPLACE FUNCTION index_truncate(src TEXT) RETURNS TEXT AS $$ SELECT LOWER(LEFT(src, 100)); $$ LANGUAGE SQL; CREATE OR REPLACE FUNCTION join_for_index(TEXT []) RETURNS TEXT LANGU

Re: query_id, pg_stat_activity, extended query protocol

2024-09-25 Thread Michael Paquier
On Wed, Sep 25, 2024 at 05:00:00PM +0300, Alexander Lakhin wrote: > Please look at the script, which triggers Assert added by 24f520594: > (assuming shared_preload_libraries=pg_stat_statements) Or just compute_query_id = on. > SELECT repeat('x', 100) INTO t FROM generate_series(1, 10); > CREA

Re: Add contrib/pg_logicalsnapinspect

2024-09-25 Thread Masahiko Sawada
Hi, On Wed, Sep 25, 2024 at 10:29 AM Bertrand Drouvot wrote: > > Hi, > > On Wed, Sep 25, 2024 at 04:04:43PM +0200, Peter Eisentraut wrote: > > Is there a reason for this elaborate error handling: > > Thanks for looking at it! > > > + fd = OpenTransientFile(path, O_RDONLY | PG_BINARY); > > + >

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2024-09-25 Thread Masahiko Sawada
On Wed, Sep 18, 2024 at 6:46 PM Noah Misch wrote: > > On Tue, Sep 17, 2024 at 09:43:41PM -0700, Masahiko Sawada wrote: > > On Mon, Sep 16, 2024 at 9:24 AM Noah Misch wrote: > > > On Thu, Sep 12, 2024 at 03:42:48PM -0700, Masahiko Sawada wrote: > > > > On Tue, Sep 10, 2024 at 3:05 PM Noah Misch w

Re: Docs pg_restore: Shouldn't there be a note about -n ?

2024-09-25 Thread Florents Tselai
On Sat, Sep 21, 2024 at 9:48 PM Florents Tselai wrote: > > > > On 21 Sep 2024, at 9:22 PM, Tom Lane wrote: > > > > Florents Tselai writes: > >> Ah, swapped them by mistake on the previous email: > >> They're both available in the pg_dump and note on -n missing in > pg_restore. > >> The questio

Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part

2024-09-25 Thread Tom Lane
Florents Tselai writes: > This patch is a follow-up and generalization to [0]. > It adds the following jsonpath methods: lower, upper, initcap, l/r/btrim, > replace, split_part. How are you going to deal with the fact that this makes jsonpath operations not guaranteed immutable? (See commit cb5

Re: not null constraints, again

2024-09-25 Thread Alvaro Herrera
On 2024-Sep-25, jian he wrote: > copy from src/test/regress/sql/index_including.sql > -- Unique index and unique constraint > CREATE TABLE tbl_include_unique1 (c1 int, c2 int, c3 int, c4 box); > INSERT INTO tbl_include_unique1 SELECT x, 2*x, 3*x, box('4,4,4,4') > FROM generate_series(1,10) AS x; >

Re: pg_ctl/miscinit: print "MyStartTime" as a long long instead of long to avoid 2038 problem.

2024-09-25 Thread Nathan Bossart
On Wed, Sep 25, 2024 at 03:17:45PM +, Max Johnson wrote: > I have amended my patch to reflect the changes that were discussed and > have verified on my system that it works the same as before. I have also > fixed a typo and changed the name of the patch to more accurately reflect > what it does

Re: pg_verifybackup: TAR format backup verification

2024-09-25 Thread Robert Haas
On Thu, Sep 12, 2024 at 7:05 AM Amul Sul wrote: > The updated version attached. Thank you for the review ! I have spent a bunch of time on this and have made numerous revisions. I hope to commit the result, aftering seeing what you and the buildfarm think (and anyone else who wishes to offer an o

PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part

2024-09-25 Thread Florents Tselai
Hello hackers, This patch is a follow-up and generalization to [0]. It adds the following jsonpath methods: lower, upper, initcap, l/r/btrim, replace, split_part. It makes jsonpath able to support expressions like these: select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().

Re: [PATCH] Support Int64 GUCs

2024-09-25 Thread Alexander Korotkov
Hi, Tom! On Wed, Sep 25, 2024 at 6:08 PM Tom Lane wrote: > FWIW, I agree with the upthread opinions that we shouldn't do this > (invent int64 GUCs). I don't think we need the added code bloat > and risk of breaking user code that isn't expecting this new GUC > type. We invented the notion of GU

Re: Add contrib/pg_logicalsnapinspect

2024-09-25 Thread Bertrand Drouvot
Hi, On Wed, Sep 25, 2024 at 04:04:43PM +0200, Peter Eisentraut wrote: > Is there a reason for this elaborate error handling: Thanks for looking at it! > + fd = OpenTransientFile(path, O_RDONLY | PG_BINARY); > + > + if (fd < 0 && errno == ENOENT) > + ereport(ERROR, > +

Re: src/backend/optimizer/util/plancat.c -> Is this correct English

2024-09-25 Thread Tom Lane
"Daniel Westermann (DWE)" writes: > That's what I would have expected. But, as said, maybe this only sounds > strange to me. "Need not" is perfectly good English, although perhaps it has a faintly archaic whiff to it. regards, tom lane

Re: Add contrib/pg_logicalsnapinspect

2024-09-25 Thread Bertrand Drouvot
Hi, On Wed, Sep 25, 2024 at 11:23:17AM +0530, shveta malik wrote: > + OUT catchange_xip xid[] > > One question, what is xid datatype, is it too int8? Sorry, could not > find the correct doc. I think that we can get the answer from pg_type: postgres=# select typname,typlen from pg_type where typ

Re: src/backend/optimizer/util/plancat.c -> Is this correct English

2024-09-25 Thread Daniel Westermann (DWE)
>>I think it's fine. It could also be phrased like this: > >We do not need to lock the relation since... That's what I would have expected. But, as said, maybe this only sounds strange to me. Regards Daniel

Re: src/backend/optimizer/util/plancat.c -> Is this correct English

2024-09-25 Thread Nathan Bossart
On Wed, Sep 25, 2024 at 04:52:47PM +, Daniel Westermann (DWE) wrote: > just came across this: > > src/backend/optimizer/util/plancat.c -> Is this correct English? > -> We need not lock the relation since it was already locked ... > > I am not a native speaker, but this sounds strange. I thi

src/backend/optimizer/util/plancat.c -> Is this correct English

2024-09-25 Thread Daniel Westermann (DWE)
Hi, just came across this: src/backend/optimizer/util/plancat.c -> Is this correct English? -> We need not lock the relation since it was already locked ... I am not a native speaker, but this sounds strange. Regards Daniel?

Re: BUG #18097: Immutable expression not allowed in generated at

2024-09-25 Thread Tom Lane
Adrien Nayrat writes: > I see. So I understand we were lucky it worked before the commit added > the check of volatility in generated column ? Pretty much. There are other cases that could trigger expansion of such a function before the restore is complete. It is unfortunate that this bit you

Re: BUG #18097: Immutable expression not allowed in generated at

2024-09-25 Thread Adrien Nayrat
On 9/25/24 4:41 PM, Tom Lane wrote: Adrien Nayrat writes: A customer encountered an issue while restoring a dump of its database after applying 15.6 minor version. It seems due to this fix : Fix function volatility checking for GENERATED and DEFAULT expressions (Tom Lane) I don't believe thi

Re: Possible null pointer dereference in afterTriggerAddEvent()

2024-09-25 Thread Alvaro Herrera
On 2024-Sep-24, Alvaro Herrera wrote: > On 2024-Sep-24, Alexander Kuznetsov wrote: > > > is there anything else we can help with or discuss in order to apply this > > fix? > > I don't think so, it seems a no-brainer to me and there are no > objections. I'll get it pushed tomorrow. OK, done.

Re: XMLSerialize: version and explicit XML declaration

2024-09-25 Thread Tom Lane
Jim Jones writes: > Is there any validation mechanism for VERSION literal>? AFAICS, all we do with an embedded XML version string is pass it to libxml2's xmlNewDoc(), which is the authority on whether it means anything. I'd be inclined to do the same here. regards, tom

Re: [PATCH] Support Int64 GUCs

2024-09-25 Thread Tom Lane
FWIW, I agree with the upthread opinions that we shouldn't do this (invent int64 GUCs). I don't think we need the added code bloat and risk of breaking user code that isn't expecting this new GUC type. We invented the notion of GUC units in part to ensure that int32 GUCs could be adapted to handl

Re: meson and check-tests

2024-09-25 Thread Nazir Bilal Yavuz
Hi, Thanks for looking into this! On Wed, 25 Sept 2024 at 13:27, Ashutosh Bapat wrote: > > On Mon, Sep 23, 2024 at 2:16 PM Nazir Bilal Yavuz wrote: > > > > On Sat, 21 Sept 2024 at 09:01, jian he wrote: > > > > > > so > > > TESTS="copy jsonb stats_ext" meson test --suite regress > > > will fail

Re: [PATCH] Support Int64 GUCs

2024-09-25 Thread Aleksander Alekseev
Hi, > ``` > postgres=# alter system set autovacuum_vacuum_threshold to 2_147_483_648; > ERROR: invalid value for parameter "autovacuum_vacuum_threshold": > "2_147_483_648" > postgres=# alter system set autovacuum_vacuum_threshold to 2_147_483_647; > ALTER SYSTEM > ``` > > IIRC, the lexer only su

Re: BUG #18097: Immutable expression not allowed in generated at

2024-09-25 Thread Tom Lane
Adrien Nayrat writes: > A customer encountered an issue while restoring a dump of its database > after applying 15.6 minor version. > It seems due to this fix : >>> Fix function volatility checking for GENERATED and DEFAULT >>> expressions (Tom Lane) I don't believe this example has anything to

Re: [PATCH] Support Int64 GUCs

2024-09-25 Thread Li Japin
On Sep 25, 2024, at 19:03, Aleksander Alekseev wrote: Hi, I found the autovacuum_vacuum_threshold, autovacuum_vacuum_insert_threshold and autovacuum_analyze_threshold is change to int64 for relation option, however the GUCs are still integers. ``` postgres=# select * from pg_settings where nam

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-09-25 Thread Jehan-Guillaume de Rorthais
On Wed, 25 Sep 2024 14:42:40 +0200 Jehan-Guillaume de Rorthais wrote: > On Thu, 5 Sep 2024 00:57:28 +0200 > Jehan-Guillaume de Rorthais wrote: […] > > > > Please, find in attachment a patch implementing this idea. > > Please, find in attachment a set of patch based on the previous one. Plea

Re: Add contrib/pg_logicalsnapinspect

2024-09-25 Thread Peter Eisentraut
Is there a reason for this elaborate error handling: + fd = OpenTransientFile(path, O_RDONLY | PG_BINARY); + + if (fd < 0 && errno == ENOENT) + ereport(ERROR, + errmsg("file \"%s\" does not exist", path)); + else if (fd < 0) +

Re: query_id, pg_stat_activity, extended query protocol

2024-09-25 Thread Alexander Lakhin
Hello Michael and Sami, 18.09.2024 08:46, Michael Paquier wrote: So, I have applied 0001 down to 14, followed by 0002 on HEAD. Please look at the script, which triggers Assert added by 24f520594: (assuming shared_preload_libraries=pg_stat_statements) SELECT repeat('x', 100) INTO t FROM genera

Re: Add support to TLS 1.3 cipher suites and curves lists

2024-09-25 Thread Daniel Gustafsson
> On 18 Sep 2024, at 22:48, Jacob Champion > wrote: > On Mon, Sep 9, 2024 at 5:00 AM Daniel Gustafsson wrote: >> The attached version also has a new 0001 which bumps the minimum required >> OpenSSL version to 1.1.1 (from 1.1.0) since this patchset requires API's only >> present in 1.1.1 and onw

XMLSerialize: version and explicit XML declaration

2024-09-25 Thread Jim Jones
Hi, I'm working on the flags VERSION (X076), INCLUDING XMLDECLARATION, and EXCLUDING XMLDECLARATION (X078) for XMLSerialize, and I have a question for SQL/XML experts on the list. Is there any validation mechanism for VERSION ? The SQL/XML spec says "The immediately contained in shall be '1.0'

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-09-25 Thread Jehan-Guillaume de Rorthais
On Thu, 5 Sep 2024 00:57:28 +0200 Jehan-Guillaume de Rorthais wrote: > On Mon, 2 Sep 2024 23:01:47 +0200 > Jehan-Guillaume de Rorthais wrote: > > […] > > > My proposal was to clean everything related to the old FK and use some > > existing code path to create a fresh and cleaner one. This

Re: Documentation to upgrade logical replication cluster

2024-09-25 Thread vignesh C
On Wed, 25 Sept 2024 at 14:06, Amit Kapila wrote: > > On Tue, Sep 24, 2024 at 9:42 PM vignesh C wrote: > > > > Reordered the docs to enable the subscription before creating the > > table. For bi-directional replication, a publication refresh is > > necessary on both nodes: a) First, refresh the p

Re: Why mention to Oracle ?

2024-09-25 Thread Greg Sabino Mullane
> And if we create a page like > https://www.postgresql.org/about/featurematrix/ > But instead of Postgres versions we have other vendors. > This sounds like something that would fit well on the Postgres wiki: https://wiki.postgresql.org/ Cheers, Greg

Re: not null constraints, again

2024-09-25 Thread jian he
in ATExecSetNotNull /* * If we find an appropriate constraint, we're almost done, but just * need to change some properties on it: if we're recursing, increment * coninhcount; if not, set conislocal if not already set. */ if (recursing) {

Re: BUG #18097: Immutable expression not allowed in generated at

2024-09-25 Thread Adrien Nayrat
Hello, A customer encountered an issue while restoring a dump of its database after applying 15.6 minor version. It seems due to this fix : > Fix function volatility checking for GENERATED and DEFAULT expressions (Tom Lane) > These places could fail to detect insertion of a volatile function

Re: [PATCH] Support Int64 GUCs

2024-09-25 Thread Aleksander Alekseev
Hi, > I found the autovacuum_vacuum_threshold, autovacuum_vacuum_insert_threshold > and autovacuum_analyze_threshold is change to int64 for relation option, > however the GUCs are still integers. > > ``` > postgres=# select * from pg_settings where name = > 'autovacuum_vacuum_threshold' \gx > -[

Re: meson and check-tests

2024-09-25 Thread Ashutosh Bapat
On Mon, Sep 23, 2024 at 2:16 PM Nazir Bilal Yavuz wrote: > > Hi, > > On Sat, 21 Sept 2024 at 09:01, jian he wrote: > > > > hi. Thanks for your work! > > Thank you for looking into this! > > > I do find some issues. > > > > > > TESTS="copy jsonb jsonb" meson test --suite regress > > one will fail.

RE: Clock-skew management in logical replication

2024-09-25 Thread Hayato Kuroda (Fujitsu)
Dear hackers, > Though this provides a way for users to control values required for > conflict resolution, I prefer a simple approach at least for the first > version which is to document that users should ensure time > synchronization via NTP. Even Oracle mentions the same in their docs I resear

Re: Add support to TLS 1.3 cipher suites and curves lists

2024-09-25 Thread Peter Eisentraut
On 18.09.24 22:48, Jacob Champion wrote: +#ssl_ciphers = 'HIGH:MEDIUM:+3DES:!aNULL' # allowed TLSv1.2 ciphers +#ssl_cipher_suites = ''# allowed TLSv1.3 cipher suites, blank for default After marinating on this a bit... I think the naming may result in some "who's on first" miscommunications

Re: Documentation to upgrade logical replication cluster

2024-09-25 Thread Amit Kapila
On Tue, Sep 24, 2024 at 9:42 PM vignesh C wrote: > > Reordered the docs to enable the subscription before creating the > table. For bi-directional replication, a publication refresh is > necessary on both nodes: a) First, refresh the publication on the old > version server to set the newly added t

Re: not null constraints, again

2024-09-25 Thread jian he
copy from src/test/regress/sql/index_including.sql -- Unique index and unique constraint CREATE TABLE tbl_include_unique1 (c1 int, c2 int, c3 int, c4 box); INSERT INTO tbl_include_unique1 SELECT x, 2*x, 3*x, box('4,4,4,4') FROM generate_series(1,10) AS x; CREATE UNIQUE INDEX tbl_include_unique1_idx

Re: AIO writes vs hint bits vs checksums

2024-09-25 Thread Antonin Houska
Antonin Houska wrote: > Andres Freund wrote: > > > What I'd instead like to propose is to implement the right to set hint bits > > as > > a bit in each buffer's state, similar to BM_IO_IN_PROGRESS. Tentatively I > > named this BM_SETTING_HINTS. It's only allowed to set BM_SETTING_HINTS when >

[ecpg bug]: can not use single '*' in multi-line comment after c preprocessor directives

2024-09-25 Thread Winter Loo
Hi hackers, The following code fails to pass the ecpg compilation, although it is accepted by the gcc compiler. ``` #if ABC /* this is a multi-line * comment including single star character */ int a = 1; #endif ``` The issue arises from the first '*' in the second line. Upon it

Re: Eager aggregation, take 3

2024-09-25 Thread Richard Guo
On Fri, Sep 13, 2024 at 3:48 PM Tender Wang wrote: > Since MERGE/SPLIT partition has been reverted, the tests *partition_merge* > and *partition_split* should be removed > from parallel_schedule. After doing the above, the 0002 patch can be applied. Yeah, that's what I need to do. Thanks Ric

Re: Opinion poll: Sending an automated email to a thread when it gets added to the commitfest

2024-09-25 Thread Jelte Fennema-Nio
On Tue, 24 Sept 2024 at 23:54, Jelte Fennema-Nio wrote: > A bunch of other improvements also got deployed: > - Improved homepage[1] (now with useful and bookmarkable links at the top) > - More links on the cf entry page[2] (cfbot results, github diff, and [1]: https://commitfest.postgresql.org/ [

Re: AIO writes vs hint bits vs checksums

2024-09-25 Thread Antonin Houska
Andres Freund wrote: > What I'd instead like to propose is to implement the right to set hint bits as > a bit in each buffer's state, similar to BM_IO_IN_PROGRESS. Tentatively I > named this BM_SETTING_HINTS. It's only allowed to set BM_SETTING_HINTS when > BM_IO_IN_PROGRESS isn't already set and

Re: Eager aggregation, take 3

2024-09-25 Thread Richard Guo
On Wed, Sep 11, 2024 at 10:52 AM Tender Wang wrote: > 1. In make_one_rel(), we have the below codes: > /* > * Build grouped base relations for each base rel if possible. > */ > setup_base_grouped_rels(root); > > As far as I know, each base rel only has one grouped base relation, if > possible. >