doubt about FullTransactionIdAdvance()

2022-10-21 Thread Zhang Mingli
Hi, hackers I don't quite understand FullTransactionIdAdvance(), correct me if I’m wrong. /*  * Advance a FullTransactionId variable, stepping over xids that would appear  * to be special only when viewed as 32bit XIDs.  */ static inline void FullTransactionIdAdvance(FullTransactionId *dest) {

Re: Add 64-bit XIDs into PostgreSQL 15

2022-10-21 Thread Zhang Mingli
Hi, On Oct 22, 2022, 00:09 +0800, Maxim Orlov , wrote: > > > > Done! Thanks! Here is the rebased version. > > > > This version has bug fix for multixact replication. Previous versions of > > the patch set does not write pd_multi_base in WAL. Thus, this field was set > > to 0 upon WAL reply on

Re: Collation version tracking for macOS

2022-10-21 Thread Thomas Munro
On Sat, Oct 22, 2022 at 10:24 AM Thomas Munro wrote: > ... But it > doesn't provide a way for me to create a new database that uses 63 on > purpose when I know what I'm doing. There are various reasons I might > want to do that. Thinking some more about this, I guess that could be addressed by

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2022-10-21 Thread Peter Geoghegan
On Thu, Oct 20, 2022 at 11:52 AM Peter Geoghegan wrote: > Leaving my patch series aside, I still don't think that it makes sense > to make it impossible to auto-cancel antiwraparound autovacuums, > across the board, regardless of the underlying table age. One small thought on the

Re: Missing update of all_hasnulls in BRIN opclasses

2022-10-21 Thread Tomas Vondra
On 10/21/22 18:44, Tomas Vondra wrote: > > ... > >> Apart from that, this patch looks good. >> Sadly, I don't think we can fix it like this :-( The problem is that all ranges start with all_nulls=true, because the new range gets initialized by brin_memtuple_initialize() like that. But this

Re: Pluggable toaster

2022-10-21 Thread Nikita Malakhov
Hi! Aleksander, we have had this in mind while developing this feature, and have checked it. Just a slight modification is needed to make it work with Pluggable Storage (Access Methods) API. On Fri, Oct 21, 2022 at 4:01 PM Aleksander Alekseev < aleksan...@timescale.com> wrote: > Hi Nikita, > >

Re: Collation version tracking for macOS

2022-10-21 Thread Thomas Munro
Hi, Here is a rebase of this experimental patch. I think the basic mechanics are promising, but we haven't agreed on a UX. I hope we can figure this out. Restating the choice made in this branch of the experiment: Here I try to be just like DB2 (if I understood its manual correctly). In DB2,

Multiple grouping set specs referencing duplicate alias

2022-10-21 Thread David Kimura
Hi all! I think I may have stumbled across a case of wrong results on HEAD (same through version 9.6, though interestingly 9.5 produces different results altogether). test=# SELECT i AS ai1, i AS ai2 FROM generate_series(1,3)i GROUP BY ai2, ROLLUP(ai1) ORDER BY ai1, ai2; ai1 | ai2 -+-

Re: refactor ownercheck and aclcheck functions

2022-10-21 Thread Peter Eisentraut
On 20.10.22 01:24, Corey Huinker wrote: I'd be inclined to remove the highly used ones as well. That way the codebase would have more examples of object_ownercheck() for readers to see. Seeing the existence of pg_FOO_ownercheck implies that a pg_BAR_ownercheck might exist, and if BAR is

patch suggestion: Fix citext_utf8 test's "Turkish I" with ICU collation provider

2022-10-21 Thread Anton Voloshin
Hello, hackers. In current master, as well as in REL_15_STABLE, installcheck in contrib/citext fails in most locales, if we use ICU as a locale provider: $ rm -fr data; initdb --locale-provider icu --icu-locale en-US -D data && pg_ctl -D data -l logfile start && make -C contrib/citext

Re: Missing update of all_hasnulls in BRIN opclasses

2022-10-21 Thread Tomas Vondra
On 10/21/22 17:50, Matthias van de Meent wrote: > On Fri, 21 Oct 2022 at 17:24, Tomas Vondra > wrote: >> >> Hi, >> >> While working on some BRIN code, I discovered a bug in handling NULL >> values - when inserting a non-NULL value into a NULL-only range, we >> reset the all_nulls flag but

Re: thinko in basic_archive.c

2022-10-21 Thread Bharath Rupireddy
On Fri, Oct 21, 2022 at 10:43 AM Kyotaro Horiguchi wrote: > > Thanks, but we don't need to wipe out the all bytes. Just putting \0 > at the beginning of the buffer is sufficient. Nah, that's not a clean way IMO. > And the Memset() at the > beginning of basic_archive_file_internal is not needed

Re: Missing update of all_hasnulls in BRIN opclasses

2022-10-21 Thread Matthias van de Meent
On Fri, 21 Oct 2022 at 17:24, Tomas Vondra wrote: > > Hi, > > While working on some BRIN code, I discovered a bug in handling NULL > values - when inserting a non-NULL value into a NULL-only range, we > reset the all_nulls flag but don't update the has_nulls flag. And > because of that we then

Re: Avoid memory leaks during base backups

2022-10-21 Thread Bharath Rupireddy
On Fri, Oct 21, 2022 at 11:58 AM Michael Paquier wrote: > > To be exact, it seems to me that tablespace_map and backup_state > should be reset before deleting backupcontext, but the reset of > backupcontext should happen after the fact. > > + backup_state = NULL; > tablespace_map =

Missing update of all_hasnulls in BRIN opclasses

2022-10-21 Thread Tomas Vondra
Hi, While working on some BRIN code, I discovered a bug in handling NULL values - when inserting a non-NULL value into a NULL-only range, we reset the all_nulls flag but don't update the has_nulls flag. And because of that we then fail to return the range for IS NULL ranges. Reproducing this is

Re: Crash after a call to pg_backup_start()

2022-10-21 Thread Bharath Rupireddy
On Fri, Oct 21, 2022 at 7:06 PM Michael Paquier wrote: > > On Fri, Oct 21, 2022 at 05:53:25PM +0800, Richard Guo wrote: > > Yeah, the two conditions could be both false. How about we update the > > comment a bit to emphasize this? Such as > > > > /* At most one of these conditions can be true

Re: ICU for global collation

2022-10-21 Thread Marina Polyakova
Hello! I discovered an interesting behaviour during installcheck runs when the cluster was initialized with ICU locale provider: $ initdb --locale-provider icu --icu-locale en-US -D data && pg_ctl -D data -l logfile start 1) The ECPG tests fail because they use the SQL_ASCII encoding [1],

Re: Avoid memory leaks during base backups

2022-10-21 Thread Robert Haas
On Fri, Oct 21, 2022 at 2:18 AM Michael Paquier wrote: > AFAIK, one of the callbacks associated to a memory context could > fail, see comments before MemoryContextCallResetCallbacks() in > MemoryContextDelete(). I agree that it should not matter here, but I > think that it is better to reset the

Re: parse partition strategy string in gram.y

2022-10-21 Thread Japin Li
On Fri, 21 Oct 2022 at 20:34, Justin Pryzby wrote: > On Fri, Oct 21, 2022 at 06:22:44PM +0800, Japin Li wrote: >> Is there any way to get the regression tests diffs from Cirrus CI? >> I did not find the diffs in [1]. >> >> [1] https://cirrus-ci.com/build/4721735111540736 > > They're called

Re: Crash after a call to pg_backup_start()

2022-10-21 Thread Michael Paquier
On Fri, Oct 21, 2022 at 05:53:25PM +0800, Richard Guo wrote: > Yeah, the two conditions could be both false. How about we update the > comment a bit to emphasize this? Such as > > /* At most one of these conditions can be true */ > or > /* These conditions can not be both true */ If you

Re: cross-platform pg_basebackup

2022-10-21 Thread Robert Haas
On Fri, Oct 21, 2022 at 4:14 AM davinder singh wrote: > Hi, > Patch v2 looks good to me, I have tested it, and pg_basebackup works fine > across the platforms (Windows to Linux and Linux to Windows). > Syntax used for testing > $ pg_basebackup -h remote_server_ip -p 5432 -U user_name -D

Re: Pluggable toaster

2022-10-21 Thread Aleksander Alekseev
Hi Nikita, > Here's rebased patch: > v23-0001-toaster-interface.patch - Pluggable TOAST API interface with > reference (original) TOAST mechanics - new API is introduced but > reference TOAST is still left unchanged; > v23-0002-toaster-default.patch - Default TOAST mechanics is re-implemented >

Re: parse partition strategy string in gram.y

2022-10-21 Thread Justin Pryzby
On Fri, Oct 21, 2022 at 06:22:44PM +0800, Japin Li wrote: > Is there any way to get the regression tests diffs from Cirrus CI? > I did not find the diffs in [1]. > > [1] https://cirrus-ci.com/build/4721735111540736 They're called "main". I'm planning on submitting a patch to rename it to

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2022-10-21 Thread Önder Kalacı
Hi Shi yu, all > In execReplication.c: > > + TypeCacheEntry **eq = NULL; /* only used when the index is not > unique */ > > Maybe the comment here should be changed. Now it is used when the index is > not > primary key or replica identity index. > > makes sense, updated > 2. > +# wait

Re: Patch proposal: make use of regular expressions for the username in pg_hba.conf

2022-10-21 Thread Drouvot, Bertrand
Hi, On 10/21/22 2:58 AM, Michael Paquier wrote: On Wed, Oct 19, 2022 at 10:45:44AM +0200, Drouvot, Bertrand wrote: Please find attached v1-0001-regex-handling-for-db-and-roles-in-hba.patch to implement regexes for databases and roles in hba. It does also contain new regexes related TAP tests

Re: START_REPLICATION SLOT causing a crash in an assert build

2022-10-21 Thread Masahiko Sawada
On Thu, Oct 20, 2022 at 6:54 AM Andres Freund wrote: > > Hi, > > On 2022-10-13 15:57:28 +0900, Masahiko Sawada wrote: > > I've attached an updated patch. I've added the common function to > > start pg_recvlogical and wait for it to become active. Please review > > it. > > > +# Start

Re: parse partition strategy string in gram.y

2022-10-21 Thread Alvaro Herrera
headerscheck fail, fixed here. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ #error "Operator lives in the wrong universe" ("Use of cookies in real-time system development", M. Gleixner, M. Mc Guire) >From c434020fc07616cdd13017135819083186d33256 Mon Sep 17

Re: parse partition strategy string in gram.y

2022-10-21 Thread Alvaro Herrera
On 2022-Oct-21, Japin Li wrote: > Is there any way to get the regression tests diffs from Cirrus CI? > I did not find the diffs in [1]. I think they should be somewhere in the artifacts, but I'm not sure. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "La

Re: parse partition strategy string in gram.y

2022-10-21 Thread Japin Li
On Fri, 21 Oct 2022 at 18:12, Alvaro Herrera wrote: > On 2022-Oct-21, Japin Li wrote: > >> Does there an error about forget the LIST partition? > > Of course. > https://cirrus-ci.com/build/4721735111540736 > > This is what you get for moving cases around at the last minute ... > Is there any

Re: parse partition strategy string in gram.y

2022-10-21 Thread Alvaro Herrera
On 2022-Oct-21, Japin Li wrote: > Does there an error about forget the LIST partition? Of course. https://cirrus-ci.com/build/4721735111540736 This is what you get for moving cases around at the last minute ... Fixed, thanks. -- Álvaro Herrera PostgreSQL Developer —

Re: parse partition strategy string in gram.y

2022-10-21 Thread Japin Li
On Fri, 21 Oct 2022 at 17:32, Alvaro Herrera wrote: > Hello > > I've had this patch sitting in a local branch for way too long. It's a > trivial thing but for some reason it bothered me: we let the partition > strategy flow into the backend as a string and only parse it into the > catalog

Re: Crash after a call to pg_backup_start()

2022-10-21 Thread Richard Guo
On Fri, Oct 21, 2022 at 3:10 PM Kyotaro Horiguchi wrote: > It seems to me that the comment is true and the condition is a thinko. Yeah, the two conditions could be both false. How about we update the comment a bit to emphasize this? Such as /* At most one of these conditions can be true

Re: Standby recovers records from wrong timeline

2022-10-21 Thread Ants Aasma
On Fri, 21 Oct 2022 at 11:44, Kyotaro Horiguchi wrote: > > At Fri, 21 Oct 2022 17:12:45 +0900 (JST), Kyotaro Horiguchi > wrote in > > latest works. It dones't consider the case of explict target timlines > > so it's just a PoC. (So this doesn't work if recovery_target_timeline > > is set to 2

Re: Standby recovers records from wrong timeline

2022-10-21 Thread Kyotaro Horiguchi
At Fri, 21 Oct 2022 17:44:40 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 21 Oct 2022 17:12:45 +0900 (JST), Kyotaro Horiguchi > wrote in > > latest works. It dones't consider the case of explict target timlines > > so it's just a PoC. (So this doesn't work if recovery_target_timeline >

parse partition strategy string in gram.y

2022-10-21 Thread Alvaro Herrera
Hello I've had this patch sitting in a local branch for way too long. It's a trivial thing but for some reason it bothered me: we let the partition strategy flow into the backend as a string and only parse it into the catalog 1-char version quite late. This patch makes gram.y responsible for

RE: Perform streaming logical transactions by background workers and parallel apply

2022-10-21 Thread houzj.f...@fujitsu.com
On Wednesday, October 19, 2022 8:50 PM Kuroda, Hayato/黒田 隼人 wrote: > > === > 01. applyparallelworker.c - SIZE_STATS_MESSAGE > > ``` > /* > * There are three fields in each message received by the parallel apply > * worker: start_lsn, end_lsn and send_time. Because we have updated these > *

Re: Data is copied twice when specifying both child and parent table in publication

2022-10-21 Thread Peter Smith
Here are my review comments for HEAD patches v13* // Patch HEAD_v13-0001 I already posted some follow-up questions. See [1] / Patch HEAD_v13-0002 1. Commit message The following usage scenarios are not described in detail in the manual: If one subscription subscribes multiple

Re: Data is copied twice when specifying both child and parent table in publication

2022-10-21 Thread Peter Smith
On Mon, Oct 17, 2022 at 4:49 PM wangw.f...@fujitsu.com wrote: > > On Wed, Oct 5, 2022 at 11:08 AM Peter Smith wrote: > > Hi Wang-san. Here are my review comments for HEAD_v12-0001 patch. > ... > > > > 3. QUESTION - pg_get_publication_tables / fetch_table_list > > > > When the same table is

Re: fix stats_fetch_consistency value in postgresql.conf.sample

2022-10-21 Thread Kyotaro Horiguchi
At Fri, 21 Oct 2022 11:58:15 +1100, Peter Smith wrote in > Hi, I was hoping to use this patch in my other thread [1], but your > latest attachment is reported broken in cfbot [2]. Please rebase it. Ouch. I haven't reach here. I'll do that next Monday. regards. -- Kyotaro Horiguchi NTT Open

Re: Standby recovers records from wrong timeline

2022-10-21 Thread Kyotaro Horiguchi
At Fri, 21 Oct 2022 17:12:45 +0900 (JST), Kyotaro Horiguchi wrote in > latest works. It dones't consider the case of explict target timlines > so it's just a PoC. (So this doesn't work if recovery_target_timeline > is set to 2 for the "standby" in the repro.) So, finally I noticed that the

Re: cross-platform pg_basebackup

2022-10-21 Thread davinder singh
Hi, Patch v2 looks good to me, I have tested it, and pg_basebackup works fine across the platforms (Windows to Linux and Linux to Windows). Syntax used for testing $ pg_basebackup -h remote_server_ip -p 5432 -U user_name -D backup/data -T olddir=newdir I have also tested with non-absolute paths,

Re: Standby recovers records from wrong timeline

2022-10-21 Thread Kyotaro Horiguchi
At Fri, 21 Oct 2022 16:45:59 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 20 Oct 2022 14:44:40 +0300, Ants Aasma wrote in > > My understanding is that backup archives are supposed to remain valid > > even after PITR or equivalently a lagging standby promoting. > > Sorry, I was dim

Re: Standby recovers records from wrong timeline

2022-10-21 Thread Kyotaro Horiguchi
At Thu, 20 Oct 2022 14:44:40 +0300, Ants Aasma wrote in > My understanding is that backup archives are supposed to remain valid > even after PITR or equivalently a lagging standby promoting. Sorry, I was dim because of maybe catching a cold:p On second thought. everything works fine if the

Crash after a call to pg_backup_start()

2022-10-21 Thread Kyotaro Horiguchi
While playing with a proposed patch, I noticed that a session crashes after a failed call to pg_backup_start(). postgres=# select pg_backup_start(repeat('x', 1026)); ERROR: backup label too long (max 1024 bytes) postgres=# \q > TRAP: failed Assert("during_backup_start ^ (sessionBackupState == >

Re: Avoid memory leaks during base backups

2022-10-21 Thread Kyotaro Horiguchi
At Thu, 20 Oct 2022 19:47:07 +0200, Alvaro Herrera wrote in > I agree that's a good idea, and the patch looks good to me, but I don't > think asserting that they are null afterwards is useful. +1 for this direction. And the patch is fine to me. > oldcontext =

Re: Avoid memory leaks during base backups

2022-10-21 Thread Michael Paquier
On Fri, Oct 21, 2022 at 11:34:27AM +0530, Bharath Rupireddy wrote: > On Fri, Oct 21, 2022 at 12:21 AM Robert Haas wrote: >> On Thu, Oct 20, 2022 at 1:35 PM Bharath Rupireddy >> wrote: >>> I think elsewhere in the code we reset dangling pointers either ways - >>> before or after

Re: Avoid memory leaks during base backups

2022-10-21 Thread Michael Paquier
On Thu, Oct 20, 2022 at 02:51:21PM -0400, Robert Haas wrote: > On Thu, Oct 20, 2022 at 1:35 PM Bharath Rupireddy > wrote: >> I think elsewhere in the code we reset dangling pointers either ways - >> before or after deleting/resetting memory context. But placing them >> before would give us extra

Re: Avoid memory leaks during base backups

2022-10-21 Thread Bharath Rupireddy
On Thu, Oct 20, 2022 at 11:17 PM Alvaro Herrera wrote: > > On 2022-Oct-20, Bharath Rupireddy wrote: > > > I think elsewhere in the code we reset dangling pointers either ways - > > before or after deleting/resetting memory context. But placing them > > before would give us extra safety in case

Re: making relfilenodes 56 bits

2022-10-21 Thread Michael Paquier
On Thu, Sep 29, 2022 at 09:23:38PM -0400, Tom Lane wrote: > Hmmm ... I'd tend to do SELECT COUNT(*) FROM. But can't we provide > any actual checks on the sanity of the output? I realize that the > output's far from static, but still ... Honestly, checking all the fields is not that exciting,