Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-20 Thread John Naylor
On Thu, Dec 21, 2023 at 8:33 AM Masahiko Sawada wrote: > > I found the following comment and wanted to discuss: > > // this might be better as "iterate over nodes", plus a callback to > RT_DUMP_NODE, > // which should really only concern itself with single nodes > RT_SCOPE void >

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Andres Freund
Hi, On 2023-12-21 08:31:57 +0100, Peter Eisentraut wrote: > On 20.12.23 18:22, Andrew Dunstan wrote: > > > > On 2023-12-20 We 11:32, Robert Haas wrote: > > > On Wed, Dec 20, 2023 at 11:03 AM Andrew Dunstan > > > wrote: > > > > > This could perhaps be related to this patch removing the module in

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Peter Eisentraut
On 21.12.23 07:35, vignesh C wrote: I noticed this is because the lex file getting added without escape characters in the C file: #line 2 "D:\postgres\pg_meson\src\backend\utils\adt\jsonpath_scan.l" There were no warnings when I used flex 2.6.4. Did anyone else get these warnings with the flex

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Peter Eisentraut
On 20.12.23 18:22, Andrew Dunstan wrote: On 2023-12-20 We 11:32, Robert Haas wrote: On Wed, Dec 20, 2023 at 11:03 AM Andrew Dunstan wrote: This could perhaps be related to this patch removing the module in src/tools/msvc/dummylib/Win32/Registry.pm ? It is. I've fixed the buildfarm to stop

Re: introduce dynamic shared memory registry

2023-12-20 Thread Andrei Lepikhov
On 20/12/2023 17:33, Nathan Bossart wrote: On Wed, Dec 20, 2023 at 11:02:58AM +0200, Andrei Lepikhov wrote: In that case, maybe change the test case to make it closer to real-life usage - with locks and concurrent access (See attachment)? I'm not following why we should make this test case

Re: index prefetching

2023-12-20 Thread Dilip Kumar
On Wed, Dec 20, 2023 at 7:11 AM Tomas Vondra wrote: > I was going through to understand the idea, couple of observations -- + for (int i = 0; i < PREFETCH_LRU_SIZE; i++) + { + entry = >prefetchCache[lru * PREFETCH_LRU_SIZE + i]; + + /* Is this the oldest prefetch request in this LRU? */ + if

Re: Function to get invalidation cause of a replication slot.

2023-12-20 Thread Michael Paquier
On Thu, Dec 21, 2023 at 11:53:04AM +0530, Amit Kapila wrote: > On Thu, Dec 21, 2023 at 11:18 AM Michael Paquier wrote: > Yeah, if one uses them independently then there is no such guarantee. This could be possible in the same query as well, still less likely, as the contents are volatile. >> A

Re: Remove MSVC scripts from the tree

2023-12-20 Thread vignesh C
On Wed, 20 Dec 2023 at 21:13, Peter Eisentraut wrote: > > On 20.12.23 12:40, Andres Freund wrote: > > Hm, or perhaps we should just get rid of sed use altogether. The sepgsql > > case > > is trivially translateable to perl, and postprocess_dtrace.sed isn't > > much harder. > > Maybe yeah, but

Re: GUC names in messages

2023-12-20 Thread Peter Smith
Hi, This thread seems to be a bit stuck, so I thought I would try to summarize my understanding to hopefully get it moving again... The original intent of the thread was just to introduce some guidelines for quoting or not quoting GUC names in messages because previously it seemed quite ad-hoc.

Re: Function to get invalidation cause of a replication slot.

2023-12-20 Thread Amit Kapila
On Thu, Dec 21, 2023 at 11:18 AM Michael Paquier wrote: > > On Thu, Dec 21, 2023 at 09:37:39AM +0530, Amit Kapila wrote: > > It depends on how one uses the function. For example, if one finds > > there is a conflict via pg_get_replication_slots() and wants to check > > the reason for the same via

Re: speed up a logical replica setup

2023-12-20 Thread Amit Kapila
On Wed, Dec 6, 2023 at 12:53 PM Euler Taveira wrote: > > On Thu, Nov 9, 2023, at 8:12 PM, Michael Paquier wrote: > > On Thu, Nov 09, 2023 at 03:41:53PM +0100, Peter Eisentraut wrote: > > On 08.11.23 00:12, Michael Paquier wrote: > >> - Should the subdirectory pg_basebackup be renamed into

RE: Synchronizing slots from primary to standby

2023-12-20 Thread Zhijie Hou (Fujitsu)
On Thursday, December 21, 2023 12:25 PM Peter Smith wrote: > > Here is a minor comment for v51-0001 > > == > src/backend/replication/slot.c > > 1. > +void > +RereadConfigAndReInitSlotList(List **standby_slots) { > + char*pre_standby_slot_names; > + > + /* > + * If we are running on a

Re: Fixing backslash dot for COPY FROM...CSV

2023-12-20 Thread vignesh C
On Tue, 19 Dec 2023 at 16:57, Daniel Verite wrote: > > vignesh C wrote: > > > I noticed that these tests are passing without applying patch too: > > > +insert into copytest2(test) values('line1'), ('\.'), ('line2'); > > +copy (select test from copytest2 order by test collate "C") to

Re: Add isCatalogRel in rmgrdesc

2023-12-20 Thread Bertrand Drouvot
Hi, On Thu, Dec 21, 2023 at 10:13:16AM +0900, Masahiko Sawada wrote: > On Thu, Dec 21, 2023 at 9:04 AM Michael Paquier wrote: > > > > On Wed, Dec 20, 2023 at 10:43:30AM +0900, Masahiko Sawada wrote: > > > Thank you for updating the patch. The v2 patch looks good to me. I'll > > > push it,

Re: Function to get invalidation cause of a replication slot.

2023-12-20 Thread Michael Paquier
On Thu, Dec 21, 2023 at 09:37:39AM +0530, Amit Kapila wrote: > It depends on how one uses the function. For example, if one finds > there is a conflict via pg_get_replication_slots() and wants to check > the reason for the same via this new function then it would give the > correct answer. My

Re: WAL Insertion Lock Improvements

2023-12-20 Thread Michael Paquier
On Mon, Dec 18, 2023 at 10:00:29PM -0600, Nathan Bossart wrote: > I found this code when searching for callers that use atomic exchanges as > atomic writes with barriers (for a separate thread [0]). Can't we use > pg_atomic_write_u64() here since the locking functions that follow should > serve

Re: Synchronizing slots from primary to standby

2023-12-20 Thread Peter Smith
Here is a minor comment for v51-0001 == src/backend/replication/slot.c 1. +void +RereadConfigAndReInitSlotList(List **standby_slots) +{ + char*pre_standby_slot_names; + + /* + * If we are running on a standby, there is no need to reload + * standby_slot_names since we do not support

Re: Function to get invalidation cause of a replication slot.

2023-12-20 Thread Amit Kapila
On Thu, Dec 21, 2023 at 8:47 AM Michael Paquier wrote: > > On Wed, Dec 20, 2023 at 12:43:45PM +0100, Drouvot, Bertrand wrote: > > + 3 = wal_level insufficient on the primary > > server > > > > "." is missing at the end (to be consistent with 1 and 2). Same > > in the commit message. >

Re: pg_upgrade failing for 200+ million Large Objects

2023-12-20 Thread Tom Lane
Nathan Bossart writes: > Wow, thanks for putting together these patches. I intend to help review, Thanks! > but I'm not sure I'll find much time to do so before the new year. There's no urgency, surely. If we can get these in during the January CF, I'll be happy.

Re: trying again to get incremental backup

2023-12-20 Thread Alexander Lakhin
Hello Robert, 20.12.2023 23:56, Robert Haas wrote: On Wed, Dec 20, 2023 at 8:11 AM Jakub Wartak wrote: the v15 patchset (posted yesterday) test results are GOOD: All right. I committed the main two patches, dropped the for-testing-only patch, and added a simple test to the remaining

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-12-20 Thread Nathan Bossart
On Wed, Dec 20, 2023 at 12:21:05PM +0530, vignesh C wrote: > On Tue, 19 Dec 2023 at 21:22, Nathan Bossart wrote: >> I'm not sure we should proceed with rewriting most/all eligible foreach >> loops. I think it's fine to use the new macros in new code or to update >> existing loops in passing when

Re: Function to get invalidation cause of a replication slot.

2023-12-20 Thread Michael Paquier
On Wed, Dec 20, 2023 at 12:43:45PM +0100, Drouvot, Bertrand wrote: > + 3 = wal_level insufficient on the primary > server > > "." is missing at the end (to be consistent with 1 and 2). Same > in the commit message. > > + * Returns ReplicationSlotInvalidationCause enum value for valid

Re: pg_upgrade failing for 200+ million Large Objects

2023-12-20 Thread Nathan Bossart
On Wed, Dec 20, 2023 at 06:47:44PM -0500, Tom Lane wrote: > I have spent some more effort in this area and developed a patch > series that I think addresses all of the performance issues that > we've discussed in this thread, both for pg_upgrade and more > general use of pg_dump/pg_restore.

Re: Track in pg_replication_slots the reason why slots conflict?

2023-12-20 Thread Michael Paquier
On Thu, Dec 21, 2023 at 08:20:16AM +0530, Amit Kapila wrote: > The invalidation cause is also required by one of the features being > discussed "Synchronize slots from primary to standby" [1] and there is > already a thread to discuss the same [2]. As that thread started > yesterday only, you may

Re: Track in pg_replication_slots the reason why slots conflict?

2023-12-20 Thread Amit Kapila
On Thu, Dec 21, 2023 at 5:51 AM Michael Paquier wrote: > > While listening at Bertrand's talk about logical decoding on standbys > last week at Prague, I got surprised by the fact that we do not > reflect in the catalogs the reason why a conflict happened for a slot. > There are three of them

Re: broken master regress tests

2023-12-20 Thread Jeff Davis
On Tue, 2023-12-12 at 18:56 -0800, Noah Misch wrote: > > Yea. I wonder if the better fix would have been to copy > > setenv("LC_MESSAGES", "C", 1); > > to the initdb template creation. That afaict also fixes the issue, > > with a > > smaller blast radius? > > +1, that would restore the testing

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-20 Thread Masahiko Sawada
On Thu, Dec 21, 2023 at 10:19 AM John Naylor wrote: > > On Wed, Dec 20, 2023 at 6:36 PM Masahiko Sawada wrote: > > > > I've updated the new patch set that incorporated comments I got so > > far. 0007, 0008, and 0012 patches are updates from the v45 patch set. > > In addition to the review

Re: Add a perl function in Cluster.pm to generate WAL

2023-12-20 Thread Michael Paquier
On Wed, Dec 20, 2023 at 12:24:04AM -0300, Euler Taveira wrote: > LGTM. I was eyeing at 020_messages.pl and it has a pg_switch_wal() after a transaction rollbacked. 020_archive_status.pl creates a table, does one segment switch, then a checkpoint (table is used afterwards). Perhaps these could

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-20 Thread John Naylor
On Wed, Dec 20, 2023 at 6:36 PM Masahiko Sawada wrote: > > I've updated the new patch set that incorporated comments I got so > far. 0007, 0008, and 0012 patches are updates from the v45 patch set. > In addition to the review comments, I made some changes in tidstore to > make it independent from

Re: Add isCatalogRel in rmgrdesc

2023-12-20 Thread Masahiko Sawada
On Thu, Dec 21, 2023 at 10:13 AM Masahiko Sawada wrote: > > On Thu, Dec 21, 2023 at 9:04 AM Michael Paquier wrote: > > > > On Wed, Dec 20, 2023 at 10:43:30AM +0900, Masahiko Sawada wrote: > > > Thank you for updating the patch. The v2 patch looks good to me. I'll > > > push it, barring any

Re: Add isCatalogRel in rmgrdesc

2023-12-20 Thread Masahiko Sawada
On Thu, Dec 21, 2023 at 9:04 AM Michael Paquier wrote: > > On Wed, Dec 20, 2023 at 10:43:30AM +0900, Masahiko Sawada wrote: > > Thank you for updating the patch. The v2 patch looks good to me. I'll > > push it, barring any objections. > > This is capturing the eight records where the flag exists,

Re: Built-in CTYPE provider

2023-12-20 Thread Robert Haas
On Wed, Dec 20, 2023 at 5:57 PM Jeff Davis wrote: > Those locales all have the same ctype behavior. Sigh. I keep getting confused about how that works... -- Robert Haas EDB: http://www.enterprisedb.com

Re: "pgoutput" options missing on documentation

2023-12-20 Thread Peter Smith
On Thu, Dec 21, 2023 at 2:58 AM Emre Hasegeli wrote: > > > We don't expect unrecognized option here and for such a thing, we use > > elog in the code. See the similar usage in > > parseCreateReplSlotOptions(). > > "pgoutput" is useful for a lot of applications other than our logical > replication

Re: libpq compression (part 3)

2023-12-20 Thread Jelte Fennema-Nio
Thanks for working on this! One thing I'm wondering: should it be possible for the client to change the compression it wants mid-connection? I can think of some scenarios where that would be useful to connection poolers: if a pooler does plain forwarding of the compressed messages, then it would

Re: Built-in CTYPE provider

2023-12-20 Thread Jeremy Schneider
On 12/20/23 4:04 PM, Jeremy Schneider wrote: > On 12/20/23 3:47 PM, Jeremy Schneider wrote: >> On 12/5/23 3:46 PM, Jeff Davis wrote: >>> CTYPE, which handles character classification and upper/lowercasing >>> behavior, may be simpler than it first appears. We may be able to get >>> a net decrease

Track in pg_replication_slots the reason why slots conflict?

2023-12-20 Thread Michael Paquier
Hi all, (Bertrand and Andres in CC.) While listening at Bertrand's talk about logical decoding on standbys last week at Prague, I got surprised by the fact that we do not reflect in the catalogs the reason why a conflict happened for a slot. There are three of them depending on

Re: Add isCatalogRel in rmgrdesc

2023-12-20 Thread Michael Paquier
On Wed, Dec 20, 2023 at 10:43:30AM +0900, Masahiko Sawada wrote: > Thank you for updating the patch. The v2 patch looks good to me. I'll > push it, barring any objections. This is capturing the eight records where the flag exists, so it looks OK seen from here. As you said, there may be a

Re: Built-in CTYPE provider

2023-12-20 Thread Jeremy Schneider
On 12/20/23 3:47 PM, Jeremy Schneider wrote: > On 12/5/23 3:46 PM, Jeff Davis wrote: >> CTYPE, which handles character classification and upper/lowercasing >> behavior, may be simpler than it first appears. We may be able to get >> a net decrease in complexity by just building in most (or perhaps

Re: Built-in CTYPE provider

2023-12-20 Thread Jeremy Schneider
On 12/5/23 3:46 PM, Jeff Davis wrote: > CTYPE, which handles character classification and upper/lowercasing > behavior, may be simpler than it first appears. We may be able to get > a net decrease in complexity by just building in most (or perhaps all) > of the functionality. > > === Character

Re: pg_upgrade failing for 200+ million Large Objects

2023-12-20 Thread Tom Lane
I have spent some more effort in this area and developed a patch series that I think addresses all of the performance issues that we've discussed in this thread, both for pg_upgrade and more general use of pg_dump/pg_restore. Concretely, it absorbs the pg_restore --transaction-size switch that I

Re: ci: Build standalone INSTALL file

2023-12-20 Thread Michael Paquier
On Wed, Dec 20, 2023 at 03:28:56PM +0100, Daniel Gustafsson wrote: > + time make -s -j${BUILD_JOBS} -C doc/src/sgml all INSTALL > unrelated pet peeve: "make -C doc/src/sgml all" doesn't build all docs > targets.. That seems relevant in terms of coverage. Why not just moving the INSTALL bit

Re: ci: Build standalone INSTALL file

2023-12-20 Thread Michael Paquier
On Wed, Dec 20, 2023 at 11:36:28AM -0500, Tom Lane wrote: > Andres Freund writes: >> We fairly regularly have commits breaking the generation of INSTALL. IIRC we >> recently discussed building it locally unconditionally, but I couldn't >> immediately find that discussion. Until then, I think we

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Michael Paquier
On Wed, Dec 20, 2023 at 07:00:50PM +0100, Daniel Gustafsson wrote: > On 20 Dec 2023, at 18:22, Andrew Dunstan wrote: >> Yes, the module will normally be present on a Windows perl. The >> only reason we had dummylib was so we could check the perl scripts on >> Unix. > > Thanks for taking care of

Re: Built-in CTYPE provider

2023-12-20 Thread Jeff Davis
On Wed, 2023-12-20 at 14:24 -0500, Robert Haas wrote: > This makes sense to me, too, but it feels like it might work out > better for speakers of English than for speakers of other languages. There's very little in the way of locale-specific tailoring for ctype behaviors in ICU or glibc -- only

Re: Add --check option to pgindent

2023-12-20 Thread Andrew Dunstan
On 2023-12-20 We 11:05, Andrew Dunstan wrote: On 2023-12-19 Tu 08:57, Daniel Gustafsson wrote:   The posted v5 seems like a good compromise I reckon. Andrew: When applying this, how do we synchronize with the buildfarm to avoid false negatives due to the BF using the wrong options? The

Re: libpq compression (part 3)

2023-12-20 Thread Jacob Burroughs
> I'm having difficulty understanding the details of this handshaking > algorithm from this description. It seems good that the handshake > proceeds in each direction somewhat separately from the other, but I > don't quite understand how the whole thing fits together. If the > client tells the

Re: trying again to get incremental backup

2023-12-20 Thread Robert Haas
On Wed, Dec 20, 2023 at 8:11 AM Jakub Wartak wrote: > the v15 patchset (posted yesterday) test results are GOOD: All right. I committed the main two patches, dropped the for-testing-only patch, and added a simple test to the remaining pg_walsummary patch. That needs more work, but here's what I

Re: libpq compression (part 3)

2023-12-20 Thread Robert Haas
On Tue, Dec 19, 2023 at 11:41 AM Jacob Burroughs wrote: > The compression "handshaking" process now looks as follows: > 1. Client sends startup packet with `_pq_.libpq_compression = alg1;alg2` > 2. At this point, the server can immediately begin compressing packets > to the client with any of the

Re: Built-in CTYPE provider

2023-12-20 Thread Robert Haas
On Wed, Dec 20, 2023 at 2:13 PM Jeff Davis wrote: > On Wed, 2023-12-20 at 13:49 +0100, Daniel Verite wrote: > > If the Postgres default was bytewise sorting+locale-agnostic > > ctype functions directly derived from Unicode data files, > > as opposed to libc/$LANG at initdb time, the main > >

Re: Built-in CTYPE provider

2023-12-20 Thread Jeff Davis
On Wed, 2023-12-20 at 13:49 +0100, Daniel Verite wrote: > If the Postgres default was bytewise sorting+locale-agnostic > ctype functions directly derived from Unicode data files, > as opposed to libc/$LANG at initdb time, the main > annoyance would be that "ORDER BY textcol" would no > longer be

Re: index prefetching

2023-12-20 Thread Robert Haas
On Tue, Dec 19, 2023 at 8:41 PM Tomas Vondra wrote: > Whatever the right abstraction is, it probably needs to do these VM > checks only once. Makes sense. > Yeah, after you pointed out the "leaky" abstraction, I also started to > think about customizing the behavior using a callback. Not sure

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Daniel Gustafsson
> On 20 Dec 2023, at 18:22, Andrew Dunstan wrote: > On 2023-12-20 We 11:32, Robert Haas wrote: >> On Wed, Dec 20, 2023 at 11:03 AM Andrew Dunstan wrote: This could perhaps be related to this patch removing the module in src/tools/msvc/dummylib/Win32/Registry.pm ? >>> It is. I've fixed

Re: Estimation rows of FULL JOIN

2023-12-20 Thread Tom Lane
Danil Anisimow writes: > In some cases, the planner underestimates FULL JOIN. Perhaps. > Are these simple changes enough to improve this situation? This looks like an entirely ad-hoc change, one that could make as many cases worse as it makes better. If you want to argue for merging it, you

Estimation rows of FULL JOIN

2023-12-20 Thread Danil Anisimow
Hi. In some cases, the planner underestimates FULL JOIN. Example: postgres=# CREATE TABLE t AS SELECT x AS a, null AS b FROM generate_series(1, 10) x; postgres=# ANALYZE; postgres=# EXPLAIN (ANALYZE, TIMING OFF) SELECT * FROM t t1 FULL JOIN t t2 ON t1.b = t2.b;

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Andrew Dunstan
On 2023-12-20 We 11:32, Robert Haas wrote: On Wed, Dec 20, 2023 at 11:03 AM Andrew Dunstan wrote: This could perhaps be related to this patch removing the module in src/tools/msvc/dummylib/Win32/Registry.pm ? It is. I've fixed the buildfarm to stop checking this script. Thanks! But I

Re: Detecting some cases of missing backup_label

2023-12-20 Thread David Steele
On 12/18/23 10:39, Stephen Frost wrote: Greetings, * Stephen Frost (sfr...@snowman.net) wrote: * Andres Freund (and...@anarazel.de) wrote: I recently mentioned to Robert (and also Heikki earlier), that I think I see a way to detect an omitted backup_label in a relevant subset of the cases

Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2023-12-20 Thread Michail Nikolaev
> Yes, good catch. > Assuming we have somehow prevented vac_truncate_clog from occurring > during CIC, we can leave frozen and potentially frozen > (xmin

Re: ci: Build standalone INSTALL file

2023-12-20 Thread Tom Lane
Andres Freund writes: > We fairly regularly have commits breaking the generation of INSTALL. IIRC we > recently discussed building it locally unconditionally, but I couldn't > immediately find that discussion. Until then, I think we should at least > build it in CI so that cfbot can warn. I

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Robert Haas
On Wed, Dec 20, 2023 at 11:03 AM Andrew Dunstan wrote: > > This could perhaps be related to this patch removing the module in > > src/tools/msvc/dummylib/Win32/Registry.pm ? > > It is. I've fixed the buildfarm to stop checking this script. Thanks! But I wonder whether the script itself also

Re: Unchecked strdup leading to segfault in pg_dump

2023-12-20 Thread Nathan Bossart
On Wed, Dec 20, 2023 at 09:39:55AM -0600, Tristan Partin wrote: > On Wed Dec 20, 2023 at 8:52 AM CST, Daniel Gustafsson wrote: >> While looking at something else I noticed that pg_dump performs strdup >> without >> checking the returned pointer, which will segfault in hasSuffix() in case of >>

Re: introduce dynamic shared memory registry

2023-12-20 Thread Nathan Bossart
On Thu, Dec 21, 2023 at 12:03:18AM +0800, Zhang Mingli wrote: > I see most xxxShmemInit functions have the logic to handle IsUnderPostmaster > env. > Do we need to consider it in DSMRegistryShmemInit() too? For example, add > some assertions. > Others LGTM. Good point. I _think_ the registry

Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2023-12-20 Thread Michail Nikolaev
Hello! > How would this deal with tuples not visible to the old snapshot? > Presumably we can assume they're newer than that snapshot (the old > snapshot didn't have it, but the new one does, so it's committed after > the old snapshot, making them newer), so that backend must have > inserted it

Re: Add --check option to pgindent

2023-12-20 Thread Andrew Dunstan
On 2023-12-19 Tu 08:57, Daniel Gustafsson wrote: On 19 Dec 2023, at 14:51, Andrew Dunstan wrote: On 2023-12-18 Mo 11:14, Jelte Fennema-Nio wrote: On Mon, 18 Dec 2023 at 13:42, Daniel Gustafsson wrote: I think this is pretty much ready to go, the attached v4 squashes the changes and fixes

Re: introduce dynamic shared memory registry

2023-12-20 Thread Zhang Mingli
Hi, all I see most xxxShmemInit functions have the logic to handle IsUnderPostmaster env. Do we need to consider it in DSMRegistryShmemInit() too? For example, add some assertions. Others LGTM. Zhang Mingli www.hashdata.xyz On Dec 5, 2023 at 11:47 +0800, Nathan Bossart , wrote: > Every once

Re: introduce dynamic shared memory registry

2023-12-20 Thread Nathan Bossart
On Wed, Dec 20, 2023 at 03:28:38PM +0530, Bharath Rupireddy wrote: > Isn't the worker_spi best place to show the use of the DSM registry > instead of a separate test extension? Note the custom wait event > feature that added its usage code to worker_spi. Since worker_spi > demonstrates typical

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Andrew Dunstan
On 2023-12-20 We 08:31, Daniel Gustafsson wrote: On 20 Dec 2023, at 01:48, Michael Paquier wrote: On Mon, Nov 20, 2023 at 05:03:28PM +0900, Michael Paquier wrote: Your suggestion to create a new sect2 for "Windows" as much as Andres' suggestion are OK by as an intermediate step, and I

Re: "pgoutput" options missing on documentation

2023-12-20 Thread Emre Hasegeli
> We don't expect unrecognized option here and for such a thing, we use > elog in the code. See the similar usage in > parseCreateReplSlotOptions(). "pgoutput" is useful for a lot of applications other than our logical replication subscriber. I think we should expect anything and handle errors

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Peter Eisentraut
On 20.12.23 12:40, Andres Freund wrote: Hm, or perhaps we should just get rid of sed use altogether. The sepgsql case is trivially translateable to perl, and postprocess_dtrace.sed isn't much harder. Maybe yeah, but also it seems fine as is and we can easily fix the present issue ... OTOH,

Re: Unchecked strdup leading to segfault in pg_dump

2023-12-20 Thread Tristan Partin
On Wed Dec 20, 2023 at 8:52 AM CST, Daniel Gustafsson wrote: While looking at something else I noticed that pg_dump performs strdup without checking the returned pointer, which will segfault in hasSuffix() in case of OOM. The attached, which should be backpatched to 16, changes to using

Re: introduce dynamic shared memory registry

2023-12-20 Thread Nathan Bossart
On Wed, Dec 20, 2023 at 11:02:58AM +0200, Andrei Lepikhov wrote: > In that case, maybe change the test case to make it closer to real-life > usage - with locks and concurrent access (See attachment)? I'm not following why we should make this test case more complicated. It is only intended to

Unchecked strdup leading to segfault in pg_dump

2023-12-20 Thread Daniel Gustafsson
While looking at something else I noticed that pg_dump performs strdup without checking the returned pointer, which will segfault in hasSuffix() in case of OOM. The attached, which should be backpatched to 16, changes to using pg_strdup instead which handles it. -- Daniel Gustafsson

Re: ci: Build standalone INSTALL file

2023-12-20 Thread Daniel Gustafsson
> On 20 Dec 2023, at 15:15, Andres Freund wrote: > The attached patch had a slight bug. Also turned out that the CI environment > didn't have pandoc installed. Fixed that. LGTM. + time make -s -j${BUILD_JOBS} -C doc/src/sgml all INSTALL unrelated pet peeve: "make -C doc/src/sgml all"

Re: ci: Build standalone INSTALL file

2023-12-20 Thread Andres Freund
Hi, The attached patch had a slight bug. Also turned out that the CI environment didn't have pandoc installed. Fixed that. Greetings, Andres Freund >From 6a8157b2f14327351798c805a873e9e910f5cd67 Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Wed, 20 Dec 2023 03:44:44 -0800 Subject: [PATCH

Re: backtrace_on_internal_error

2023-12-20 Thread Jelte Fennema-Nio
On Wed, 20 Dec 2023 at 11:30, Andres Freund wrote: > Tom's patch imo doesn't really introduce anything really new - we already deal > with EOF that way in other places. And it's how the standard APIs deal with > the issue. I'd not design it this way on a green field, but given the current > state

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2023-12-20 Thread Jelte Fennema-Nio
On Thu, 14 Dec 2023 at 13:57, Jelte Fennema-Nio wrote: > I changed all the places that were not adhering to those spellings. It seems I forgot a /g on my sed command to do this so it turned out I missed one that caused the test to fail to compile... Attached is a fixed version. I also updated

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Daniel Gustafsson
> On 20 Dec 2023, at 01:48, Michael Paquier wrote: > > On Mon, Nov 20, 2023 at 05:03:28PM +0900, Michael Paquier wrote: >> Your suggestion to create a new sect2 for "Windows" as much as Andres' >> suggestion are OK by as an intermediate step, and I suspect that the >> end result will likely not

Re: trying again to get incremental backup

2023-12-20 Thread Jakub Wartak
Hi Robert, On Tue, Dec 19, 2023 at 9:36 PM Robert Haas wrote: > > On Fri, Dec 15, 2023 at 5:36 AM Jakub Wartak > wrote: > > I've played with with initdb/pg_upgrade (17->17) and i don't get DBID > > mismatch (of course they do differ after initdb), but i get this > > instead: > > > > $

Re: Table AM Interface Enhancements

2023-12-20 Thread Pavel Borisov
> > > Additionally changes in 0007 looks dependent from 0005. Does replacement > of slot inside ExecInsert, that is already used in the code below the call > of > > >/* insert the tuple normally */ > >- table_tuple_insert(resultRelationDesc, slot, > >- estate->es_output_cid, > >- 0, NULL); > >

Re: speed up a logical replica setup

2023-12-20 Thread Euler Taveira
On Wed, Dec 20, 2023, at 9:22 AM, Shlok Kyal wrote: > When I built the code using Visual Studio, on installing postgres, > pg_subscriber binary was not created. > But when I built the code using Meson, on installing postgres, > pg_subscriber binary was created. > Is this behaviour intentional?

Re: Table AM Interface Enhancements

2023-12-20 Thread Pavel Borisov
Hi, Alexander! I've reviewed patch 0004. It's clear enough and I think does what it's supposed. One thing, in function signature +bool (*tuple_is_current) (Relation rel, TupleTableSlot *slot); there is a Relation agrument, which is unused in both existing heapam method. Also it's unused in

Re: Built-in CTYPE provider

2023-12-20 Thread Daniel Verite
Jeff Davis wrote: > But there are a lot of users for whom neither of those things are true, > and it makes zero sense to order all of the text indexes in the > database according to any one particular locale. I think these users > would prioritize stability and performance for the

RE: Synchronizing slots from primary to standby

2023-12-20 Thread Zhijie Hou (Fujitsu)
On Tuesday, December 19, 2023 9:05 PM Kuroda, Hayato/黒田 隼人 wrote: > > Dear Shveta, > > I resumed to review the patch. I will play more about it, but I can post some > cosmetic comments. Thanks for the comments. > > > walsender.c > > 01. WalSndWaitForStandbyConfirmation > > ``` > +

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-12-20 Thread Masahiko Sawada
On Wed, Dec 20, 2023 at 1:07 PM jian he wrote: > > On Tue, Dec 19, 2023 at 9:14 AM Masahiko Sawada wrote: > > > > > > The error table hub idea is still unclear to me. I assume that there > > are error tables at least on each database. And an error table can > > have error data that happened

Re: Transaction timeout

2023-12-20 Thread wenhui qiu
Hi Junwang Zhao Agree +1 Best whish Junwang Zhao 于2023年12月20日周三 10:35写道: > On Wed, Dec 20, 2023 at 9:58 AM Thomas wen > wrote: > > > > Hi Junwang Zhao > > #should we invalidate lock_timeout? Or maybe just document this. > > I think you mean when lock_time is greater than

Re: speed up a logical replica setup

2023-12-20 Thread Shlok Kyal
Hi, On Wed, 6 Dec 2023 at 12:53, Euler Taveira wrote: > > On Thu, Nov 9, 2023, at 8:12 PM, Michael Paquier wrote: > > On Thu, Nov 09, 2023 at 03:41:53PM +0100, Peter Eisentraut wrote: > > On 08.11.23 00:12, Michael Paquier wrote: > >> - Should the subdirectory pg_basebackup be renamed into

Re: Clang optimiser vs preproc.c

2023-12-20 Thread Alexander Lakhin
Hello Thomas, 19.12.2023 07:20, Thomas Munro wrote: On Tue, Dec 19, 2023 at 11:42 AM Thomas Munro wrote: Hrmph. Well something weird is going on, but it might indeed involve me being confused about debug options of the compiler itself. How can one find out which build options were used for

ci: Build standalone INSTALL file

2023-12-20 Thread Andres Freund
Hi, We fairly regularly have commits breaking the generation of INSTALL. IIRC we recently discussed building it locally unconditionally, but I couldn't immediately find that discussion. Until then, I think we should at least build it in CI so that cfbot can warn. Greetings, Andres Freund >From

Re: Function to get invalidation cause of a replication slot.

2023-12-20 Thread Drouvot, Bertrand
Hi, On 12/20/23 10:55 AM, shveta malik wrote: On Wed, Dec 20, 2023 at 2:20 PM Amit Kapila wrote: PFA v2 patch. Addressed below comments: 1) Added test in 019_replslot_limit.pl 2) 'pg_get_slot_invalidation_cause' now returns error if the given slot does not exist 3) Corrected doc and commit

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Andres Freund
On 2023-12-20 16:15:55 +0530, vignesh C wrote: > On Tue, 19 Dec 2023 at 20:54, Peter Eisentraut wrote: > > > > On 18.12.23 14:52, Peter Eisentraut wrote: > > >> 2) I had seen that if sed/gzip is not available meson build will fail: > > >> 2.a) > > >> Program gsed sed found: NO > > >>

Re: Synchronizing slots from primary to standby

2023-12-20 Thread Amit Kapila
On Wed, Dec 20, 2023 at 3:29 PM shveta malik wrote: > > On Wed, Dec 20, 2023 at 9:12 AM Amit Kapila wrote: > > > > On Tue, Dec 19, 2023 at 5:30 PM shveta malik wrote: > > > > > > Thanks for reviewing. I have addressed these in v50. > > > > > > > I was looking at this patch to see if something

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-20 Thread Masahiko Sawada
On Tue, Dec 19, 2023 at 4:37 PM John Naylor wrote: > > On Tue, Dec 19, 2023 at 12:37 PM Masahiko Sawada > wrote: > > > > On Mon, Dec 18, 2023 at 3:41 PM John Naylor wrote: > > > Let's do it in just one place. In TidStoreCreate(), do > > > > > > /* clamp max_bytes to at least the size of the

Support a wildcard in backtrace_functions

2023-12-20 Thread Jelte Fennema-Nio
I would like to be able to add backtraces to all ERROR logs. This is useful to me, because during postgres or extension development any error that I hit is usually unexpected. This avoids me from having to change backtrace_functions every time I get an error based on the function name listed in

Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2023-12-20 Thread Matthias van de Meent
On Wed, 20 Dec 2023 at 10:56, Michail Nikolaev wrote: > > Note that the use of such expressions would be a violation of the > > function's definition; it would depend on data from other tables which > > makes the function behave like a STABLE function, as opposed to the > > IMMUTABLE that is

Re: Function to get invalidation cause of a replication slot.

2023-12-20 Thread Amit Kapila
On Wed, Dec 20, 2023 at 4:10 PM Drouvot, Bertrand wrote: > > On 12/20/23 9:50 AM, Amit Kapila wrote: > > On Wed, Dec 20, 2023 at 12:46 PM Drouvot, Bertrand > > wrote: > >> > > > >> 4 === > >> > >> + * Returns ReplicationSlotInvalidationCause enum value for valid > >> slot_name; > >> > >> I

Re: Remove MSVC scripts from the tree

2023-12-20 Thread vignesh C
On Tue, 19 Dec 2023 at 20:54, Peter Eisentraut wrote: > > On 18.12.23 14:52, Peter Eisentraut wrote: > >> 2) I had seen that if sed/gzip is not available meson build will fail: > >> 2.a) > >> Program gsed sed found: NO > >> meson.build:334:6: ERROR: Program 'gsed sed' not found or not executable

Re: Function to get invalidation cause of a replication slot.

2023-12-20 Thread Drouvot, Bertrand
Hi, On 12/20/23 10:57 AM, shveta malik wrote: On Wed, Dec 20, 2023 at 12:46 PM Drouvot, Bertrand Worth to add some coverage in 019_replslot_limit.pl and 035_standby_logical_decoding.pl? I have recently added a test in 019_replslot_limit.pl in v2. Thanks! Do you suggest adding in

Re: Function to get invalidation cause of a replication slot.

2023-12-20 Thread Drouvot, Bertrand
Hi, On 12/20/23 9:50 AM, Amit Kapila wrote: On Wed, Dec 20, 2023 at 12:46 PM Drouvot, Bertrand wrote: On 12/20/23 7:01 AM, shveta malik wrote: Hello hackers, Attached is a patch which attempts to implement a new system function pg_get_slot_invalidation_cause('slot_name') to get

Re: backtrace_on_internal_error

2023-12-20 Thread Andres Freund
Hi, On 2023-12-20 10:08:42 +0100, Jelte Fennema-Nio wrote: > On Sun, 10 Dec 2023 at 00:14, Tom Lane wrote: > > I'm not actually sure that the fe-secure.c part of v3-0002 is > > necessary, because it's guarding plain recv(2) which really shouldn't > > return -1 without setting errno. Still, it's

Re: Allow custom parameters with more than one dot in config files.

2023-12-20 Thread Alexander Kukushkin
On Tue, 19 Dec 2023 at 16:13, Tom Lane wrote: > > I wonder if we wouldn't be better advised to require exactly one dot. > This isn't a feature that we really encourage users to use, and the > further we move the goalposts for it, the harder it will be to replace > it. In particular I doubt the

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Andres Freund
On 2023-12-20 09:48:37 +0900, Michael Paquier wrote: > On Mon, Nov 20, 2023 at 05:03:28PM +0900, Michael Paquier wrote: > > Your suggestion to create a new sect2 for "Windows" as much as Andres' > > suggestion are OK by as an intermediate step, and I suspect that the > > end result will likely not

  1   2   >