Re: Test 002_pg_upgrade fails with olddump on Windows

2023-12-04 Thread Michael Paquier
On Sat, Nov 25, 2023 at 11:00:01PM +0300, Alexander Lakhin wrote: > diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl > b/src/bin/pg_upgrade/t/002_pg_upgrade.pl > index c6d83d3c21..d34b45e346 100644 > --- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl > +++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl >

Re: btree: downlink right separator/HIKEY optimization

2023-12-04 Thread Heikki Linnakangas
On 01/11/2023 00:08, Matthias van de Meent wrote: calling _bt_compare in _bt_moveright in many common cases. This patch, when stacked on top of the prefix truncation patch, improves INSERT performance by an additional 2-9%pt, with an extreme case of 45% in the worscase index tests at [0]. The

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-04 Thread Shlok Kyal
Hi, > As for the test results, I very much doubt the differences are not > caused simply by random timing variations, or something like that. And I > don't understand what "Performance Machine Linux" is, considering those > timings are slower than the other two machines. The machine has Total

Re: Show WAL write and fsync stats in pg_stat_io

2023-12-04 Thread Michael Paquier
On Fri, Dec 01, 2023 at 12:02:05PM +0300, Nazir Bilal Yavuz wrote: > Thanks for all the feedback. I am sharing the new version of the patchset. > > - I didn't move 'PendingWalStats.wal_sync' out from the > 'pgstat_count_io_op_n' function because they count the same thing > (block vs system calls)

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-12-04 Thread Hayato Kuroda (Fujitsu)
Dear Sawada-san, hackers, Based on comments I made a fix. PSA the patch. > > Is there any reason why this function can be executed only in binary > upgrade mode? It seems to me that other functions in > pg_upgrade_support.c must be called only in binary upgrade mode > because it does some hacky

Re: pg_upgrade and logical replication

2023-12-04 Thread Michael Paquier
On Mon, Dec 04, 2023 at 04:30:49PM +0530, Amit Kapila wrote: > I have made minor changes in the comments and code at various places. > See and let me know if you are not happy with the changes. I think > unless there are more suggestions or comments, we can proceed with > committing it. Yeah. I

Re: [DOCS] HOT - correct claim about indexes not referencing old line pointers

2023-12-04 Thread Shubham Khanna
On Tue, Dec 5, 2023 at 10:51 AM James Coleman wrote: > > Hello, > > While working on my talk for PGConf.NYC next week I came across this > bullet in the docs on heap only tuples: > > > Old versions of updated rows can be completely removed during normal > > operation, including SELECTs, instead

Re: SET ROLE documentation improvement

2023-12-04 Thread Shubham Khanna
On Fri, Nov 10, 2023 at 11:11 PM Nathan Bossart wrote: > > On Tue, Sep 26, 2023 at 08:33:25AM -0700, Yurii Rashkovskii wrote: > > This is a good start, indeed. I've amended my patch to include it. > > Thanks for the new patch. > > Looking again, I'm kind of hesitant to add too much qualification

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-12-04 Thread Amit Kapila
On Mon, Dec 4, 2023 at 11:59 AM Hayato Kuroda (Fujitsu) wrote: > > Dear hackers, > > I found another failure related with the commit [1]. This is caused by missing > wait on the test code. Amit helped me for this analysis and fix. > Pushed! -- With Regards, Amit Kapila.

Re: Synchronizing slots from primary to standby

2023-12-04 Thread shveta malik
On Mon, Dec 4, 2023 at 10:07 PM Drouvot, Bertrand wrote: > > Hi, > > On 12/4/23 6:10 AM, shveta malik wrote: > > On Fri, Dec 1, 2023 at 5:40 PM Nisha Moond wrote: > >> > >> Review for v41 patch. > > > > Thanks for the feedback. > > > >> ~~~ > >> 2. > >> IIUC, the slotsyncworker's connection to

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2023-12-04 Thread Peter Geoghegan
On Mon, Dec 4, 2023 at 7:25 PM Peter Geoghegan wrote: > 2. The optimization that has _bt_checkkeys skip non-required scan keys > that are *only* required in the direction *opposite* the current scan > direction -- this can work even without any precheck from > _bt_readpage. > > Note that this

Re: table inheritance versus column compression and storage settings

2023-12-04 Thread Ashutosh Bapat
On Mon, Dec 4, 2023 at 4:23 PM Peter Eisentraut wrote: > > Looking at the code, I suspect these rules were just sort of > copy-and-pasted from the nearby rules for types and collations. The > latter are needed so that a table with inheritance children can present > a logically consistent view

Re: CRC32C Parallel Computation Optimization on ARM

2023-12-04 Thread Nathan Bossart
On Mon, Dec 04, 2023 at 07:27:01AM +, Xiang Gao wrote: > This is the latest patch. Looking forward to your feedback, thanks! Thanks for the new patch. I am hoping to spend much more time on this in the near future... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Use PGAlignedBlock instead of "char buf[BLCKSZ]" in more places

2023-12-04 Thread Michael Paquier
On Mon, Dec 04, 2023 at 09:47:24AM -0800, Andres Freund wrote: > If we change something, we should consider making buffers like these aligned > to page sizes, rather than just MAXALIGNED. You mean 4k kernel pages, right? That makes sense to me. -- Michael signature.asc Description: PGP

introduce dynamic shared memory registry

2023-12-04 Thread Nathan Bossart
Every once in a while, I find myself wanting to use shared memory in a loadable module without requiring it to be loaded at server start via shared_preload_libraries. The DSM API offers a nice way to create and manage dynamic shared memory segments, so creating a segment after server start is

Re: Rename ShmemVariableCache and initialize it in more standard way

2023-12-04 Thread Richard Guo
On Tue, Dec 5, 2023 at 12:31 AM Tristan Partin wrote: > On Mon Dec 4, 2023 at 6:49 AM CST, Heikki Linnakangas wrote: > > This came up in the "Refactoring backend fork+exec code" thread recently > > [0], but is independent of that work: > > > > Here's a patch to allocate and initialize it with a

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2023-12-04 Thread Peter Geoghegan
On Mon, Nov 27, 2023 at 5:39 AM Heikki Linnakangas wrote: > - +1 on the general idea. Hard to see any downsides if implemented right. Glad you think so. The "no possible downside" perspective is one that the planner sort of relies on, so this isn't just a nice-to-have -- it might actually be a

Re: Emitting JSON to file using COPY TO

2023-12-04 Thread Joe Conway
On 12/4/23 17:55, Davin Shearer wrote: Sorry about the top posting / top quoting... the link you sent me gives me a 404.  I'm not exactly sure what top quoting / posting means and Googling those terms wasn't helpful for me, but I've removed the quoting that my mail client is automatically

Re: gai_strerror() is not thread-safe on Windows

2023-12-04 Thread Kyotaro Horiguchi
At Tue, 5 Dec 2023 08:26:54 +1300, Thomas Munro wrote in > On second thoughts, I guess it would make more sense to use the exact > messages Windows' own implementation would return instead of whatever > we had in the past (probably cribbed from some other OS or just made > up?). I asked CI to

Re: Avoid detoast overhead when possible

2023-12-04 Thread zhihuifan1213
Hi, Matthias van de Meent writes: > SELECT toastable_col FROM t1 > WHERE f(t1.toastable_col) > ORDER BY nonindexed; Thanks for this example! it's true that the current design requires more memory to sort since toastable_col is detoasted at the scan stage and it is output to the sort node. It

Re: CREATE FUNCTION ... SEARCH { DEFAULT | SYSTEM | SESSION }

2023-12-04 Thread Jeff Davis
On Tue, 2023-11-21 at 09:24 -0500, Robert Haas wrote: > As to the second, could we somehow come > up with an API for guc.c where you can ask for an opaque handle that > will later allow you to do a fast-SET of a GUC? Yes, attached. That provides a significant speedup: my test goes from around

Re: PATCH: Add REINDEX tag to event triggers

2023-12-04 Thread Michael Paquier
On Tue, Dec 05, 2023 at 12:49:12AM +0800, jian he wrote: > On Mon, Dec 4, 2023 at 11:00 PM Alexander Lakhin wrote: >> Please look at the assertion failure triggered when REINDEX processed by an >> event trigger: >> CREATE FUNCTION etf() RETURNS EVENT_TRIGGER AS $$ BEGIN PERFORM 1; END $$ >>

Re: Emitting JSON to file using COPY TO

2023-12-04 Thread Davin Shearer
Sorry about the top posting / top quoting... the link you sent me gives me a 404. I'm not exactly sure what top quoting / posting means and Googling those terms wasn't helpful for me, but I've removed the quoting that my mail client is automatically "helpfully" adding to my emails. I mean no

Re: Remove MSVC scripts from the tree

2023-12-04 Thread Michael Paquier
On Mon, Dec 04, 2023 at 03:11:47PM -0500, Andrew Dunstan wrote: > I just had a look at shifting bowerbird to use meson, and it got stymied at > the c99 test, which apparently doesn't compile with anything less than > VS2019. > > I can upgrade bowerbird, but that will take rather longer. It looks

Re: Failure with pgbench and --disable-thread-safety in ~v16

2023-12-04 Thread Michael Paquier
On Mon, Dec 04, 2023 at 12:17:37PM +0100, Alvaro Herrera wrote: > So, the fix is just to remove the -j2 in the command line. I'll do that > in a jiffy. There's no other test that uses -j, except the one > specifically designed to test threads. Thanks for the quick fix. -- Michael

Re: Extensible storage manager API - SMGR hook Redux

2023-12-04 Thread Matthias van de Meent
On Mon, 4 Dec 2023 at 22:03, Kirill Reshke wrote: > > On Mon, 4 Dec 2023 at 22:21, Matthias van de Meent > wrote: >> >> On Mon, 4 Dec 2023 at 17:51, Kirill Reshke wrote: >> > >> > So, 0002 patch uses the `get_tablespace` function, which searches Catalog >> > to tablespace SMGR id. I wonder

Re: optimize atomic exchanges

2023-12-04 Thread Nathan Bossart
On Mon, Dec 04, 2023 at 12:18:05PM -0600, Nathan Bossart wrote: > Barring objections or additional feedback, I think I'm inclined to press > forward with this one and commit it in the next week or two. I'm currently > planning to keep the inline assembly, but I'm considering removing the >

Re: Extensible storage manager API - SMGR hook Redux

2023-12-04 Thread Kirill Reshke
On Mon, 4 Dec 2023 at 22:21, Matthias van de Meent < boekewurm+postg...@gmail.com> wrote: > On Mon, 4 Dec 2023 at 17:51, Kirill Reshke wrote: > > > > So, 0002 patch uses the `get_tablespace` function, which searches > Catalog to tablespace SMGR id. I wonder how `smgr_redo` would work with it? >

Re: trying again to get incremental backup

2023-12-04 Thread Robert Haas
On Thu, Nov 30, 2023 at 9:33 AM Robert Haas wrote: > 0005: Incremental backup itself. Changes: > - Remove UPLOAD_MANIFEST replication command and instead add > INCREMENTAL_WAL_RANGE replication command. Unfortunately, I think this change is going to need to be reverted. Jakub reported out a

Re: meson: Stop using deprecated way getting path of files

2023-12-04 Thread Tristan Partin
On Mon Dec 4, 2023 at 2:10 PM CST, Tom Lane wrote: "Tristan Partin" writes: > On Fri Dec 1, 2023 at 12:16 PM CST, Tristan Partin wrote: >>> Ok, so what I found is that we still have build farm animals using >>> Python 3.4, specifically the AIX machines. There was also at least one >>> Python

Re: Remove MSVC scripts from the tree

2023-12-04 Thread Andrew Dunstan
On 2023-12-04 Mo 03:05, Michael Paquier wrote: On Tue, Sep 26, 2023 at 12:17:04PM -0400, Andrew Dunstan wrote: On 2023-09-26 Tu 01:25, NINGWEI CHEN wrote: hamerkop is not yet prepared for Meson builds, but we plan to work on this support soon. If we go with Meson builds exclusively right

Re: meson: Stop using deprecated way getting path of files

2023-12-04 Thread Tom Lane
"Tristan Partin" writes: > On Fri Dec 1, 2023 at 12:16 PM CST, Tristan Partin wrote: >>> Ok, so what I found is that we still have build farm animals using >>> Python 3.4, specifically the AIX machines. There was also at least one >>> Python 3.5 user too. Note that this was a manual check. > I

Re: Postgres Partitions Limitations (5.11.2.3)

2023-12-04 Thread Laurenz Albe
On Fri, 2023-12-01 at 18:49 +0530, Ashutosh Bapat wrote: > On Thu, Nov 30, 2023 at 10:29 PM Laurenz Albe > wrote: > > > > On Thu, 2023-11-30 at 19:22 +0530, Ashutosh Bapat wrote: > > > May be attach the patch to hackers thread (this) as well? > > > > If you want, sure.  I thought it was good

Re: Emitting JSON to file using COPY TO

2023-12-04 Thread Andrew Dunstan
On 2023-12-04 Mo 13:37, Davin Shearer wrote: Looking great! For testing, in addition to the quotes, include DOS and Unix EOL, \ and /, Byte Order Markers, and mulitbyte characters like UTF-8. Essentially anything considered textural is fair game to be a value. Joe already asked you to

Re: meson: Stop using deprecated way getting path of files

2023-12-04 Thread Tristan Partin
On Fri Dec 1, 2023 at 12:16 PM CST, Tristan Partin wrote: On Fri Dec 1, 2023 at 12:07 PM CST, Tom Lane wrote: > "Tristan Partin" writes: > > On the buildfarm page[0], it would be nice if more than just the > > compiler versions were stated. It would be nice to have all > > build/runtime

Re: gai_strerror() is not thread-safe on Windows

2023-12-04 Thread Thomas Munro
On second thoughts, I guess it would make more sense to use the exact messages Windows' own implementation would return instead of whatever we had in the past (probably cribbed from some other OS or just made up?). I asked CI to spit those out[1]. Updated patch attached. Will add to CF. [1]

Re: Extensible storage manager API - SMGR hook Redux

2023-12-04 Thread Matthias van de Meent
On Mon, 4 Dec 2023 at 17:51, Kirill Reshke wrote: > > So, 0002 patch uses the `get_tablespace` function, which searches Catalog to > tablespace SMGR id. I wonder how `smgr_redo` would work with it? That's a very good point I hadn't considered in detail yet. Quite clearly, the current code is

Re: Change GUC hashtable to use simplehash?

2023-12-04 Thread Jeff Davis
On Mon, 2023-12-04 at 12:12 +0700, John Naylor wrote: > That's a good thing to clear up. This thread has taken simplehash as > a > starting point from the very beginning. It initially showed no > improvement, and then we identified problems with the hashing and > equality computations. The latter

Re: Emitting JSON to file using COPY TO

2023-12-04 Thread Davin Shearer
Looking great! For testing, in addition to the quotes, include DOS and Unix EOL, \ and /, Byte Order Markers, and mulitbyte characters like UTF-8. Essentially anything considered textural is fair game to be a value. On Mon, Dec 4, 2023, 10:46 Joe Conway wrote: > On 12/4/23 09:25, Andrew

Re: optimize atomic exchanges

2023-12-04 Thread Nathan Bossart
On Thu, Nov 30, 2023 at 10:35:22PM -0600, Nathan Bossart wrote: > One thing on my mind is whether we should bother with the inline assembly > versions. It looks like gcc has had __atomic since 4.7.0 (2012), so I'm > not sure we gain much from them. OTOH they are pretty simple and seem > unlikely

Re: [PATCH] pg_convert improvement

2023-12-04 Thread Nathan Bossart
Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Use PGAlignedBlock instead of "char buf[BLCKSZ]" in more places

2023-12-04 Thread Andres Freund
Hi, On 2023-12-04 15:53:44 +0100, Peter Eisentraut wrote: > On 04.12.23 06:46, Michael Paquier wrote: > > On Mon, Dec 04, 2023 at 06:59:13AM +0530, Bharath Rupireddy wrote: > > > The commit 44cac934 replaced "char buf[BLCKSZ]" with PGAlignedBlock to > > > avoid issues on alignment-picky hardware.

Re: Clean up some signal usage mainly related to Windows

2023-12-04 Thread Tristan Partin
On Mon Dec 4, 2023 at 9:22 AM CST, Peter Eisentraut wrote: On 01.12.23 23:10, Tristan Partin wrote: > On Wed Jul 12, 2023 at 9:35 AM CDT, Tristan Partin wrote: >> On Wed Jul 12, 2023 at 9:31 AM CDT, Peter Eisentraut wrote: >> > On 12.07.23 16:23, Tristan Partin wrote: >> > > It has come to my

Fwd: Extensible storage manager API - SMGR hook Redux

2023-12-04 Thread Kirill Reshke
Sorry for double-posting, I accidentally replied to Matthias, not the mailing list :( -- Forwarded message - From: Kirill Reshke Date: Mon, 4 Dec 2023 at 19:46 Subject: Re: Extensible storage manager API - SMGR hook Redux To: Matthias van de Meent Hi! On Fri, 30 Jun 2023 at

Re: PATCH: Add REINDEX tag to event triggers

2023-12-04 Thread jian he
On Mon, Dec 4, 2023 at 11:00 PM Alexander Lakhin wrote: > > Hi Michael, > > 04.12.2023 04:04, Michael Paquier wrote: > > Anyway, I've been working on the patch for the last few days, and > > applied it after tweaking a bit its style, code and comments. > > Please look at the assertion failure

Re: Proposal to use JSON for Postgres Parser format

2023-12-04 Thread Matthias van de Meent
On Mon, 31 Oct 2022 at 15:56, Michel Pelletier wrote: > On Mon, Oct 31, 2022 at 6:15 AM Matthias van de Meent > wrote: >> On Mon, 31 Oct 2022 at 13:46, Alexander Korotkov >> wrote: >>> On Fri, Oct 28, 2022 at 4:27 PM Andrew Dunstan wrote: On 2022-10-27 Th 19:38, Andres Freund wrote:

Re: Synchronizing slots from primary to standby

2023-12-04 Thread Drouvot, Bertrand
Hi, On 12/4/23 6:10 AM, shveta malik wrote: On Fri, Dec 1, 2023 at 5:40 PM Nisha Moond wrote: Review for v41 patch. Thanks for the feedback. ~~~ 2. IIUC, the slotsyncworker's connection to the primary is to execute a query. Its aim is not walsender type connection, but at primary when

Re: Rename ShmemVariableCache and initialize it in more standard way

2023-12-04 Thread Tristan Partin
On Mon Dec 4, 2023 at 6:49 AM CST, Heikki Linnakangas wrote: This came up in the "Refactoring backend fork+exec code" thread recently [0], but is independent of that work: On 11/07/2023 01:50, Andres Freund wrote: >> --- a/src/backend/storage/ipc/shmem.c >> +++

Re: Proposal: In-flight explain logging

2023-12-04 Thread Ashutosh Bapat
Hi Rafael, On Sun, Dec 3, 2023 at 2:27 AM Rafael Thofehrn Castro wrote: > > Hello Maciek, > > Thanks for pointing that out. They are indeed super similar. Before I wrote > the patch I searched for > "explain" related ones. I guess I should have performed a better search. > > Comparing the

Re: pg_upgrade failing for 200+ million Large Objects

2023-12-04 Thread Kumar, Sachin
> "Tom Lane" mailto:t...@sss.pgh.pa.us>> wrote: > FWIW, I agree with Jacob's concern about it being a bad idea to let > users of pg_upgrade pass down arbitrary options to pg_dump/pg_restore. > I think we'd regret going there, because it'd hugely expand the set > of cases pg_upgrade has to deal

Re: Emitting JSON to file using COPY TO

2023-12-04 Thread Joe Conway
On 12/4/23 09:25, Andrew Dunstan wrote: On 2023-12-04 Mo 08:37, Joe Conway wrote: On 12/4/23 07:41, Andrew Dunstan wrote: On 2023-12-03 Su 20:14, Joe Conway wrote: (please don't top quote on the Postgres lists) On 12/3/23 17:38, Davin Shearer wrote: " being quoted as \\" breaks the JSON.

Re: Parallel CREATE INDEX for BRIN indexes

2023-12-04 Thread Tomas Vondra
On 12/4/23 16:00, Matthias van de Meent wrote: > On Sun, 3 Dec 2023 at 17:46, Tomas Vondra > wrote: >> On 11/30/23 18:47, Matthias van de Meent wrote: >>> ... >>> >>> I just ran some more tests in less favorable environments, and it >>> looks like I hit a bug: >>> >>> % SET

Re: Clean up some signal usage mainly related to Windows

2023-12-04 Thread Peter Eisentraut
On 01.12.23 23:10, Tristan Partin wrote: On Wed Jul 12, 2023 at 9:35 AM CDT, Tristan Partin wrote: On Wed Jul 12, 2023 at 9:31 AM CDT, Peter Eisentraut wrote: > On 12.07.23 16:23, Tristan Partin wrote: > > It has come to my attention that STDOUT_FILENO might not be portable and > > fileno(3)

Re: Parallel CREATE INDEX for BRIN indexes

2023-12-04 Thread Matthias van de Meent
On Sun, 3 Dec 2023 at 17:46, Tomas Vondra wrote: > On 11/30/23 18:47, Matthias van de Meent wrote: > > ... > > > > I just ran some more tests in less favorable environments, and it > > looks like I hit a bug: > > > > % SET max_parallel_workers = 0; > > % CREATE INDEX ... USING brin (...); > >

Re: PATCH: Add REINDEX tag to event triggers

2023-12-04 Thread Alexander Lakhin
Hi Michael, 04.12.2023 04:04, Michael Paquier wrote: Anyway, I've been working on the patch for the last few days, and applied it after tweaking a bit its style, code and comments. Please look at the assertion failure triggered when REINDEX processed by an event trigger: CREATE FUNCTION

Re: Use PGAlignedBlock instead of "char buf[BLCKSZ]" in more places

2023-12-04 Thread Peter Eisentraut
On 04.12.23 06:46, Michael Paquier wrote: On Mon, Dec 04, 2023 at 06:59:13AM +0530, Bharath Rupireddy wrote: The commit 44cac934 replaced "char buf[BLCKSZ]" with PGAlignedBlock to avoid issues on alignment-picky hardware. While it replaced most of the instances, there are still some more left.

Re: Avoid detoast overhead when possible

2023-12-04 Thread Matthias van de Meent
On Mon, 4 Dec 2023 at 14:23, wrote: > > > Hi, > > Matthias van de Meent writes: > > > On Mon, 4 Dec 2023 at 07:56, wrote: > > > ..It would also add overhead when > > we write results to disk, such as spilling merge sorts, hash join > > spills, or CTE materializations. > > > > Could you find a

Re: Synchronizing slots from primary to standby

2023-12-04 Thread Drouvot, Bertrand
Hi, On 12/4/23 4:33 AM, Zhijie Hou (Fujitsu) wrote: On Wednesday, November 29, 2023 5:55 PM Drouvot, Bertrand wrote: I think I'm fine with documenting the fact that the user should not change the failover value. But if he does change it (because at the end nothing prevents it to do so) then

Re: Emitting JSON to file using COPY TO

2023-12-04 Thread Andrew Dunstan
On 2023-12-04 Mo 08:37, Joe Conway wrote: On 12/4/23 07:41, Andrew Dunstan wrote: On 2023-12-03 Su 20:14, Joe Conway wrote: (please don't top quote on the Postgres lists) On 12/3/23 17:38, Davin Shearer wrote: " being quoted as \\" breaks the JSON. It needs to be \".  This has been my

回复: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-12-04 Thread Thomas wen
Hi orlo...@gmail.com That's good news, I think we can continue discuss for (https://commitfest.postgresql.org/43/3594/) Best regards 发件人: John Naylor 发送时间: 2023年12月4日 18:22 收件人: Pavel Borisov 抄送: Alexander Korotkov ; Tom Lane ; Alexander Lakhin ;

Re: Which parts of src/backend/nodes/print.c are used?

2023-12-04 Thread Tom Lane
Peter Eisentraut writes: > src/backend/nodes/print.c contains a number of functions that print node > types, mostly to stdout. Most of these are not actually used anywhere > in the code. Are they meant to be inserted into the code ad hoc for > debugging? Is anyone using these? Personally,

Re: Move bki file pre-processing from initdb - part 1 - initdb->genbki.pl

2023-12-04 Thread John Naylor
On Mon, Dec 4, 2023 at 5:03 PM Krishnakumar R wrote: > > Hi, > > As per discussion in [1] splitting the patch. Part1 moves replacement > logic in initdb of NAMEDATALEN, FLOAT8PASSBYVAL, SIZEOF_VOID_P, > ALIGNOF_POINTER to compile time via genbki.pl. Hi Krishnakumar, Note this comment in

Re: Emitting JSON to file using COPY TO

2023-12-04 Thread Joe Conway
On 12/4/23 07:41, Andrew Dunstan wrote: On 2023-12-03 Su 20:14, Joe Conway wrote: (please don't top quote on the Postgres lists) On 12/3/23 17:38, Davin Shearer wrote: " being quoted as \\" breaks the JSON. It needs to be \".  This has been my whole problem with COPY TO for JSON. Please

Re: Avoid detoast overhead when possible

2023-12-04 Thread zhihuifan1213
Hi, Matthias van de Meent writes: > On Mon, 4 Dec 2023 at 07:56, wrote: > ..It would also add overhead when > we write results to disk, such as spilling merge sorts, hash join > spills, or CTE materializations. > > Could you find a way to reduce this memory and IO usage when the value > is

Re: Should REINDEX be listed under DDL?

2023-12-04 Thread Isaac Morland
On Mon, 4 Dec 2023 at 02:54, Laurenz Albe wrote: > REINDEX is philosophically a maintenance command and a Postgres > > extension not in the SQL standard, so it does not really qualify as a > > DDL because it does not do in object definitions, so we could just > > delete this comment. Or could

Rename ShmemVariableCache and initialize it in more standard way

2023-12-04 Thread Heikki Linnakangas
This came up in the "Refactoring backend fork+exec code" thread recently [0], but is independent of that work: On 11/07/2023 01:50, Andres Freund wrote: --- a/src/backend/storage/ipc/shmem.c +++ b/src/backend/storage/ipc/shmem.c @@ -144,6 +144,8 @@ InitShmemAllocation(void) /*

Re: Emitting JSON to file using COPY TO

2023-12-04 Thread Andrew Dunstan
On 2023-12-03 Su 20:14, Joe Conway wrote: (please don't top quote on the Postgres lists) On 12/3/23 17:38, Davin Shearer wrote: " being quoted as \\" breaks the JSON. It needs to be \".  This has been my whole problem with COPY TO for JSON. Please validate that the output is in proper

Adding OLD/NEW support to RETURNING

2023-12-04 Thread Dean Rasheed
I have been playing around with the idea of adding support for OLD/NEW to RETURNING, partly motivated by the discussion on the MERGE RETURNING thread [1], but also because I think it would be a very useful addition for other commands (UPDATE in particular). This was discussed a long time ago [2],

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-04 Thread Amit Kapila
On Mon, Dec 4, 2023 at 5:30 PM Tomas Vondra wrote: > > On 12/4/23 12:37, Amit Kapila wrote: > > On Sat, Dec 2, 2023 at 9:52 PM Shlok Kyal wrote: > >> > >>> thread. I think you can compare the timing of regression tests in > >>> subscription, with and without the patch to show there is no > >>>

Re: Avoid detoast overhead when possible

2023-12-04 Thread Matthias van de Meent
On Mon, 4 Dec 2023 at 07:56, wrote: > 'SELECT f1(toast_col) FROM t;' will apply this code path, but nothing > gain and nothing lost. Applying this code path only when the toast > datum is accessed 1+ times needs some extra run-time effort. I don't > implement this so far, I'd like to see if I

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-04 Thread Tomas Vondra
On 12/4/23 12:37, Amit Kapila wrote: > On Sat, Dec 2, 2023 at 9:52 PM Shlok Kyal wrote: >> >>> thread. I think you can compare the timing of regression tests in >>> subscription, with and without the patch to show there is no >>> regression. And probably some tests with a large number of

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-04 Thread Amit Kapila
On Sat, Dec 2, 2023 at 9:52 PM Shlok Kyal wrote: > > > thread. I think you can compare the timing of regression tests in > > subscription, with and without the patch to show there is no > > regression. And probably some tests with a large number of tables for > > sync with very little data. > > I

Re: Failure with pgbench and --disable-thread-safety in ~v16

2023-12-04 Thread Alvaro Herrera
On 2023-Dec-04, Michael Paquier wrote: > While running some tests for 8984480b545d, I have noticed that the TAP > tests of pgbench fail when ~16 is compiled with > --disable-thread-safety: > [16:51:10.467](0.004s) not ok 227 - working \startpipeline with serializable > status (got 1 vs expected

Re: Failure with pgbench and --disable-thread-safety in ~v16

2023-12-04 Thread Alvaro Herrera
On 2023-Dec-04, Michael Paquier wrote: > While running some tests for 8984480b545d, I have noticed that the TAP > tests of pgbench fail when ~16 is compiled with > --disable-thread-safety: > [16:51:10.467](0.004s) not ok 227 - working \startpipeline with serializable > status (got 1 vs expected

Re: pg_upgrade and logical replication

2023-12-04 Thread Amit Kapila
On Fri, Dec 1, 2023 at 11:24 PM vignesh C wrote: > > The attached v22 version patch has the changes for the same. > I have made minor changes in the comments and code at various places. See and let me know if you are not happy with the changes. I think unless there are more suggestions or

table inheritance versus column compression and storage settings

2023-12-04 Thread Peter Eisentraut
I found the way that MergeAttributes() handles column compression and storage settings very weird. For example, I don't understand the purpose of this error: create table t1 (a int, b text compression pglz); create table t1a (b text compression lz4) inherits (t1); ... ERROR: 42804: column

Commitfest 2023-11 is now closed

2023-12-04 Thread John Naylor
Thank you to all who participated! Here are the stats: At start: Needs review: 210. Waiting on Author: 42. Ready for Committer: 29. Committed: 55. Withdrawn: 10. Returned with Feedback: 1. Total: 347. Today: Committed: 87. Moved to next CF: 234. Withdrawn: 14. Returned with Feedback: 9.

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-12-04 Thread John Naylor
On Mon, Dec 4, 2023 at 3:12 PM Pavel Borisov wrote: > I think this is complete and could be closed. Done.

Move bki file pre-processing from initdb - part 1 - initdb->genbki.pl

2023-12-04 Thread Krishnakumar R
Hi, As per discussion in [1] splitting the patch. Part1 moves replacement logic in initdb of NAMEDATALEN, FLOAT8PASSBYVAL, SIZEOF_VOID_P, ALIGNOF_POINTER to compile time via genbki.pl. -- Thanks and regards, Krishnakumar (KK). [Microsoft] [1]

Re: collect_corrupt_items_vacuum.patch

2023-12-04 Thread Alexander Lakhin
Hi Alexander, 04.12.2023 03:23, Alexander Korotkov wrote: I managed to reproduce this on a Linux VM. This problem should arise because in extension I don't have access to ProcArrayStruct. So, my code is iterating the whole PGPROC's array. I reimplemented the new horizon calculation function

Re: Avoid detoast overhead when possible

2023-12-04 Thread zhihuifan1213
Nikita Malakhov writes: Hi! > > There's a view from the other angle - detoast just attributes that are needed > (partial detoast), with optimized storage mechanics for JSONb. Very glad to know that, looking forward your design & patch! -- Best Regards Andy Fan

Re: [PATCH] plpython function causes server panic

2023-12-04 Thread Hao Zhang
Thanks for your reply. These patches look good to me! > The only readily-reachable error case in BeginInternalSubTransaction > is this specific one about IsInParallelMode, which was added later > than the original design and evidently with not a lot of thought or > testing. The comment for it

Re: Add pg_basetype() function to obtain a DOMAIN base type

2023-12-04 Thread John Naylor
On Thu, Sep 28, 2023 at 12:22 AM Alexander Korotkov wrote: > The one thing triggering my perfectionism is that the patch does two > syscache lookups instead of one. For an admin function used interactively, I'm not sure why that matters? Or do you see another use case?

Re: Add missing error codes to PANIC/FATAL error reports in xlogrecovery

2023-12-04 Thread Krishnakumar R
Hi, Updated the patch with ERRCODE_CLUSTER_CORRUPTED & kept ERRCODE_DATA_CORRUPTED when recovery is not consistent. > > > Hm, this one arguably is not corruption, but we still cannot > > > continue. ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE or maybe a new error > > > code? Added a

Re: A wrong comment about search_indexed_tlist_for_var

2023-12-04 Thread Richard Guo
On Sat, Dec 2, 2023 at 2:27 AM Tom Lane wrote: > Alvaro Herrera writes: > > On 2023-Dec-01, Richard Guo wrote: > >> However, this cross-check will also be performed in non-debug builds > >> ever since commit 867be9c07, which converts this check from Asserts to > >> test-and-elog. The commit

Re: Avoid detoast overhead when possible

2023-12-04 Thread Nikita Malakhov
Hi! There's a view from the other angle - detoast just attributes that are needed (partial detoast), with optimized storage mechanics for JSONb. I'm preparing a patch for it, so maybe the best results could be acquired by combining these two techniques. What do you think? -- Regards, Nikita

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-04 Thread John Naylor
On Mon, Nov 27, 2023 at 1:45 PM Masahiko Sawada wrote: > Since the variable-length values support is a big deal and would be > related to API design I'd like to discuss the API design first. Thanks for the fine summary of the issues here. [Swapping this back in my head] > RT_VALUE_TYPE >

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-12-04 Thread Pavel Borisov
On Mon, 4 Dec 2023 at 10:34, John Naylor wrote: > On Thu, Nov 30, 2023 at 4:37 PM Alexander Korotkov > wrote: > > > > On Thu, Nov 30, 2023 at 10:29 AM Pavel Borisov > wrote: > > > Agree. The fix is attached. > > > > What an oversight. > > Thank you, pushed! > > With that, is there any more

Re: Remove MSVC scripts from the tree

2023-12-04 Thread Michael Paquier
On Tue, Sep 26, 2023 at 12:17:04PM -0400, Andrew Dunstan wrote: > On 2023-09-26 Tu 01:25, NINGWEI CHEN wrote: >> hamerkop is not yet prepared for Meson builds, but we plan to work on this >> support soon. >> If we go with Meson builds exclusively right now, we will have to >> temporarily remove

Failure with pgbench and --disable-thread-safety in ~v16

2023-12-04 Thread Michael Paquier
Hi all, (Alvaro in CC.) While running some tests for 8984480b545d, I have noticed that the TAP tests of pgbench fail when ~16 is compiled with --disable-thread-safety: [16:51:10.467](0.004s) not ok 227 - working \startpipeline with serializable status (got 1 vs expected 0) [16:51:10.467](0.000s)