Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-12-05 Thread Richard Guo
I've self-reviewed this patch again and I think it's now in a committable state. I'm wondering if we can mark it as 'Ready for Committer' now, or we need more review comments/feedbacks. To recap, this patch postpones reparameterization of paths until createplan.c, which would help avoid building

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-05 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Wed, 6 Dec 2023 15:11:34 +0800, Junwang Zhao wrote: > For the extra curly braces, I mean the following code block in > CopyToFormatBinaryStart: > > + {<-- I thought this is useless? > + /* Generate

Re: speed up a logical replica setup

2023-12-05 Thread Euler Taveira
On Thu, Nov 9, 2023, at 8:12 PM, Michael Paquier wrote: > On Thu, Nov 09, 2023 at 03:41:53PM +0100, Peter Eisentraut wrote: > > On 08.11.23 00:12, Michael Paquier wrote: > >> - Should the subdirectory pg_basebackup be renamed into something more > >> generic at this point? All these things are fro

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-05 Thread Junwang Zhao
On Wed, Dec 6, 2023 at 2:19 PM Sutou Kouhei wrote: > > Hi, > > In > "Re: Make COPY format extendable: Extract COPY TO format implementations" > on Wed, 6 Dec 2023 11:18:35 +0800, > Junwang Zhao wrote: > > > For the modern formats(parquet, orc, avro, etc.), will they be > > implemented as ex

Re: PATCH: Add REINDEX tag to event triggers

2023-12-05 Thread Alexander Lakhin
Hi Michael, 05.12.2023 02:45, Michael Paquier wrote: Popping a snapshot at this stage when there are no indexes has been a decision taken by the original commit in 5dc92b844e68 because we had no need for it yet, but we may do now depending on the function triggered. I have been looking at the w

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-12-05 Thread Richard Guo
On Fri, Oct 20, 2023 at 2:52 AM Alena Rybakina wrote: > Thank you for your work on the subject. > Thanks for taking an interest in this patch. > During review your patch I didn't understand why are you checking that the > variable is path and not new_path of type T_SamplePath (I highlighted it)

Re: Remove MSVC scripts from the tree

2023-12-05 Thread Shubham Khanna
On Fri, Nov 17, 2023 at 6:31 AM Michael Paquier wrote: > > On Wed, Nov 15, 2023 at 05:07:03PM -0800, Andres Freund wrote: > > On 2023-11-15 13:49:06 +0900, Michael Paquier wrote: > > Where "Windows" actually seems to solely describe visual studio? That seems > > confusing. > > Yeah, switch that to

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-05 Thread John Naylor
On Wed, Dec 6, 2023 at 4:34 AM Masahiko Sawada wrote: > > On Mon, Dec 4, 2023 at 5:21 PM John Naylor wrote: > > > Given variable-length value support, RT_GET() would have to do > > > repalloc() if the existing value size is not big enough for the new > > > value, but it cannot as the radix tree

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-05 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Wed, 6 Dec 2023 11:18:35 +0800, Junwang Zhao wrote: > For the modern formats(parquet, orc, avro, etc.), will they be > implemented as extensions or in core? I think that they should be implemented as exten

Re: Remove MSVC scripts from the tree

2023-12-05 Thread Peter Eisentraut
On 04.12.23 21:11, Andrew Dunstan wrote: I just had a look at shifting bowerbird to use meson, and it got stymied at the c99 test, which apparently doesn't compile with anything less than VS2019. If that is the case, then wouldn't that invalidate the documented claim that you can build with V

Re: Synchronizing slots from primary to standby

2023-12-05 Thread shveta malik
On Wed, Dec 6, 2023 at 10:56 AM Amit Kapila wrote: > > On Tue, Dec 5, 2023 at 7:38 PM Drouvot, Bertrand > wrote: > > > > On 12/5/23 12:32 PM, Amit Kapila wrote: > > > On Tue, Dec 5, 2023 at 10:38 AM shveta malik > > > wrote: > > >> > > >> On Mon, Dec 4, 2023 at 10:07 PM Drouvot, Bertrand > > >>

Re: reindexing an invalid index should not use ERRCODE_INDEX_CORRUPTED

2023-12-05 Thread Michael Paquier
On Sat, Nov 18, 2023 at 04:32:36PM -0800, Noah Misch wrote: > On Sat, Nov 18, 2023 at 03:09:58PM -0800, Andres Freund wrote: >> Unfortunately, there is a case of such an sqlstate that's not at all >> indicating >> corruption, namely REINDEX CONCURRENTLY when the index is invalid: >> >>

Re: Is WAL_DEBUG related code still relevant today?

2023-12-05 Thread Michael Paquier
On Mon, Dec 04, 2023 at 10:14:36AM +0900, Michael Paquier wrote: > I can add the flag in one of my nix animals if we don't have any to > provide minimal coverage, that's not an issue for me. I'd suggest to > just fix the build on Windows, this flag is a low maintenance burden. Hearing nothing abo

Re: logical decoding and replication of sequences, take 2

2023-12-05 Thread Dilip Kumar
On Sun, Dec 3, 2023 at 11:22 PM Tomas Vondra wrote: > > Some time ago I floated the idea of maybe "queuing" the sequence changes > and only replay them on the next commit, somehow. But we did ran into > problems with which snapshot to use, that I didn't know how to solve. > Maybe we should try ag

Re: RFI: Extending the TOAST Pointer

2023-12-05 Thread Michael Paquier
On Wed, May 24, 2023 at 11:50:21AM +0200, Matthias van de Meent wrote: > I think you might not have picked up on what I was arguing for, but I > agree with what you just said. > > My comment on not needing to invent a new ToastCompressionId was on > the topic of adding capabilities^ to toast point

Re: Synchronizing slots from primary to standby

2023-12-05 Thread Amit Kapila
On Tue, Dec 5, 2023 at 7:38 PM Drouvot, Bertrand wrote: > > On 12/5/23 12:32 PM, Amit Kapila wrote: > > On Tue, Dec 5, 2023 at 10:38 AM shveta malik wrote: > >> > >> On Mon, Dec 4, 2023 at 10:07 PM Drouvot, Bertrand > >> wrote: > > >>> > >>> Maybe another option could be to have the walrece

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-05 Thread Shlok Kyal
On Tue, 5 Dec 2023 at 17:18, Tomas Vondra wrote: > > On 12/5/23 08:14, Shlok Kyal wrote: > > Hi, > > > >> As for the test results, I very much doubt the differences are not > >> caused simply by random timing variations, or something like that. And I > >> don't understand what "Performance Machine

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-12-05 Thread Amit Kapila
On Wed, Dec 6, 2023 at 9:40 AM vignesh C wrote: > > On Tue, 5 Dec 2023 at 11:11, Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Sawada-san, hackers, > > > > Based on comments I made a fix. PSA the patch. > > > > Thanks for the patch, the changes look good to me. > Thanks, I have added a comment a

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

2023-12-05 Thread Peter Geoghegan
On Tue, Dec 5, 2023 at 8:06 PM Alexander Korotkov wrote: > Thank you for raising this issue. Preprocessing of btree scan keys is > normally removing the redundant scan keys. However, redundant scan > keys aren't removed when they have arguments of different types. > Please give me a bit of time

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-12-05 Thread vignesh C
On Tue, 5 Dec 2023 at 11:11, Hayato Kuroda (Fujitsu) wrote: > > Dear Sawada-san, hackers, > > Based on comments I made a fix. PSA the patch. > Thanks for the patch, the changes look good to me. Regards, Vignesh

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

2023-12-05 Thread Alexander Korotkov
Hi, Peter! On Wed, Dec 6, 2023 at 3:46 AM Peter Geoghegan wrote: > On Tue, Dec 5, 2023 at 4:41 PM Peter Geoghegan wrote: > > "In general, when inequality keys are present, the initial-positioning > > code only promises to position before the first possible match, not > > exactly at the first mat

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-05 Thread Junwang Zhao
On Wed, Dec 6, 2023 at 10:45 AM Sutou Kouhei wrote: > > Hi, > > Thanks for replying to this proposal! > > In <20231205182458.GC2757816@nathanxps13> > "Re: Make COPY format extendable: Extract COPY TO format implementations" > on Tue, 5 Dec 2023 12:24:58 -0600, > Nathan Bossart wrote: > > > I

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-05 Thread Sutou Kouhei
Hi, Thanks for replying to this proposal! In <20231205182458.GC2757816@nathanxps13> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Tue, 5 Dec 2023 12:24:58 -0600, Nathan Bossart wrote: > I think it makes sense to do this part independently, but we should be >

Re: ExecSetupTransitionCaptureState not declared in nodeModifyTable.c

2023-12-05 Thread Tom Lane
jian he writes: > static void > ExecSetupTransitionCaptureState(ModifyTableState *mtstate, EState *estate); > not declared in src/backend/executor/nodeModifyTable.c. > do we need to add the declaration? Not if the compiler's not complaining about it. We don't have a policy requiring all static

ExecSetupTransitionCaptureState not declared in nodeModifyTable.c

2023-12-05 Thread jian he
hi. static void ExecSetupTransitionCaptureState(ModifyTableState *mtstate, EState *estate); not declared in src/backend/executor/nodeModifyTable.c. do we need to add the declaration?

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

2023-12-05 Thread Peter Geoghegan
On Tue, Dec 5, 2023 at 4:41 PM Peter Geoghegan wrote: > "In general, when inequality keys are present, the initial-positioning > code only promises to position before the first possible match, not > exactly at the first match, for a forward scan; or after the last > match for a backward scan." > >

Re: Test 002_pg_upgrade fails with olddump on Windows

2023-12-05 Thread Michael Paquier
On Tue, Dec 05, 2023 at 12:00:00PM +0300, Alexander Lakhin wrote: > So removing the condition "if ($oldnode->pg_version != $newnode->pg_version)" > works here as well, but maybe switching the file mode (to preserve EOLs > produced by pg_dump) in the block "After dumping, update references ..." > is

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

2023-12-05 Thread Peter Geoghegan
On Tue, Dec 5, 2023 at 4:53 PM Tom Lane wrote: > Hmm ... I had not paid any attention to this commit, but the rationale > given in the commit message is just flat wrong: > > Imagine the ordered B-tree scan for the query like this. > > SELECT * FROM tbl WHERE col > 'a' AND col < 'b' ORDER B

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

2023-12-05 Thread Tom Lane
Peter Geoghegan writes: > As I recently went into on the thread where we've been discussing my > nbtree SAOP patch [1], there is good reason to suspect that one of the > optimizations added by commit e0b1ee17 is buggy in the presence of an > opfamily lacking the full set of cross-type comparisons.

Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

2023-12-05 Thread Peter Geoghegan
As I recently went into on the thread where we've been discussing my nbtree SAOP patch [1], there is good reason to suspect that one of the optimizations added by commit e0b1ee17 is buggy in the presence of an opfamily lacking the full set of cross-type comparisons. The attached test case confirms

Re: Change GUC hashtable to use simplehash?

2023-12-05 Thread John Naylor
On Tue, Dec 5, 2023 at 1:57 AM Jeff Davis wrote: > > On Mon, 2023-12-04 at 12:12 +0700, John Naylor wrote: > There's already a patch to use simplehash, and the API is a bit > cleaner, and there's a minor performance improvement. It seems fairly > non-controversial -- should I just proceed with th

Re: Faster "SET search_path"

2023-12-05 Thread Jeff Davis
On Mon, 2023-11-20 at 17:13 -0800, Jeff Davis wrote: > Will commit 0005 soon. Committed. > I also attached a trivial 0006 patch that uses SH_STORE_HASH. I > wasn't > able to show much benefit, though, even when there's a bucket > collision. Perhaps there just aren't enough elements to matter -- I

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Davin Shearer
> Am I understanding something incorrectly? No, you've got it. You already covered the concerns there. > That seems quite absurd, TBH. I know we've catered for some absurdity in > the CSV code (much of it down to me), so maybe we need to be liberal in > what we accept here too. IMNSHO, we should

Re: [PATCH] ltree hash functions

2023-12-05 Thread Tommy Pavlicek
Thanks. I've attached the latest version that updates the naming in line with the convention. On Mon, Dec 4, 2023 at 12:46 AM jian he wrote: > > On Fri, Dec 1, 2023 at 8:44 AM Tommy Pavlicek wrote: > > > > > > Patch updated for those comments (and a touch of cleanup in the tests) > > attached.

Re: [PATCH] plpython function causes server panic

2023-12-05 Thread Tom Lane
Hao Zhang writes: >> The only readily-reachable error case in BeginInternalSubTransaction >> is this specific one about IsInParallelMode, which was added later >> than the original design and evidently with not a lot of thought or >> testing. The comment for it speculates about whether we could g

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Joe Conway
On 12/5/23 16:20, Andrew Dunstan wrote: On 2023-12-05 Tu 16:09, Joe Conway wrote: On 12/5/23 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: and in any other case (e.g. LINES) I can't see why you would have them. Oh I didn't address this -- I saw examples in the interwebs of

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-05 Thread Masahiko Sawada
On Mon, Dec 4, 2023 at 5:21 PM John Naylor wrote: > > On Mon, Nov 27, 2023 at 1:45 PM Masahiko Sawada wrote: > > > Since the variable-length values support is a big deal and would be > > related to API design I'd like to discuss the API design first. > > Thanks for the fine summary of the issues

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Andrew Dunstan
On 2023-12-05 Tu 16:09, Joe Conway wrote: On 12/5/23 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: and in any other case (e.g. LINES) I can't see why you would have them. Oh I didn't address this -- I saw examples in the interwebs of MSSQL server I think [1] which had th

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Joe Conway
On 12/5/23 16:12, Andrew Dunstan wrote: On 2023-12-05 Tu 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: On 2023-12-05 Tu 14:50, Davin Shearer wrote: Hi Joe, In reviewing the 005 patch, I think that when used with FORCE ARRAY, we should also _imply_ FORCE ROW DELIMITER.  I

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Andrew Dunstan
On 2023-12-05 Tu 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: On 2023-12-05 Tu 14:50, Davin Shearer wrote: Hi Joe, In reviewing the 005 patch, I think that when used with FORCE ARRAY, we should also _imply_ FORCE ROW DELIMITER.  I can't envision a use case where someon

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Joe Conway
On 12/5/23 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: and in any other case (e.g. LINES) I can't see why you would have them. Oh I didn't address this -- I saw examples in the interwebs of MSSQL server I think [1] which had the non-array with commas import and export st

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Joe Conway
On 12/5/23 15:55, Andrew Dunstan wrote: On 2023-12-05 Tu 14:50, Davin Shearer wrote: Hi Joe, In reviewing the 005 patch, I think that when used with FORCE ARRAY, we should also _imply_ FORCE ROW DELIMITER.  I can't envision a use case where someone would want to use FORCE ARRAY without also

Re: UBSan pointer overflow in xlogreader.c

2023-12-05 Thread Nathan Bossart
On Tue, Dec 05, 2023 at 03:48:33PM -0500, Robert Haas wrote: > The patch LGTM, too. Thanks for investigating and writing the code. > The part about how the reserved kernel memory prevents the bug from > appearing on 32-bit systems but not 64-bit systems running in 32-bit > mode is pretty interestin

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Andrew Dunstan
On 2023-12-05 Tu 14:50, Davin Shearer wrote: Hi Joe, In reviewing the 005 patch, I think that when used with FORCE ARRAY, we should also _imply_ FORCE ROW DELIMITER.  I can't envision a use case where someone would want to use FORCE ARRAY without also using FORCE ROW DELIMITER.  I can, howe

Re: UBSan pointer overflow in xlogreader.c

2023-12-05 Thread Robert Haas
On Tue, Dec 5, 2023 at 1:04 PM Nathan Bossart wrote: > On Wed, Dec 06, 2023 at 12:03:53AM +1300, Thomas Munro wrote: > > xlogreader.c has a pointer overflow bug, as revealed by the > > combination of -fsanitize=undefined -m32, the new 039_end_of_wal.pl > > test and Robert's incremental backup patc

Re: Add checks in pg_rewind to abort if backup_label file is present

2023-12-05 Thread Heikki Linnakangas
On 05/12/2023 19:36, Krishnakumar R wrote: Hi, Please find the attached patch for $subject and associated test. Please review. Thanks for picking up this long-standing TODO! +/* + * Check if a file is present using the connection to the + * database. + */ +static bool +libpq_is_file_present(

Re: backtrace_on_internal_error

2023-12-05 Thread Tom Lane
Matthias van de Meent writes: > On Tue, 5 Dec 2023 at 19:30, Robert Haas wrote: >>> I think we should consider unconditionally emitting a backtrace when >>> an elog() is hit, instead of requiring a GUC. >> Perhaps this should be a GUC that defaults to LOG or ERROR. > I can't speak for Nathan, b

Re: backtrace_on_internal_error

2023-12-05 Thread Nathan Bossart
On Tue, Dec 05, 2023 at 07:47:25PM +0100, Matthias van de Meent wrote: > On Tue, 5 Dec 2023 at 19:30, Robert Haas wrote: >> On Tue, Dec 5, 2023 at 1:28 PM Nathan Bossart >> wrote: >> > Perhaps this should be a GUC that defaults to LOG or ERROR. >> >> Why? Sorry, I should've explained why in my

Re: Possible segfault when sending notification within a ProcessUtility hook

2023-12-05 Thread Tom Lane
Anthonin Bonnefoy writes: > This happens when a transaction block fails and a ProcessUtility hook > sends a notification during the rollback command. Why should we regard that as anything other than a bug in the ProcessUtility hook? A failed transaction should not send any notifies. > Moving no

Re: backtrace_on_internal_error

2023-12-05 Thread Robert Haas
On Tue, Dec 5, 2023 at 1:47 PM Matthias van de Meent wrote: > I can't speak for Nathan, but my reason would be that I'm not in the > habit to attach a debugger to my program to keep track of state > progression, but instead use elog() during patch development. I'm not > super stoked for getting my

Re: CREATE FUNCTION ... SEARCH { DEFAULT | SYSTEM | SESSION }

2023-12-05 Thread Jeff Davis
On Tue, 2023-12-05 at 23:22 +0700, John Naylor wrote: > Copy-paste-o of the other function name. ... > Part of this code this was copied from a function that returned int, > but this one returns a pointer. Thank you, fixed. Also, I forward-declared config_generic in guc.h to eliminate the cast.

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Davin Shearer
Hi Joe, In reviewing the 005 patch, I think that when used with FORCE ARRAY, we should also _imply_ FORCE ROW DELIMITER. I can't envision a use case where someone would want to use FORCE ARRAY without also using FORCE ROW DELIMITER. I can, however, envision a use case where someone would want FO

Re: collect_corrupt_items_vacuum.patch

2023-12-05 Thread Dmitry Koval
Hi! I agree with Alexander Lakhin about PROC_IN_VACUUM and VISHORIZON_DATA_STRICT: 1) probably manipulations with the PROC_IN_VACUUM flag in pg_visibility.c were needed for condition [1] and can be removed now; 2) the VISHORIZON_DATA_STRICT macro is probably unnecessary too (since we are not g

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Joe Conway
On 12/5/23 12:43, Davin Shearer wrote: Joe, those test cases look great and the outputs are the same as `jq`. Forward slash escaping is optional, so not escaping them in Postgres is okay. The important thing is that the software _reading_ JSON interprets both '\/' and '/' as '/'. Thanks f

Re: backtrace_on_internal_error

2023-12-05 Thread Matthias van de Meent
On Tue, 5 Dec 2023 at 19:30, Robert Haas wrote: > > On Tue, Dec 5, 2023 at 1:28 PM Nathan Bossart > wrote: > > On Tue, Dec 05, 2023 at 01:16:22PM -0500, Robert Haas wrote: > > > I think we should consider unconditionally emitting a backtrace when > > > an elog() is hit, instead of requiring a GU

Remove WIN32 conditional compilation from win32common.c

2023-12-05 Thread Tristan Partin
The file is only referenced in Meson and MSVC scripts from what I can tell, and the Meson reference is protected by a Windows check. -- Tristan Partin Neon (https://neon.tech) From bc9ffc7b0b141959a4c2a3f8b731f457ff5825c1 Mon Sep 17 00:00:00 2001 From: Tristan Partin Date: Tue, 5 Dec 2023 10:18

Re: psql not responding to SIGINT upon db reconnection

2023-12-05 Thread Tristan Partin
On Wed Nov 29, 2023 at 11:48 AM CST, Tristan Partin wrote: I am not completely in love with the code I have written. Lots of conditional compilation which makes it hard to read. Looking forward to another round of review to see what y'all think. Ok. Here is a patch which just uses select(2) wi

Re: backtrace_on_internal_error

2023-12-05 Thread Robert Haas
On Tue, Dec 5, 2023 at 1:28 PM Nathan Bossart wrote: > On Tue, Dec 05, 2023 at 01:16:22PM -0500, Robert Haas wrote: > > I think we should consider unconditionally emitting a backtrace when > > an elog() is hit, instead of requiring a GUC. Or at least any elog() > > that's not at a DEBUGn level. If

Re: backtrace_on_internal_error

2023-12-05 Thread Nathan Bossart
On Tue, Dec 05, 2023 at 01:16:22PM -0500, Robert Haas wrote: > I think we should consider unconditionally emitting a backtrace when > an elog() is hit, instead of requiring a GUC. Or at least any elog() > that's not at a DEBUGn level. If the extra output annoys anybody, that > means they're regular

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-05 Thread Nathan Bossart
On Mon, Dec 04, 2023 at 03:35:48PM +0900, Sutou Kouhei wrote: > I want to work on making COPY format extendable. I attach > the first patch for it. I'll send more patches after this is > merged. Given the current discussion about adding JSON, I think this could be a nice bit of refactoring that co

Re: backtrace_on_internal_error

2023-12-05 Thread Robert Haas
On Tue, Dec 5, 2023 at 12:40 PM Nathan Bossart wrote: > On Tue, Dec 05, 2023 at 11:55:05AM +0100, Peter Eisentraut wrote: > > Would others find this useful? > > Yes. I think I would use this pretty frequently. I think we should consider unconditionally emitting a backtrace when an elog() is hit,

Re: UBSan pointer overflow in xlogreader.c

2023-12-05 Thread Nathan Bossart
On Wed, Dec 06, 2023 at 12:03:53AM +1300, Thomas Munro wrote: > xlogreader.c has a pointer overflow bug, as revealed by the > combination of -fsanitize=undefined -m32, the new 039_end_of_wal.pl > test and Robert's incremental backup patch[1]. The bad code tests > whether an object could fit using

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Davin Shearer
Thanks for the wayback machine link Andrew. I read it, understood it, and will comply. Joe, those test cases look great and the outputs are the same as `jq`. As for forward slashes being escaped, I found this: https://stackoverflow.com/questions/1580647/json-why-are-forward-slashes-escaped . Fo

Re: backtrace_on_internal_error

2023-12-05 Thread Nathan Bossart
On Tue, Dec 05, 2023 at 11:55:05AM +0100, Peter Eisentraut wrote: > Would others find this useful? Yes. I think I would use this pretty frequently. > Any other settings or variants in this area > that should be considered while we're here? IMO it would be nice to have a way to turn on backtrace

Add checks in pg_rewind to abort if backup_label file is present

2023-12-05 Thread Krishnakumar R
Hi, Please find the attached patch for $subject and associated test. Please review. -- Thanks and Regards, Krishnakumar (KK). [Microsoft] From 80ad7293b57a2b346b0775b8f6e0e06198617154 Mon Sep 17 00:00:00 2001 From: "Krishnakumar R (KK)" Date: Tue, 5 Dec 2023 02:36:32 -0800 Subject: [PATCH v1] Ad

Possible segfault when sending notification within a ProcessUtility hook

2023-12-05 Thread Anthonin Bonnefoy
Hi, I've encountered the following segfault: #0: 0x000104e821a8 postgres`list_head(l=0x7f7f7f7f7f7f7f7f) at pg_list.h:130:17 #1: 0x000104e81c9c postgres`PreCommit_Notify at async.c:932:16 #2: 0x000104dd02f8 postgres`CommitTransaction at xact.c:2236:2 #3: 0x000104dcfc24 postgres`Co

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Joe Conway
On 12/4/23 21:54, Joe Conway wrote: On 12/4/23 17:55, Davin Shearer wrote: There are however a few characters that need to be escaped 1. |"|(double quote) 2. |\|(backslash) 3. |/|(forward slash) 4. |\b|(backspace) 5. |\f|(form feed) 6. |\n|(new line) 7. |\r|(carriage return) 8. |\t|(h

Re: logical decoding and replication of sequences, take 2

2023-12-05 Thread Tomas Vondra
On 12/5/23 13:17, Amit Kapila wrote: > ... >> I was hopeful the global hash table would be an improvement, but that >> doesn't seem to be the case. I haven't done much profiling yet, but I'd >> guess most of the overhead is due to ReorderBufferQueueSequence() >> starting and aborting a transaction

Re: CREATE FUNCTION ... SEARCH { DEFAULT | SYSTEM | SESSION }

2023-12-05 Thread John Naylor
On Tue, Dec 5, 2023 at 7:55 AM Jeff Davis wrote: > > On Tue, 2023-11-21 at 09:24 -0500, Robert Haas wrote: > > As to the second, could we somehow come > > up with an API for guc.c where you can ask for an opaque handle that > > will later allow you to do a fast-SET of a GUC? > > Yes, attached. Tha

Re: introduce dynamic shared memory registry

2023-12-05 Thread Robert Haas
On Tue, Dec 5, 2023 at 10:35 AM Joe Conway wrote: > Notwithstanding any dragons there may be, and not having actually looked > at the the patches, I love the concept! + Seems fine to me too. I haven't looked at the patches or searched for dragons either, though. -- Robert Haas EDB: http://www.e

Re: Detecting some cases of missing backup_label

2023-12-05 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > I recently mentioned to Robert (and also Heikki earlier), that I think I see a > way to detect an omitted backup_label in a relevant subset of the cases (it'd > apply to the pg_control as well, if we moved to that). Robert encouraged me > t

Re: introduce dynamic shared memory registry

2023-12-05 Thread Joe Conway
On 12/4/23 22:46, Nathan Bossart wrote: Every once in a while, I find myself wanting to use shared memory in a loadable module without requiring it to be loaded at server start via shared_preload_libraries. The DSM API offers a nice way to create and manage dynamic shared memory segments, so cre

Re: Postgres Partitions Limitations (5.11.2.3)

2023-12-05 Thread Ashutosh Bapat
On Tue, Dec 5, 2023 at 1:40 AM Laurenz Albe wrote: > > On Fri, 2023-12-01 at 18:49 +0530, Ashutosh Bapat wrote: > > On Thu, Nov 30, 2023 at 10:29 PM Laurenz Albe > > wrote: > > > > > > On Thu, 2023-11-30 at 19:22 +0530, Ashutosh Bapat wrote: > > > > May be attach the patch to hackers thread (thi

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Andrew Dunstan
On 2023-12-04 Mo 17:55, Davin Shearer wrote: Sorry about the top posting / top quoting... the link you sent me gives me a 404.  I'm not exactly sure what top quoting / posting means and Googling those terms wasn't helpful for me, but I've removed the quoting that my mail client is automatical

Re: Synchronizing slots from primary to standby

2023-12-05 Thread Drouvot, Bertrand
Hi, On 12/5/23 12:32 PM, Amit Kapila wrote: On Tue, Dec 5, 2023 at 10:38 AM shveta malik wrote: On Mon, Dec 4, 2023 at 10:07 PM Drouvot, Bertrand wrote: Maybe another option could be to have the walreceiver a way to let the slot sync worker knows that it (the walreceiver) was not able to

Re: Synchronizing slots from primary to standby

2023-12-05 Thread Drouvot, Bertrand
Hi, On 12/5/23 11:29 AM, shveta malik wrote: On Tue, Dec 5, 2023 at 2:18 PM Drouvot, Bertrand wrote: Wouldn't that make sense to move it once we are sure that walrcv_startstreaming() returns true and first_stream is true, here? " if (first_stream) +

Re: table inheritance versus column compression and storage settings

2023-12-05 Thread Peter Eisentraut
On 05.12.23 05:26, Ashutosh Bapat wrote: - When inheriting from multiple parents with different settings, an explicit setting in the child is required. When no explicit setting for child is specified, it will throw an error as it does today. Right? Yes, it would throw an error, but a different

Make attstattarget nullable

2023-12-05 Thread Peter Eisentraut
In [0] it was discussed that we could make attstattarget a nullable column, instead of always storing an explicit -1 default value for most columns. This patch implements this. This changes the pg_attribute field attstattarget into a nullable field in the variable-length part of the row. If

Re: Avoid detoast overhead when possible

2023-12-05 Thread Andy Fan
Hi Nikita Malakhov writes: > > With your setup (table created with setup.sql): You need to "set jit to off" to turn on this feature, as I state in [1] [2]. [1] https://www.postgresql.org/message-id/87ttoyihgm.fsf%40163.com [2] https://www.postgresql.org/message-id/877cltvxgt.fsf%40163.com -

Re: logical decoding and replication of sequences, take 2

2023-12-05 Thread Amit Kapila
On Sun, Dec 3, 2023 at 11:22 PM Tomas Vondra wrote: > > Thanks for the script. Are you also measuring the time it takes to > decode this using test_decoding? > > FWIW I did more comprehensive suite of tests over the weekend, with a > couple more variations. I'm attaching the updated scripts, runni

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-05 Thread Tomas Vondra
On 12/5/23 08:14, Shlok Kyal wrote: > Hi, > >> As for the test results, I very much doubt the differences are not >> caused simply by random timing variations, or something like that. And I >> don't understand what "Performance Machine Linux" is, considering those >> timings are slower than the ot

Re: Synchronizing slots from primary to standby

2023-12-05 Thread Amit Kapila
On Tue, Dec 5, 2023 at 10:38 AM shveta malik wrote: > > On Mon, Dec 4, 2023 at 10:07 PM Drouvot, Bertrand > wrote: > > > > > > > >> ~~~ > > >> 4. primary_slot_name GUC value test: > > >> > > >> When standby is started with a non-existing primary_slot_name, the > > >> wal-receiver gives an error b

UBSan pointer overflow in xlogreader.c

2023-12-05 Thread Thomas Munro
Hi, xlogreader.c has a pointer overflow bug, as revealed by the combination of -fsanitize=undefined -m32, the new 039_end_of_wal.pl test and Robert's incremental backup patch[1]. The bad code tests whether an object could fit using something like base + size <= end, which can be converted to some

backtrace_on_internal_error

2023-12-05 Thread Peter Eisentraut
We have backtrace support for server errors. You can activate that either by setting backtrace_functions or by explicitly attaching errbacktrace() to an ereport() call. I would like an additional mode that essentially triggers a backtrace anytime elog() (for internal errors) is called. This

Re: POC, WIP: OR-clause support for indexes

2023-12-05 Thread Andrei Lepikhov
Here is fresh version with the pg_dump.pl regex fixed. Now it must pass buildfarm. Under development: 1. Explanation of the general idea in comments (Robert's note) 2. Issue with hiding some optimizations (Alexander's note and example with overlapping clauses on two partial indexes) -- regard

Re: Add semi-join pushdown to postgres_fdw

2023-12-05 Thread Alexander Pyhalov
Alexander Korotkov писал(а) 2023-12-03 23:52: Hi, Alexander! On Mon, Nov 27, 2023 at 5:11 PM Alexander Pyhalov wrote: Alexander Korotkov писал(а) 2023-11-27 03:49: > Thank you for the revision. > > I've revised the patch myself. I've replaced StringInfo with > additional conds into a list of

Re: Synchronizing slots from primary to standby

2023-12-05 Thread shveta malik
On Tue, Dec 5, 2023 at 2:18 PM Drouvot, Bertrand wrote: > > Hi, > > On 12/5/23 6:08 AM, shveta malik wrote: > > On Mon, Dec 4, 2023 at 10:07 PM Drouvot, Bertrand > > wrote: > >> Maybe another option could be to have the walreceiver a way to let the > >> slot sync > >> worker knows that it (the w

Re: Avoid detoast overhead when possible

2023-12-05 Thread Nikita Malakhov
Hi, With your setup (table created with setup.sql): postgres@postgres=# explain analyze select big->'1', big->'2', big->'3', big->'5', big->'10' from b; QUERY PLAN

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-12-05 Thread Alena Rybakina
Hi! Thank you for your contribution to this thread. On 04.12.2023 05:23, jian he wrote: hi. here is my implementation based on previous discussions add a new COPY FROM flag save_error. save_error only works with non-BINARY flags. save_error is easier for me to implement, if using "save error"

Re: [PoC] Federated Authn/z with OAUTHBEARER

2023-12-05 Thread Daniel Gustafsson
> On 8 Nov 2023, at 20:00, Jacob Champion wrote: > Unfortunately the configure/Makefile build of libpq now seems to be > pulling in an `exit()` dependency in a way that Meson does not. I believe this comes from the libcurl and specifically the ntlm_wb support which is often enabled in system and

Re: pg_upgrade and logical replication

2023-12-05 Thread Amit Kapila
On Tue, Dec 5, 2023 at 10:56 AM Michael Paquier wrote: > > On Mon, Dec 04, 2023 at 04:30:49PM +0530, Amit Kapila wrote: > > I have made minor changes in the comments and code at various places. > > See and let me know if you are not happy with the changes. I think > > unless there are more suggest

Re: Test 002_pg_upgrade fails with olddump on Windows

2023-12-05 Thread Alexander Lakhin
Hi Michael, 05.12.2023 10:56, Michael Paquier wrote: Or you have used the test suite with an old installation that has the same major version as the new installation, meaning that the filtering was not happening, still you have detected some diffs? It sounds to me that we should just apply the

Re: Avoid detoast overhead when possible

2023-12-05 Thread zhihuifan1213
Nikita Malakhov writes: > Hi, > > Hmmm, I've checked this patch and can't see performance difference on a large > (2 key-value pairs) json, using toasted json column several times makes no > difference between current implementation on master (like queries mentioned > above). > > Maybe I'm

Re: Synchronizing slots from primary to standby

2023-12-05 Thread Drouvot, Bertrand
Hi, On 12/5/23 6:08 AM, shveta malik wrote: On Mon, Dec 4, 2023 at 10:07 PM Drouvot, Bertrand wrote: Maybe another option could be to have the walreceiver a way to let the slot sync worker knows that it (the walreceiver) was not able to start due to non existing replication slot on the primary

Re: Avoid detoast overhead when possible

2023-12-05 Thread Nikita Malakhov
Hi, Hmmm, I've checked this patch and can't see performance difference on a large (2 key-value pairs) json, using toasted json column several times makes no difference between current implementation on master (like queries mentioned above). Maybe I'm doing something wrong? On Tue, Dec 5, 202