Re: Perform streaming logical transactions by background workers and parallel apply

2023-02-02 Thread Masahiko Sawada
On Fri, Feb 3, 2023 at 12:29 PM houzj.f...@fujitsu.com wrote: > > On Friday, February 3, 2023 11:04 AM Amit Kapila > wrote: > > > > On Thu, Feb 2, 2023 at 4:52 AM Peter Smith > > wrote: > > > > > > Some minor review comments for v91-0001 > > > > > > > Pushed this yesterday after addressing

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Andres Freund
Hi, On 2023-02-03 20:34:36 +1300, Thomas Munro wrote: > What if we block signals, fork, then in the child, install the default > SIGTERM handler, then unblock, and then exec the shell? Yep. I was momentarily wondering why we'd even need to unblock signals, but while exec (et al) reset the

RE: Deadlock between logrep apply worker and tablesync worker

2023-02-02 Thread houzj.f...@fujitsu.com
On Thursday, February 2, 2023 7:21 PM Amit Kapila wrote: > > On Thu, Feb 2, 2023 at 12:05 PM houzj.f...@fujitsu.com > wrote: > > > > On Tuesday, January 31, 2023 1:07 AM vignesh C > wrote: > > > On Mon, 30 Jan 2023 at 17:30, vignesh C wrote: > > > > > > > I also tried to test the time of

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Tom Lane
Andres Freund writes: > On 2023-02-03 02:24:03 -0500, Tom Lane wrote: >> setsid(2) is required since SUSv2, so I'm not sure which systems >> are of concern here ... other than Redmond's of course. > I was thinking of windows, yes. But given the lack of fork(2), Windows requires a completely

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Tom Lane
Thomas Munro writes: > On Fri, Feb 3, 2023 at 8:35 PM Andres Freund wrote: >> we don't have a hard dependency on setsid() > FTR There are no Unixes without setsid()... Yeah. What I just got done reading in SUSv2 (1997) is "Derived from the POSIX.1-1988 standard". We need not concern

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Andres Freund
Hi, On 2023-02-03 02:24:03 -0500, Tom Lane wrote: > Andres Freund writes: > > Ugh, I think I might understand what's happening: > > > The signal arrives just after the fork() (within system()). Because we > > have all our processes configure themselves as process group leaders, > > and we

Re: In-placre persistance change of a relation

2023-02-02 Thread Heikki Linnakangas
I want to call out this part of this patch: Also this allows for the cleanup of files left behind in the crash of the transaction that created it. This is interesting to a lot wider audience than ALTER TABLE SET LOGGED/UNLOGGED. It also adds most of the complexity, with the new marker

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Thomas Munro
On Fri, Feb 3, 2023 at 8:35 PM Andres Freund wrote: > we don't have a > hard dependency on setsid() FTR There are no Unixes without setsid()... HAVE_SETSID was only left in the tree because we were discussing whether to replace it with !defined(WIN32) or whether that somehow made things more

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Andres Freund
Hi, On 2023-02-02 14:39:19 -0800, Nathan Bossart wrote: > Actually, this still doesn't really explain why we need to exit immediately > in the SIGTERM handler for restore_command. We already have handling for > when the command indicates it exited due to SIGTERM, so it should be no > problem if

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Thomas Munro
On Fri, Feb 3, 2023 at 8:24 PM Tom Lane wrote: > Andres Freund writes: > > Ugh, I think I might understand what's happening: > > > The signal arrives just after the fork() (within system()). Because we > > have all our processes configure themselves as process group leaders, > > and we signal

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Tom Lane
Andres Freund writes: > Ugh, I think I might understand what's happening: > The signal arrives just after the fork() (within system()). Because we > have all our processes configure themselves as process group leaders, > and we signal the entire process group (c.f. signal_child()), both the >

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Andres Freund
Hi, On 2023-02-02 10:23:21 +0900, Michael Paquier wrote: > On Wed, Feb 01, 2023 at 10:18:27AM -0800, Andres Freund wrote: > > On 2023-02-01 12:27:19 -0500, Tom Lane wrote: > >> Andres Freund writes: > >> The main thing that system() brings to the table is platform-specific > >> knowledge of

Re: [PATCH] Add pretty-printed XML output option

2023-02-02 Thread Jim Jones
The system somehow returns different error messages in Linux and MacOS/Windows, which was causing the cfbot to fail. SELECT xmlpretty('')::xml;   ^ -DETAIL:  line 1: chunk is not well balanced +DETAIL:  line 1: Premature end of data in tag foo line 1 Test removed in v2.

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-02 Thread Takamichi Osumi (Fujitsu)
Hi, On Friday, February 3, 2023 2:21 PM Amit Kapila wrote: > On Fri, Feb 3, 2023 at 6:41 AM Peter Smith wrote: > > On Thu, Feb 2, 2023 at 7:21 PM Takamichi Osumi (Fujitsu) > > wrote: > > > > > ... > > > > Besides, I am not sure it's a stable test to check the log. Is it > > > > possible that

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-02 Thread shveta malik
On Fri, Feb 3, 2023 at 11:50 AM shveta malik wrote: > > On Thu, Feb 2, 2023 at 5:01 PM shveta malik wrote: > > > > 2e) > When it goes to reuse flow (i.e. before walrcv_slot_snapshot), if > needed we can dump newly obtained origin_startpos also: > > ereport(DEBUG2, > (errmsg("[subid:%d]

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-02 Thread shveta malik
On Thu, Feb 2, 2023 at 5:01 PM shveta malik wrote: > > Reviewing further > Few more comments for v10-0002 and v7-0001: 1) + * need_full_snapshot + * if true, create a snapshot able to read all tables, + * otherwise do not create any snapshot. + *

Re: Use windows VMs instead of windows containers on the CI

2023-02-02 Thread Andres Freund
Hi, On 2023-02-02 17:47:37 +0300, Nazir Bilal Yavuz wrote: > On 1/12/2023 3:30 AM, Andres Freund wrote: > > Yea, there's a problem where packer on windows doesn't seem to abort after a > > powershell script error out. The reason isn't yet quiete clear. I think > > Bilal > > is working on a

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-02 Thread Amit Kapila
On Fri, Feb 3, 2023 at 11:12 AM Peter Smith wrote: > > On Fri, Feb 3, 2023 at 4:21 PM Amit Kapila wrote: > > > > On Fri, Feb 3, 2023 at 6:41 AM Peter Smith wrote: > > > > > > On Thu, Feb 2, 2023 at 7:21 PM Takamichi Osumi (Fujitsu) > > > wrote: > > > > > > > ... > > > > > > > > > > > > >

Re: O(n) tasks cause lengthy startups and checkpoints

2023-02-02 Thread Nathan Bossart
rebased for cfbot -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From abbd26a3bcfcc828e196187e9f6abf6af64f3393 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 5 Jan 2022 19:24:22 + Subject: [PATCH v19 1/4] Introduce custodian. The custodian process is a new

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-02 Thread Peter Smith
On Fri, Feb 3, 2023 at 4:21 PM Amit Kapila wrote: > > On Fri, Feb 3, 2023 at 6:41 AM Peter Smith wrote: > > > > On Thu, Feb 2, 2023 at 7:21 PM Takamichi Osumi (Fujitsu) > > wrote: > > > > > ... > > > > > > > > > > Besides, I am not sure it's a stable test to check the log. Is it > > > >

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-02 Thread Amit Kapila
On Fri, Feb 3, 2023 at 8:02 AM Peter Smith wrote: > > I think there is some misunderstanding. I was not suggesting removing > the condition -- only that I thought it could be written without the > > 0 as: > > if (IsSet(supported_opts, SUBOPT_MIN_APPLY_DELAY) && > opts->min_apply_delay &&

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Nathan Bossart
On Thu, Feb 02, 2023 at 02:39:19PM -0800, Nathan Bossart wrote: > Maybe we could just > remove this exit-in-SIGTERM-handler business... I've spent some time testing this. It seems to work pretty well, but only if I keep the exit-on-SIGTERM logic in shell_restore(). Without that, I'm seeing

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-02 Thread Amit Kapila
On Fri, Feb 3, 2023 at 6:41 AM Peter Smith wrote: > > On Thu, Feb 2, 2023 at 7:21 PM Takamichi Osumi (Fujitsu) > wrote: > > > ... > > > > > > > Besides, I am not sure it's a stable test to check the log. Is it > > > possible that there's > > > no such log on a slow machine? I modified the code

Re: Deadlock between logrep apply worker and tablesync worker

2023-02-02 Thread Amit Kapila
On Thu, Feb 2, 2023 at 4:51 PM Amit Kapila wrote: > > Thanks for the tests. I also see a reduction in test time variability > with Vignesh's patch. I think we can release the locks in case the > origin is concurrently dropped as in the attached patch. I am planning > to commit this patch tomorrow

Re: Logical replication timeout problem

2023-02-02 Thread Amit Kapila
On Wed, Feb 1, 2023 at 10:04 AM Amit Kapila wrote: > > On Tue, Jan 31, 2023 at 8:24 PM Ashutosh Bapat > wrote: > > > > On Tue, Jan 31, 2023 at 5:12 PM Amit Kapila wrote: > > > > > > On Tue, Jan 31, 2023 at 5:03 PM Ashutosh Bapat > > > wrote: > > > > > > > > On Tue, Jan 31, 2023 at 4:58 PM Amit

Re: Index problem Need an urgent fix

2023-02-02 Thread Julien Rouhaud
hi, Le ven. 3 févr. 2023 à 11:33, chanukya SDS a écrit : > Hey All, > > I have a query like below > > > SELECT * FROM data WHERE val=trunc(val) AND > acc_id='kfd50ed6-0bc3-44a9-881f-ec89713fdd80'::uuid ORDER BY ct DESC LIMIT > 10; > > table structure is > > data > (id uuid, > c_id uuid, >

Re: Index problem Need an urgent fix

2023-02-02 Thread Bharath Rupireddy
On Fri, Feb 3, 2023 at 9:03 AM chanukya SDS wrote: > > Hey All, > > I have a query like below > > SELECT * FROM data WHERE val=trunc(val) AND > acc_id='kfd50ed6-0bc3-44a9-881f-ec89713fdd80'::uuid ORDER BY ct DESC LIMIT 10; > > table structure is > > data > (id uuid, > c_id uuid, > acc_id

Re: when the startup process doesn't (logging startup delays)

2023-02-02 Thread Bharath Rupireddy
On Fri, Feb 3, 2023 at 2:29 AM Robert Haas wrote: > Thanks for looking at this. > On Tue, Nov 22, 2022 at 6:05 AM Bharath Rupireddy > wrote: > > If we place just the Assert(!StandbyMode); in > > enable_startup_progress_timeout(), it fails for > > begin_startup_progress_phase() in

Re: How to write a new tuple into page?

2023-02-02 Thread Bharath Rupireddy
On Thu, Feb 2, 2023 at 7:31 PM jack...@gmail.com wrote: > > Hi, I'm trying to construct a new tuple type, that's not heaptuple, > When I get a tupleTableSlot, I will get data info from it and then I > will constuct a new tuple, and now I need to put it into a physical > page, how should I do?

Index problem Need an urgent fix

2023-02-02 Thread chanukya SDS
Hey All, I have a query like below SELECT * FROM data WHERE val=trunc(val) AND acc_id='kfd50ed6-0bc3-44a9-881f-ec89713fdd80'::uuid ORDER BY ct DESC LIMIT 10; table structure is data (id uuid, c_id uuid, acc_id uuid, val numeric, ct timestamptz); Can you please help me to write an

RE: Perform streaming logical transactions by background workers and parallel apply

2023-02-02 Thread houzj.f...@fujitsu.com
On Friday, February 3, 2023 11:04 AM Amit Kapila wrote: > > On Thu, Feb 2, 2023 at 4:52 AM Peter Smith > wrote: > > > > Some minor review comments for v91-0001 > > > > Pushed this yesterday after addressing your comments! Thanks for pushing. Currently, we have two remaining patches which we

Re: Perform streaming logical transactions by background workers and parallel apply

2023-02-02 Thread Amit Kapila
On Thu, Feb 2, 2023 at 4:52 AM Peter Smith wrote: > > Some minor review comments for v91-0001 > Pushed this yesterday after addressing your comments! -- With Regards, Amit Kapila.

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-02 Thread Peter Smith
Here are my review comments for patch v26-0001. On Thu, Feb 2, 2023 at 7:18 PM Takamichi Osumi (Fujitsu) wrote: > > Hi, > > On Wednesday, February 1, 2023 1:37 PM Peter Smith > wrote: > > Here are my review comments for the patch v25-0001. > Thank you for your review ! > > > 8. > > + if

Re: heapgettup refactoring

2023-02-02 Thread David Rowley
On Fri, 3 Feb 2023 at 06:23, Melanie Plageman wrote: > Your code also switches to saving the OffsetNumber -- just in a separate > variable of OffsetNumber type. I am fine with this if it your rationale > is that it is not a good idea to store a smaller number in a larger > datatype. However, the

Re: CI and test improvements

2023-02-02 Thread Thomas Munro
On Thu, Feb 2, 2023 at 2:12 PM Tom Lane wrote: > Thomas Munro writes: > > Some observations: > > So what should our policy be on when to roll the CI image forward? I > > guess around New Year/now (~6 months after release) is a good time and > > we should just do it. Anyone got a reason why we

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-02 Thread Peter Smith
On Thu, Feb 2, 2023 at 7:21 PM Takamichi Osumi (Fujitsu) wrote: > ... > > > > Besides, I am not sure it's a stable test to check the log. Is it possible > > that there's > > no such log on a slow machine? I modified the code to sleep 1s at the > > beginning > > of apply_dispatch(), then the new

Re: Support logical replication of DDLs

2023-02-02 Thread Peter Smith
Here are some review comments for patch v63-0001. == General 1. (This is not really a review comment - more just an observation...) This patch seemed mostly like an assortment of random changes that don't seem to have anything in common except that some *later* patches of this set are

Re: Remove some useless casts to (void *)

2023-02-02 Thread Corey Huinker
On Thu, Feb 2, 2023 at 5:22 PM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > I have found that in some corners of the code some calls to standard C > functions are decorated with casts to (void *) for no reason, and this > code pattern then gets copied around. I have gone

Re: Amcheck verification of GiST and GIN

2023-02-02 Thread Peter Geoghegan
On Thu, Feb 2, 2023 at 12:56 PM Nikolay Samokhvalov wrote: > I already realized my mistake – indeed, having multiple errors for 1 index > doesn't seem to be super practically helpful. I wouldn't mind supporting it if the cost wasn't too high. But I believe that it's not a good trade-off. >> I

Re: Remove unused code related to unknown type

2023-02-02 Thread Tom Lane
Peter Eisentraut writes: > Here is a patch that removes some unused leftovers from commit > cfd9be939e9c516243c5b6a49ad1e1a9a38f1052 (old). Ugh. Those are outright wrong now, aren't they? Better nuke 'em. regards, tom lane

Re: Move defaults toward ICU in 16?

2023-02-02 Thread Tom Lane
Thomas Munro writes: > It's still important to have libc support as an option, though, > because it's a totally reasonable thing to want sort order to agree > with the "sort" command on the same host, and you are willing to deal > with all the complexities that we're trying to escape. Yeah. I

Re: Move defaults toward ICU in 16?

2023-02-02 Thread Peter Geoghegan
On Thu, Feb 2, 2023 at 2:15 PM Thomas Munro wrote: > Sure, there could be a clean-room implementation that replaces it in > some sense (just as there is a Java implementation) but it would very > likely be "the same" because the real thing we're buying here is the > set of algorithms and data

Re: Underscores in numeric literals

2023-02-02 Thread Peter Eisentraut
On 31.01.23 17:09, Dean Rasheed wrote: On Tue, 31 Jan 2023 at 15:28, Peter Eisentraut wrote: Did you have any thoughts about what to do with the float types? I guess we could handle those in a separate patch? I was assuming that we'd do nothing for float types, because anything we did

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Nathan Bossart
On Thu, Feb 02, 2023 at 02:01:13PM -0800, Nathan Bossart wrote: > I've been digging into the history here. This e-mail seems to have the > most context [0]. IIUC this was intended to prevent "fast" shutdowns from > escalating to "immediate" shutdowns because the restore command died >

Remove some useless casts to (void *)

2023-02-02 Thread Peter Eisentraut
I have found that in some corners of the code some calls to standard C functions are decorated with casts to (void *) for no reason, and this code pattern then gets copied around. I have gone through and cleaned this up a bit, in the attached patches. The involved functions are: repalloc,

Re: Move defaults toward ICU in 16?

2023-02-02 Thread Thomas Munro
On Fri, Feb 3, 2023 at 5:31 AM Jeff Davis wrote: > On Thu, 2023-02-02 at 08:44 -0500, Robert Haas wrote: > > On Thu, Feb 2, 2023 at 8:13 AM Jeff Davis wrote: > > > If we don't want to nudge users toward ICU, is it because we are > > > waiting for something, or is there a lack of consensus that

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Nathan Bossart
On Thu, Feb 02, 2023 at 04:14:54PM -0500, Robert Haas wrote: > + /* > +* When exitOnSigterm is set and we are in the startup process, use > the > +* special wrapper for system() that enables exiting immediately upon > +* receiving SIGTERM. This ensures we can break

Remove unused code related to unknown type

2023-02-02 Thread Peter Eisentraut
Here is a patch that removes some unused leftovers from commit cfd9be939e9c516243c5b6a49ad1e1a9a38f1052 (old).From 638fa48b0852bf501069b4a6993454c011d9d42a Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Thu, 2 Feb 2023 22:53:26 +0100 Subject: [PATCH] Remove unused code related to unknown

Re: Prevent accidental whole-table DELETEs and UPDATEs

2023-02-02 Thread Tom Lane
Nikolay Samokhvalov writes: > In many cases, a DELETE or UPDATE not having a WHERE clause (or having it > with a condition matching all rows in the table) is a sign of some kind of > mistake, leading to accidental data loss, performance issues, producing a > lot of dead tuples, and so on.

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Robert Haas
On Thu, Feb 2, 2023 at 3:10 PM Nathan Bossart wrote: > Hm. I don't know if we want to encourage further use of > in_restore_command since it seems to be prone to misuse. Here's a v2 that > demonstrateѕ Tom's idea (bikeshedding on names and comments is welcome). I > personally like this

Re: when the startup process doesn't (logging startup delays)

2023-02-02 Thread Robert Haas
On Tue, Nov 22, 2022 at 6:05 AM Bharath Rupireddy wrote: > If we place just the Assert(!StandbyMode); in > enable_startup_progress_timeout(), it fails for > begin_startup_progress_phase() in ResetUnloggedRelations() because the > InitWalRecovery(), that sets StandbyMode to true, is called before

Re: Amcheck verification of GiST and GIN

2023-02-02 Thread Nikolay Samokhvalov
On Thu, Feb 2, 2023 at 12:43 PM Peter Geoghegan wrote: > I agree that this matters at the level of whole indexes. > I already realized my mistake – indeed, having multiple errors for 1 index doesn't seem to be super practically helpful. > I think that that problem should be solved at a higher

Re: Amcheck verification of GiST and GIN

2023-02-02 Thread Peter Geoghegan
On Thu, Feb 2, 2023 at 12:31 PM Nikolay Samokhvalov wrote: > I understand your thoughts (I think) and agree with them, but at least one > scenario where I do want to see *all* errors is corruption prevention – > running > amcheck in lower environments, not in production, to predict and prevent

[PATCH] Add pretty-printed XML output option

2023-02-02 Thread Jim Jones
Hi, This small patch introduces a XML pretty print function. It basically takes advantage of the indentation feature of xmlDocDumpFormatMemory from libxml2 to format XML strings. postgres=# SELECT xmlpretty('id="z">42');     xmlpretty -- +   

Re: Amcheck verification of GiST and GIN

2023-02-02 Thread Nikolay Samokhvalov
On Thu, Feb 2, 2023 at 12:15 PM Peter Geoghegan wrote: > On Thu, Feb 2, 2023 at 11:51 AM Peter Geoghegan wrote: > ... > Admittedly there is some value in seeing multiple WARNINGs to true > experts that are performing some kind of forensic analysis, but that > doesn't seem worth it to me -- I'm

Re: transition tables and UPDATE

2023-02-02 Thread Corey Huinker
> > > even uglier than what I already had. So yeah, I think it might be > useful if we had a way to inject a counter or something in there. > > This came up for me when I was experimenting with making the referential integrity triggers fire on statements rather than rows. Doing so has the

a very minor bug and a couple of comment changes for basebackup.c

2023-02-02 Thread Robert Haas
Here are a few small patches for basebackup.c: 0001 fixes what I believe to be a slight logical error in sendFile(), introduced by me during the v15 development cycle when I introduced the bbsink abstraction. I believe that it is theoretically possible for this to cause an assertion failure,

Re: run pgindent on a regular basis / scripted manner

2023-02-02 Thread Andrew Dunstan
On 2023-02-02 Th 10:00, Tom Lane wrote: > Andrew Dunstan writes: >> Sure. There's probably some work that could still be done in this area >> too, such as making pgperltidy work similarly to how we've now make >> pgindent work. > Perhaps. But before we commit to that, I'd like to see some

Prevent accidental whole-table DELETEs and UPDATEs

2023-02-02 Thread Nikolay Samokhvalov
In many cases, a DELETE or UPDATE not having a WHERE clause (or having it with a condition matching all rows in the table) is a sign of some kind of mistake, leading to accidental data loss, performance issues, producing a lot of dead tuples, and so on. Recently, this topic was again discussed [1]

Re: Amcheck verification of GiST and GIN

2023-02-02 Thread Peter Geoghegan
On Thu, Feb 2, 2023 at 11:51 AM Peter Geoghegan wrote: > I also have some questions about the verification functionality itself: I forgot to include another big concern here: * Why are there only WARNINGs, never ERRORs here? It's far more likely that you'll run into problems when running

Re: Weird failure with latches in curculio on v15

2023-02-02 Thread Nathan Bossart
On Thu, Feb 02, 2023 at 01:24:15PM +0900, Michael Paquier wrote: > On Wed, Feb 01, 2023 at 09:34:44PM -0500, Tom Lane wrote: >> I was vaguely wondering about removing both of those functions >> in favor of an integrated function that does a system() call >> with those things before and after it. >

Re: Amcheck verification of GiST and GIN

2023-02-02 Thread Peter Geoghegan
On Fri, Jan 13, 2023 at 8:15 PM Andrey Borodin wrote: > (v21 of patch series) I can see why the refactoring patch is necessary overall, but I have some concerns about the details. More specifically: * PageGetItemIdCareful() doesn't seem like it needs to be moved to amcheck.c and generalized to

Re: recovery modules

2023-02-02 Thread Nathan Bossart
On Thu, Feb 02, 2023 at 02:34:17PM +0900, Michael Paquier wrote: > Okay, the changes done here look straight-forward seen from here. I > got one small-ish comment. > > +basic_archive_startup(ArchiveModuleState *state) > +{ > + BasicArchiveData *data = palloc0(sizeof(BasicArchiveData)); > >

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2023-02-02 Thread Nathan Bossart
On Thu, Feb 02, 2023 at 06:59:51AM -0800, Andres Freund wrote: > On 2022-09-20 11:32:02 -0700, Nathan Bossart wrote: >> Note that this change also disallows XMAX_COMMITTED together with >> the special pre-v9.3 locked-only bit pattern that >> HEAP_XMAX_IS_LOCKED_ONLY checks for. This locked-only

Re: heapgettup refactoring

2023-02-02 Thread Melanie Plageman
On Thu, Feb 02, 2023 at 07:00:37PM +1300, David Rowley wrote: > I've attached a version of the next patch in the series. I admit to > making a couple of adjustments. I couldn't bring myself to remove the > snapshot local variable in this commit. We can deal with that when we > come to it in

Re: Move defaults toward ICU in 16?

2023-02-02 Thread Jeff Davis
On Thu, 2023-02-02 at 08:44 -0500, Robert Haas wrote: > On Thu, Feb 2, 2023 at 8:13 AM Jeff Davis wrote: > > If we don't want to nudge users toward ICU, is it because we are > > waiting for something, or is there a lack of consensus that ICU is > > actually better? > > Do you think it's better?

Re: Improve logging when using Huge Pages

2023-02-02 Thread Alvaro Herrera
On 2023-Jan-24, Justin Pryzby wrote: > On Mon, Jan 23, 2023 at 05:33:35PM -0800, Andres Freund wrote: > > I'm ok with this being a GUC, it doesn't feel elegant, but I suspect there's > > no realistic elegant answer. > Whether it's a GUC or a function, I propose to name it hugepages_active. > If

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2023-02-02 Thread Reid Thompson
Regarding the shared counter noted here, > What you could do is to have a single, imprecise, shared counter for the total > memory allocation, and have a backend-local "allowance". When the allowance is > used up, refill it from the shared counter (a single atomic op). Is there a preferred or

Re: Error "initial slot snapshot too large" in create replication slot

2023-02-02 Thread Andres Freund
Hi, On 2022-10-12 14:10:15 +0900, Michael Paquier wrote: > The discussion seems to have stopped here. As this is classified as a > bug fix, I have moved this patch to next CF, waiting on author for the > moment. FWIW, I view this more as lifting a limitation. I wouldn't want to backpatch the

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-02-02 Thread Justin Pryzby
On Wed, Feb 01, 2023 at 07:24:48PM +0100, Matthias van de Meent wrote: > On Wed, 1 Feb 2023 at 18:51, Ilya Gladyshev > wrote: > > > > 1 февр. 2023 г., в 20:27, Matthias van de Meent > > написал(а): > > > >> In HEAD we set TOTAL to whatever number partitioned table we're > >> currently

Re: pg_stat_statements and "IN" conditions

2023-02-02 Thread Dmitry Dolgov
> On Thu, Feb 02, 2023 at 03:07:27PM +0100, Alvaro Herrera wrote: > This appears to have massive conflicts. Would you please rebase? Sure, I was already mentally preparing myself to do so in the view of recent changes in query jumbling. Will post soon.

Re: pg_dump versus hash partitioning

2023-02-02 Thread Tom Lane
Alvaro Herrera writes: > ... so for --load-via-partition-root=auto (or > whatever), we need to ensure that we detect hash partitioning all the > way down from the topmost to the leaves. Yeah, that had already occurred to me, which is why I was not feeling confident about it being an easy hack in

Re: run pgindent on a regular basis / scripted manner

2023-02-02 Thread Tom Lane
Andrew Dunstan writes: > Sure. There's probably some work that could still be done in this area > too, such as making pgperltidy work similarly to how we've now make > pgindent work. Perhaps. But before we commit to that, I'd like to see some tweaks to the pgperltidy rules to make it less eager

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2023-02-02 Thread Andres Freund
Hi, On 2022-09-20 11:32:02 -0700, Nathan Bossart wrote: > Note that this change also disallows XMAX_COMMITTED together with > the special pre-v9.3 locked-only bit pattern that > HEAP_XMAX_IS_LOCKED_ONLY checks for. This locked-only bit pattern > may still be present on servers pg_upgraded from

Re: pg_dump versus hash partitioning

2023-02-02 Thread Tom Lane
Andrew Dunstan writes: > On 2023-02-01 We 20:03, Tom Lane wrote: >> Anyway, after re-reading the old thread I wonder if my first instinct >> (force --load-via-partition-root for enum hash cases only) was the >> best compromise after all. I'm not sure how painful it is to get >> pg_dump to detect

Re: pg_dump versus hash partitioning

2023-02-02 Thread Robert Haas
On Wed, Feb 1, 2023 at 6:14 PM Tom Lane wrote: > You waved your arms about inventing some new hashing infrastructure, > but it was phrased in such a way that it wasn't clear to me if that > was actually a serious proposal or not. But if it is: how will you > get around the fact that any change

Re: generic plans and "initial" pruning

2023-02-02 Thread Amit Langote
On Fri, Jan 27, 2023 at 4:01 PM Amit Langote wrote: > On Fri, Jan 20, 2023 at 12:52 PM Amit Langote wrote: > > Alright, I'll try to get something out early next week. Thanks for > > all the pointers. > > Sorry for the delay. Attached is what I've come up with so far. > > I didn't actually go

Re: Use windows VMs instead of windows containers on the CI

2023-02-02 Thread Nazir Bilal Yavuz
Hi, On 1/12/2023 3:30 AM, Andres Freund wrote: Yea, there's a problem where packer on windows doesn't seem to abort after a powershell script error out. The reason isn't yet quiete clear. I think Bilal is working on a workaround. That should be fixed now. Also, adding a patch for PG15.

Re: Temporary tables versus wraparound... again

2023-02-02 Thread Alvaro Herrera
On 2022-Dec-13, Greg Stark wrote: > So here I've done it that way. It is a bit of an unfortunate layering > since it means the heapam_handler is doing the catalog change but it > does seem inconvenient to pass relfrozenxid etc back up and have the > caller make the changes when there are no other

Re: Non-superuser subscription owners

2023-02-02 Thread Robert Haas
On Wed, Feb 1, 2023 at 3:37 PM Andres Freund wrote: > The general point is that IMO is that in many setups you should use a > user with fewer privileges than a superuser. It doesn't really matter > whether we have an ad-hoc restriction for system catalogs. More often > than not being able to

Re: Non-superuser subscription owners

2023-02-02 Thread Robert Haas
On Wed, Feb 1, 2023 at 1:09 PM Mark Dilger wrote: > > On Feb 1, 2023, at 6:43 AM, Robert Haas wrote: > > The thing I'm > > struggling to understand is: if you only want to replicate into tables > > that Alice can write, why not just make Alice own the subscription? > > For a run-as user to make

Re: pg_stat_statements and "IN" conditions

2023-02-02 Thread Alvaro Herrera
This appears to have massive conflicts. Would you please rebase? -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ "¿Cómo puedes confiar en algo que pagas y que no ves, y no confiar en algo que te dan y te lo muestran?" (Germán Poo)

Re: run pgindent on a regular basis / scripted manner

2023-02-02 Thread Andrew Dunstan
On 2023-02-02 Th 01:40, Tom Lane wrote: > Noah Misch writes: >> Regarding the concern about a pre-receive hook blocking an emergency push, >> the >> hook could approve every push where a string like "pgindent: no" appears in a >> commit message within the push. You'd still want to make the

How to write a new tuple into page?

2023-02-02 Thread jack...@gmail.com
Hi, I'm trying to construct a new tuple type, that's not heaptuple, When I get a tupleTableSlot, I will get data info from it and then I will constuct a new tuple, and now I need to put it into a physical page, how should I do? -- jack...@gmail.com

Re: Move defaults toward ICU in 16?

2023-02-02 Thread Robert Haas
On Thu, Feb 2, 2023 at 8:13 AM Jeff Davis wrote: > If we don't want to nudge users toward ICU, is it because we are > waiting for something, or is there a lack of consensus that ICU is > actually better? Do you think it's better? -- Robert Haas EDB: http://www.enterprisedb.com

Re: WAL Insertion Lock Improvements

2023-02-02 Thread Bharath Rupireddy
On Tue, Jan 24, 2023 at 7:00 PM Bharath Rupireddy wrote: > > I'm attaching the v3 patch with the above review comments addressed. > Hopefully, no memory ordering issues now. FWIW, I've added it to CF > https://commitfest.postgresql.org/42/4141/. > > Test results with the v3 patch and insert

Move defaults toward ICU in 16?

2023-02-02 Thread Jeff Davis
As a project, do we want to nudge users toward ICU as the collation provider as the best practice going forward? If so, is version 16 the right time to adjust defaults to favor ICU? * At build time, default to --with-icu (-Dicu=enabled); users who don't want ICU can specify --without-icu

Re: pg_dump versus hash partitioning

2023-02-02 Thread Andrew Dunstan
On 2023-02-01 We 20:03, Tom Lane wrote: > > Anyway, after re-reading the old thread I wonder if my first instinct > (force --load-via-partition-root for enum hash cases only) was the > best compromise after all. I'm not sure how painful it is to get > pg_dump to detect such cases, but it's

Re: [PATCH] New [relation] option engine

2023-02-02 Thread Alvaro Herrera
On 2023-Feb-02, vignesh C wrote: > On Wed, 1 Feb 2023 at 14:49, Alvaro Herrera wrote: > > Well, no feedback has been given, so I'm not sure this is a great > > outcome. In the interest of keeping it alive, I've rebased it. It > > turns out that the only conflict is with the 2022 -> 2023

Re: run pgindent on a regular basis / scripted manner

2023-02-02 Thread Amit Kapila
On Thu, Feb 2, 2023 at 5:05 PM Dean Rasheed wrote: > > On Thu, 2 Feb 2023 at 06:40, Tom Lane wrote: > > > > Noah Misch writes: > > > Regarding the concern about a pre-receive hook blocking an emergency > > > push, the > > > hook could approve every push where a string like "pgindent: no"

Re: run pgindent on a regular basis / scripted manner

2023-02-02 Thread Thomas Munro
On Fri, Feb 3, 2023 at 12:35 AM Dean Rasheed wrote: > And as someone who runs pgindent regularly, I think this will be a net > time saver, since I won't have to skip over other unrelated indent > changes all the time. +1

Re: run pgindent on a regular basis / scripted manner

2023-02-02 Thread Dean Rasheed
On Thu, 2 Feb 2023 at 06:40, Tom Lane wrote: > > Noah Misch writes: > > Regarding the concern about a pre-receive hook blocking an emergency push, > > the > > hook could approve every push where a string like "pgindent: no" appears in > > a > > commit message within the push. You'd still want

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-02 Thread shveta malik
On Wed, Feb 1, 2023 at 5:37 PM Melih Mutlu wrote: > > Hi, > Please see attached patches. > > Thanks, > -- > Melih Mutlu > Microsoft Hi Melih, Few suggestions on v10-0002-Reuse patch 1) for (int64 i = 1; i <= lastusedid; i++) { char

Re: Deadlock between logrep apply worker and tablesync worker

2023-02-02 Thread Amit Kapila
On Thu, Feb 2, 2023 at 12:05 PM houzj.f...@fujitsu.com wrote: > > On Tuesday, January 31, 2023 1:07 AM vignesh C wrote: > > On Mon, 30 Jan 2023 at 17:30, vignesh C wrote: > > > > I also tried to test the time of "src/test/subscription/t/002_types.pl" > before and after the patch(change the lock

Re: Syncrep and improving latency due to WAL throttling

2023-02-02 Thread Jakub Wartak
On Thu, Feb 2, 2023 at 11:03 AM Tomas Vondra wrote: > > I agree that some other concurrent backend's > > COMMIT could fsync it, but I was wondering if that's sensible > > optimization to perform (so that issue_fsync() would be called for > > only commit/rollback records). I can imagine a

Re: pg_dump versus hash partitioning

2023-02-02 Thread Alvaro Herrera
On 2023-Feb-01, Robert Haas wrote: > I think you can construct plausible cases where it's not just > academic. For instance, suppose I intend to use some kind of logical > replication system, not necessarily the one built into PostgreSQL, to > replicate data between two systems. Before engaging

Re: Syncrep and improving latency due to WAL throttling

2023-02-02 Thread Tomas Vondra
On 2/1/23 14:40, Jakub Wartak wrote: > On Wed, Feb 1, 2023 at 2:14 PM Tomas Vondra > wrote: > >>> Maybe we should avoid calling fsyncs for WAL throttling? (by teaching >>> HandleXLogDelayPending()->XLogFlush()->XLogWrite() to NOT to sync when >>> we are flushing just because of WAL thortting ?)

Re: pg_dump versus hash partitioning

2023-02-02 Thread Laurenz Albe
On Wed, 2023-02-01 at 17:49 -0500, Tom Lane wrote: > Robert Haas writes: > > On Wed, Feb 1, 2023 at 5:08 PM Tom Lane wrote: > > > I can agree with that argument for range or list partitioning, where > > > the partitions have some semantic meaning to the user.  I don't buy it > > > for hash

Re: [PATCH] Fix old thinko in formula to compute sweight in numeric_sqrt().

2023-02-02 Thread Dean Rasheed
On Tue, 31 Jan 2023 at 21:59, Joel Jacobson wrote: > > Nice, you managed to simplify it even further. > I think the comment and the code now are crystal clear together. > > I've tested it successfully, test report attached. > Cool. Thanks for testing. Committed. Regards, Dean

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-02 Thread shveta malik
On Thu, Feb 2, 2023 at 9:18 AM shveta malik wrote: > > > Hi Melih, > I think I am able to identify the root cause. It is not memory > corruption, but the way origin-names are stored in system-catalog > mapped to a particular relation-id before even those are created. > Apart from the problem

  1   2   >