Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-08-04 Thread Thomas Munro
On Sat, Aug 5, 2023 at 1:39 PM Thomas Munro wrote: > Here is a rebase over 26669757, which introduced > PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT. Oops, please disregard v7 (somehow lost a precious line of code). V8 is better. From 6544931e533aa015f39215f9c9d2df3e06700a96 Mon Sep 17 00:00:00 2001

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-08-04 Thread Thomas Munro
Here is a rebase over 26669757, which introduced PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT. I got a bit confused about why this new conflict reason didn't follow the usual ERROR->FATAL promotion rules and pinged Andres who provided: "Logical decoding slots are only acquired while performing logical

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-04-07 Thread Tom Lane
Michael Paquier writes: > On Sat, Apr 08, 2023 at 01:32:22AM +1200, Thomas Munro wrote: >> I'm hoping to get just the regex changes in ASAP, and then take a >> little bit longer on the recovery conflict patch itself (v6-0005) on >> the basis that it's bugfix work and not subject to the feature

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-04-07 Thread Michael Paquier
On Sat, Apr 08, 2023 at 01:32:22AM +1200, Thomas Munro wrote: > I'm hoping to get just the regex changes in ASAP, and then take a > little bit longer on the recovery conflict patch itself (v6-0005) on > the basis that it's bugfix work and not subject to the feature freeze. Agreed. It would be

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-04-07 Thread Thomas Munro
On Tue, Apr 4, 2023 at 1:25 AM Tom Lane wrote: > Sorry for not looking at this sooner. I am okay with the regex > changes proposed in v5-0001 through 0003, but I think you need to > take another mopup pass there. Some specific complaints: > * header comment for pg_regprefix has been falsified

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-04-03 Thread Tom Lane
Thomas Munro writes: > I think this experiment worked out pretty well. I think it's a nice > side-effect that you can see what memory the regexp subsystem is > using, and that's likely to lead to more improvements. (Why is it > limited to caching 32 entries? Why is it a linear search, not a

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-04-02 Thread Thomas Munro
On Sat, Jan 14, 2023 at 3:23 PM Thomas Munro wrote: > On Thu, Jan 5, 2023 at 2:14 PM Tom Lane wrote: > > The rcancelrequested API is something that I devised out of whole cloth > > awhile ago. It's not in Tcl's copy of the code, which AFAIK is the > > only other project using this regex engine.

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-13 Thread Thomas Munro
On Thu, Jan 5, 2023 at 2:14 PM Tom Lane wrote: > The rcancelrequested API is something that I devised out of whole cloth > awhile ago. It's not in Tcl's copy of the code, which AFAIK is the > only other project using this regex engine. I do still have vague > hopes of someday seeing the engine

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Tom Lane
Thomas Munro writes: > On Thu, Jan 5, 2023 at 11:55 AM Tom Lane wrote: >> ... But if that is the direction >> we're going to go in, we should probably revise these APIs to make them >> less odd. I'm not sure why we'd keep the REG_CANCEL error code at all. > Ah, OK. I had the impression from

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Thomas Munro
On Thu, Jan 5, 2023 at 11:55 AM Tom Lane wrote: > Andres Freund writes: > > Hm. Seems confusing for this to continue being called rcancelrequested() and > > to be called via if(CANCEL_REQUESTED()), if we're not even documenting that > > it's intended to be usable that way? > > Yeah. I'm not

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Andres Freund
Hi, On 2023-01-05 13:21:54 +1300, Thomas Munro wrote: > Right, I contemplated variations on that theme. I'd be willing to > code something like that to kick the tyres, but it seems like it would > make back-patching more painful? We're trying to fix bugs here... I think we need to accept that

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Thomas Munro
On Thu, Jan 5, 2023 at 12:33 PM Andres Freund wrote: > What about using a version of errsave() that can save FATALs too? We could > have something roughly like the ProcessInterrupts() in the proposed patch that > is used from within rcancelrequested(). But instead of actually throwing the >

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Andres Freund
Hi, On 2023-01-04 17:55:43 -0500, Tom Lane wrote: > I'm not very happy with this line of development at all, because I think we > are painting ourselves into a corner by not allowing code to detect whether > a cancel is pending without having it happen immediately. (That is, I do > not believe

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Tom Lane
Andres Freund writes: > Hm. Seems confusing for this to continue being called rcancelrequested() and > to be called via if(CANCEL_REQUESTED()), if we're not even documenting that > it's intended to be usable that way? Yeah. I'm not very happy with this line of development at all, because I

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Andres Freund
Hi, On 2023-01-04 16:46:05 +1300, Thomas Munro wrote: > postgres=# select 'x' ~ 'hello world .*'; > -[ RECORD 1 ] > ?column? | f > > postgres=# select * from pg_backend_memory_contexts where name = > 'RegexpMemoryContext'; > -[ RECORD 1 ]-+- > name |

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Andres Freund
Hi, On 2022-12-29 00:40:52 -0800, Noah Misch wrote: > Incidentally, the affected test contains comment "# DROP TABLE containing > block which standby has in a pinned buffer". The standby holds no pin at > that moment; the LOCK TABLE pins system catalog pages, but it drops every > pin it

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-03 Thread Thomas Munro
On Mon, Jan 2, 2023 at 8:38 AM Thomas Munro wrote: > On Sat, Dec 31, 2022 at 6:36 PM Noah Misch wrote: > > On Sat, Dec 31, 2022 at 10:06:53AM +1300, Thomas Munro wrote: > > > Idea #8 is a realisation that twisting oneself into a pretzel to avoid > > > having to change the regexp code or its

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-01 Thread Thomas Munro
On Sat, Dec 31, 2022 at 6:36 PM Noah Misch wrote: > On Sat, Dec 31, 2022 at 10:06:53AM +1300, Thomas Munro wrote: > > Idea #8 is a realisation that twisting oneself into a pretzel to avoid > > having to change the regexp code or its REG_CANCEL control flow may be > > a bit silly. If the only

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-12-30 Thread Noah Misch
On Sat, Dec 31, 2022 at 10:06:53AM +1300, Thomas Munro wrote: > On Thu, Dec 29, 2022 at 9:40 PM Noah Misch wrote: > > On Tue, Jul 26, 2022 at 07:22:52PM -0400, Tom Lane wrote: > > > Thomas Munro writes: > > > > ... The regular expression machinery is capable of > > > > consuming a lot of CPU,

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-12-30 Thread Thomas Munro
On Thu, Dec 29, 2022 at 9:40 PM Noah Misch wrote: > On Tue, Jul 26, 2022 at 07:22:52PM -0400, Tom Lane wrote: > > Thomas Munro writes: > > > ... The regular expression machinery is capable of > > > consuming a lot of CPU, and does CANCEL_REQUESTED(nfa->v->re) > > > frequently to avoid getting

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-12-29 Thread Noah Misch
On Tue, Jul 26, 2022 at 07:22:52PM -0400, Tom Lane wrote: > Thomas Munro writes: > > ... The regular expression machinery is capable of > > consuming a lot of CPU, and does CANCEL_REQUESTED(nfa->v->re) > > frequently to avoid getting stuck. With the patch as it stands, that > > would never be

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-07-26 Thread Tom Lane
Thomas Munro writes: > ... The regular expression machinery is capable of > consuming a lot of CPU, and does CANCEL_REQUESTED(nfa->v->re) > frequently to avoid getting stuck. With the patch as it stands, that > would never be true. Surely that can't be too hard to fix. We might have to

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-07-26 Thread Thomas Munro
On Sat, Jul 2, 2022 at 11:18 AM Andres Freund wrote: > On 2022-07-01 13:14:23 -0700, Andres Freund wrote: > > - the pg_usleep(5000) in ResolveRecoveryConflictWithVirtualXIDs() can > > completely swamp the target(s) on a busy system. This surely is > > exascerbated > > by the usleep I added

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-07-01 Thread Andres Freund
HHi, On 2022-07-01 13:14:23 -0700, Andres Freund wrote: > I saw a couple failures of 031 on CI for the meson patch - which obviously > doesn't change anything around this. However it adds a lot more distributions, > and the added ones run in docker containers on a shared host, presumably > adding

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-07-01 Thread Andres Freund
Hi, On 2022-06-21 19:33:01 -0700, Andres Freund wrote: > On 2022-06-21 17:22:05 +1200, Thomas Munro wrote: > > Problem: I saw 031_recovery_conflict.pl time out while waiting for a > > buffer pin conflict, but so far once only, on CI: > > > > https://cirrus-ci.com/task/5956804860444672 > > > >

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-06-21 Thread Andres Freund
Hi, On 2022-06-21 17:22:05 +1200, Thomas Munro wrote: > Problem: I saw 031_recovery_conflict.pl time out while waiting for a > buffer pin conflict, but so far once only, on CI: > > https://cirrus-ci.com/task/5956804860444672 > > timed out waiting for match: (?^:User was holding shared buffer

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-06-21 Thread Thomas Munro
On Wed, Jun 22, 2022 at 1:04 PM Michael Paquier wrote: > With the patch, we should always have QueryCancelPending set to false, > as long as there are no QueryCancelHoldoffCount. Perhaps an extra > assertion for QueryCancelPending could be added at the beginning of >

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-06-21 Thread Michael Paquier
On Tue, Jun 21, 2022 at 11:02:57PM +1200, Thomas Munro wrote: > On Tue, Jun 21, 2022 at 7:44 PM Michael Paquier wrote: >> The extra business with QueryCancelHoldoffCount and DoingCommandRead >> is the only addition for the snapshot, lock and tablespace conflict >> handling part. I don't see why

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-06-21 Thread Thomas Munro
On Tue, Jun 21, 2022 at 7:44 PM Michael Paquier wrote: > The extra business with QueryCancelHoldoffCount and DoingCommandRead > is the only addition for the snapshot, lock and tablespace conflict > handling part. I don't see why a reason why that could be wrong on a > close lookup. Anyway, why

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-06-21 Thread Michael Paquier
On Tue, Jun 21, 2022 at 05:22:05PM +1200, Thomas Munro wrote: > Here's one thing I got a bit confused about along the way, but it > seems the comment was just wrong: > > + /* > +* If we can abort just the current > subtransaction then we are OK > +

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-06-20 Thread Thomas Munro
Here's a new version, but there's something wrong that I haven't figured out yet (see CI link below). Here's one thing I got a bit confused about along the way, but it seems the comment was just wrong: + /* +* If we can abort just the current

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-06-15 Thread Robert Haas
On Wed, Jun 15, 2022 at 1:51 AM Michael Paquier wrote: > Handle is more consistent with the other types of interruptions in the > SIGUSR1 handler, so the name proposed in the patch in not that > confusing to me. And so does Process, in spirit with > ProcessProcSignalBarrier() and

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-06-14 Thread Michael Paquier
On Sun, May 22, 2022 at 05:10:01PM -0700, Andres Freund wrote: > On 2022-05-10 16:39:11 +1200, Thomas Munro wrote: >> Ok, in this version there's two levels of flag: >> RecoveryConflictPending, so we do nothing if that's not set, and then >> the loop over RecoveryConflictPendingReasons is moved

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-05-22 Thread Andres Freund
Hi, It'd be cool to commit and backpatch this - I'd like to re-enable the conflict tests in the backbranches, and I don't think we want to do so with this issue in place. On 2022-05-10 16:39:11 +1200, Thomas Munro wrote: > On Tue, Apr 12, 2022 at 10:50 AM Andres Freund wrote: > > On 2022-04-12

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-05-09 Thread Thomas Munro
On Tue, Apr 12, 2022 at 10:50 AM Andres Freund wrote: > On 2022-04-12 10:33:28 +1200, Thomas Munro wrote: > > Instead of bothering to create N different XXXPending variables for > > the different conflict "reasons", I used an array. Other than that, > > it's much like existing examples. > > It

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-04-11 Thread Andres Freund
Hi, On 2022-04-12 10:33:28 +1200, Thomas Munro wrote: > Huh. I wouldn't have started a separate thread for this if I'd > realised I was getting close to the cause of the CI failure... :) > Instead of bothering to create N different XXXPending variables for > the different conflict "reasons",

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-04-11 Thread Thomas Munro
On Sun, Apr 10, 2022 at 11:00 AM Andres Freund wrote: > On 2022-04-09 14:39:16 -0700, Andres Freund wrote: > > On 2022-04-09 17:00:41 -0400, Tom Lane wrote: > > > Thomas Munro writes: > > > > Unlike most "procsignal" handler routines, RecoveryConflictInterrupt() > > > > doesn't just set a

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-04-09 Thread Andres Freund
Hi, On 2022-04-09 14:39:16 -0700, Andres Freund wrote: > On 2022-04-09 17:00:41 -0400, Tom Lane wrote: > > Thomas Munro writes: > > > Unlike most "procsignal" handler routines, RecoveryConflictInterrupt() > > > doesn't just set a sig_atomic_t flag and poke the latch. Is the extra > > > stuff it

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-04-09 Thread Andres Freund
Hi, On 2022-04-09 17:00:41 -0400, Tom Lane wrote: > Thomas Munro writes: > > Unlike most "procsignal" handler routines, RecoveryConflictInterrupt() > > doesn't just set a sig_atomic_t flag and poke the latch. Is the extra > > stuff it does safe? For example, is this call stack OK (to pick one

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-04-09 Thread Tom Lane
Thomas Munro writes: > Unlike most "procsignal" handler routines, RecoveryConflictInterrupt() > doesn't just set a sig_atomic_t flag and poke the latch. Is the extra > stuff it does safe? For example, is this call stack OK (to pick one > that jumps out, but not the only one)? >

Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-04-09 Thread Thomas Munro
Hi, Unlike most "procsignal" handler routines, RecoveryConflictInterrupt() doesn't just set a sig_atomic_t flag and poke the latch. Is the extra stuff it does safe? For example, is this call stack OK (to pick one that jumps out, but not the only one)? procsignal_sigusr1_handler ->