Re: Race condition in TransactionIdIsInProgress

2022-06-26 Thread Heikki Linnakangas
On 25/06/2022 13:10, Simon Riggs wrote: On Sat, 25 Jun 2022 at 10:18, Heikki Linnakangas wrote: On 24/06/2022 04:43, Andres Freund wrote: On 2022-06-23 22:03:27 +0300, Heikki Linnakangas wrote: In summary, I think we should: - commit and backpatch Simon's

Re: Race condition in TransactionIdIsInProgress

2022-06-25 Thread Simon Riggs
On Sat, 25 Jun 2022 at 10:18, Heikki Linnakangas wrote: > > On 24/06/2022 04:43, Andres Freund wrote: > > On 2022-06-23 22:03:27 +0300, Heikki Linnakangas wrote: > >> In summary, I think we should: > >> - commit and backpatch Simon's > >> just_remove_TransactionIdIsKnownCompleted_call.v1.patch >

Re: Race condition in TransactionIdIsInProgress

2022-06-25 Thread Heikki Linnakangas
On 24/06/2022 04:43, Andres Freund wrote: On 2022-06-23 22:03:27 +0300, Heikki Linnakangas wrote: In summary, I think we should: - commit and backpatch Simon's just_remove_TransactionIdIsKnownCompleted_call.v1.patch - fix pg_xact_status() to check TransactionIdIsInProgress() - in v15, remove

Re: Race condition in TransactionIdIsInProgress

2022-06-23 Thread Andres Freund
Hi, On 2022-06-23 22:03:27 +0300, Heikki Linnakangas wrote: > On 12/02/2022 05:42, Andres Freund wrote: > > The only reason I'm so far not succeeding in turning it into an > > isolationtester spec is that a transaction waiting for SyncRep doesn't count > > as waiting for isolationtester. > > > >

Re: Race condition in TransactionIdIsInProgress

2022-06-23 Thread Heikki Linnakangas
On 12/02/2022 05:42, Andres Freund wrote: On 2022-02-11 16:41:24 -0800, Andres Freund wrote: FWIW, I've indeed reproduced this fairly easily with such a setup. A pgbench r/w workload that's been modified to start 70 savepoints at the start shows pgbench: error: client 22 script 0 aborted in

Re: Race condition in TransactionIdIsInProgress

2022-02-12 Thread Andres Freund
Hi, On 2022-02-12 13:25:58 +, Simon Riggs wrote: > On Fri, 11 Feb 2022 at 19:08, Andres Freund wrote: > > > On 2022-02-11 13:48:59 +, Simon Riggs wrote: > > > On Fri, 11 Feb 2022 at 08:48, Simon Riggs > > > wrote: > > > > > > > > On Fri, 11 Feb 2022 at 06:11, Andres Freund wrote: > >

Re: Race condition in TransactionIdIsInProgress

2022-02-12 Thread Simon Riggs
On Fri, 11 Feb 2022 at 19:08, Andres Freund wrote: > On 2022-02-11 13:48:59 +, Simon Riggs wrote: > > On Fri, 11 Feb 2022 at 08:48, Simon Riggs > > wrote: > > > > > > On Fri, 11 Feb 2022 at 06:11, Andres Freund wrote: > > > > > > > Looks lik syncrep will make this a lot worse, because it

Re: Race condition in TransactionIdIsInProgress

2022-02-11 Thread Andres Freund
Hi, On 2022-02-11 16:41:24 -0800, Andres Freund wrote: > FWIW, I've indeed reproduced this fairly easily with such a setup. A pgbench > r/w workload that's been modified to start 70 savepoints at the start shows > > pgbench: error: client 22 script 0 aborted in command 12 query 0: ERROR: >

Re: Race condition in TransactionIdIsInProgress

2022-02-11 Thread Andres Freund
Hi, On 2022-02-10 22:11:38 -0800, Andres Freund wrote: > Looks lik syncrep will make this a lot worse, because it can drastically > increase the window between the TransactionIdCommitTree() and > ProcArrayEndTransaction() due to the SyncRepWaitForLSN() inbetween. But at > least it might make it

Re: Race condition in TransactionIdIsInProgress

2022-02-11 Thread Andres Freund
Hi, On 2022-02-11 13:48:59 +, Simon Riggs wrote: > On Fri, 11 Feb 2022 at 08:48, Simon Riggs > wrote: > > > > On Fri, 11 Feb 2022 at 06:11, Andres Freund wrote: > > > > > Looks lik syncrep will make this a lot worse, because it can drastically > > > increase the window between the

Re: Race condition in TransactionIdIsInProgress

2022-02-11 Thread Simon Riggs
On Fri, 11 Feb 2022 at 08:48, Simon Riggs wrote: > > On Fri, 11 Feb 2022 at 06:11, Andres Freund wrote: > > > Looks lik syncrep will make this a lot worse, because it can drastically > > increase the window between the TransactionIdCommitTree() and > > ProcArrayEndTransaction() due to the

Re: Race condition in TransactionIdIsInProgress

2022-02-11 Thread Simon Riggs
On Fri, 11 Feb 2022 at 06:11, Andres Freund wrote: > Looks lik syncrep will make this a lot worse, because it can drastically > increase the window between the TransactionIdCommitTree() and > ProcArrayEndTransaction() due to the SyncRepWaitForLSN() inbetween. But at > least it might make it

Re: Race condition in TransactionIdIsInProgress

2022-02-10 Thread Andres Freund
Hi, On 2022-02-10 21:56:09 -0800, Andres Freund wrote: > I think this may actually mean that the hot corruption problem fixed in > > commit 18b87b201f7 > Author: Andres Freund > Date: 2021-12-10 20:12:26 -0800 > > Fix possible HOT corruption when RECENTLY_DEAD changes to DEAD while >

Re: Race condition in TransactionIdIsInProgress

2022-02-10 Thread Andres Freund
Hi, On 2022-02-10 15:21:27 -0500, Robert Haas wrote: > On Thu, Feb 10, 2022 at 11:46 AM Konstantin Knizhnik > wrote: > > Postgres first records state transaction in CLOG, then removes > > transaction from procarray and finally releases locks. > > But it can happen that transaction is marked as

Re: Race condition in TransactionIdIsInProgress

2022-02-10 Thread Robert Haas
On Thu, Feb 10, 2022 at 11:46 AM Konstantin Knizhnik wrote: > Postgres first records state transaction in CLOG, then removes > transaction from procarray and finally releases locks. > But it can happen that transaction is marked as committed in CLOG, > XMAX_COMMITTED bit is set in modified tuple

Re: Race condition in TransactionIdIsInProgress

2022-02-10 Thread Alvaro Herrera
On 2022-Feb-10, Andrey Borodin wrote: > > On 10 Feb 2022, at 21:46, Konstantin Knizhnik wrote: > > As a result "t_xmin %u is uncommitted in tuple (%u,%u) to be updated in > > table" > > error is reported. > > Wow, cool, that seem to be a solution to one more mysterious > corruption thread -

Re: Race condition in TransactionIdIsInProgress

2022-02-10 Thread Andrey Borodin
> On 10 Feb 2022, at 21:46, Konstantin Knizhnik wrote: > As a result "t_xmin %u is uncommitted in tuple (%u,%u) to be updated in table" > error is reported. Wow, cool, that seem to be a solution to one more mysterious corruption thread - “reporting TID/table with corruption error" [0]. Thank

Race condition in TransactionIdIsInProgress

2022-02-10 Thread Konstantin Knizhnik
Hi hackers, Postgres first records state transaction in CLOG, then removes transaction from procarray and finally releases locks. But it can happen that transaction is marked as committed in CLOG, XMAX_COMMITTED bit is set in modified tuple but TransactionIdIsInProgress still returns true. As