Re: fixing old_snapshot_threshold's time->xid mapping

2020-11-26 Thread Anastasia Lubennikova
On 06.10.2020 08:32, Thomas Munro wrote: On Fri, Sep 25, 2020 at 2:00 PM Michael Paquier wrote: On Thu, Sep 24, 2020 at 03:46:14PM -0400, Robert Haas wrote: Thomas, with respect to your part of this patch set, I wonder if we can make the functions that you're using to write tests safe enough

Re: fixing old_snapshot_threshold's time->xid mapping

2020-10-05 Thread Thomas Munro
On Fri, Sep 25, 2020 at 2:00 PM Michael Paquier wrote: > On Thu, Sep 24, 2020 at 03:46:14PM -0400, Robert Haas wrote: > > Committed. > > Cool, thanks. +1 > > Thomas, with respect to your part of this patch set, I wonder if we > > can make the functions that you're using to write tests safe enoug

Re: fixing old_snapshot_threshold's time->xid mapping

2020-09-24 Thread Michael Paquier
On Thu, Sep 24, 2020 at 03:46:14PM -0400, Robert Haas wrote: > Committed. Cool, thanks. > Thomas, with respect to your part of this patch set, I wonder if we > can make the functions that you're using to write tests safe enough > that we could add them to contrib/old_snapshot and let users run th

Re: fixing old_snapshot_threshold's time->xid mapping

2020-09-24 Thread Robert Haas
On Wed, Sep 23, 2020 at 9:16 PM Thomas Munro wrote: > LGTM. Committed. Thomas, with respect to your part of this patch set, I wonder if we can make the functions that you're using to write tests safe enough that we could add them to contrib/old_snapshot and let users run them if they want. As yo

Re: fixing old_snapshot_threshold's time->xid mapping

2020-09-24 Thread Hamid Akhtar
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: not tested Spec compliant: not tested Documentation:not tested Patch looks good to me.

Re: fixing old_snapshot_threshold's time->xid mapping

2020-09-23 Thread Thomas Munro
On Sat, Sep 19, 2020 at 12:19 PM Robert Haas wrote: > On Thu, Sep 17, 2020 at 10:40 AM Robert Haas wrote: > > Yeah, I plan to push forward with 0001 through 0003 soon, but 0001 > > needs to be revised with a PGDLLIMPORT marking, I think, and 0002 > > needs documentation. > > So here's an updated

Re: fixing old_snapshot_threshold's time->xid mapping

2020-09-18 Thread Robert Haas
On Thu, Sep 17, 2020 at 10:40 AM Robert Haas wrote: > Yeah, I plan to push forward with 0001 through 0003 soon, but 0001 > needs to be revised with a PGDLLIMPORT marking, I think, and 0002 > needs documentation. So here's an updated version of those three, with proposed commit messages, a PGDLLIM

Re: fixing old_snapshot_threshold's time->xid mapping

2020-09-17 Thread Robert Haas
On Thu, Sep 17, 2020 at 1:47 AM Michael Paquier wrote: > On Sat, Aug 15, 2020 at 10:09:15AM +1200, Thomas Munro wrote: > > And ... now that this has a commitfest entry, cfbot told me about a > > small problem in a makefile. Third time lucky? > > Still lucky since then, and the CF bot does not com

Re: fixing old_snapshot_threshold's time->xid mapping

2020-09-16 Thread Michael Paquier
On Sat, Aug 15, 2020 at 10:09:15AM +1200, Thomas Munro wrote: > And ... now that this has a commitfest entry, cfbot told me about a > small problem in a makefile. Third time lucky? Still lucky since then, and the CF bot does not complain. So... The meat of the patch is in 0003 which is fixing a

Re: fixing old_snapshot_threshold's time->xid mapping

2020-08-14 Thread Thomas Munro
On Fri, Aug 14, 2020 at 1:04 PM Thomas Munro wrote: > On Fri, Aug 14, 2020 at 12:52 PM Thomas Munro wrote: > > Here's a rebase. > > And another, since I was too slow and v6 is already in conflict... > sorry for the high frequency patches. And ... now that this has a commitfest entry, cfbot told

Re: fixing old_snapshot_threshold's time->xid mapping

2020-08-13 Thread Thomas Munro
On Fri, Aug 14, 2020 at 12:52 PM Thomas Munro wrote: > Here's a rebase. And another, since I was too slow and v6 is already in conflict... sorry for the high frequency patches. From 1ced7b8c881676f21623c048f5e9e012ca8416ec Mon Sep 17 00:00:00 2001 From: Robert Haas Date: Thu, 16 Apr 2020 09:37:3

Re: fixing old_snapshot_threshold's time->xid mapping

2020-08-13 Thread Thomas Munro
On Wed, Apr 22, 2020 at 5:39 PM Dilip Kumar wrote: > - if (ts == update_ts) > + if (ts >= update_ts) Hi Dilip, I didn't follow this bit -- could you explain? Here's a rebase. In the 0004 patch I chose to leave behind some unnecessary braces to avoid r

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-21 Thread Dilip Kumar
On Tue, Apr 21, 2020 at 4:52 PM Dilip Kumar wrote: > > On Tue, Apr 21, 2020 at 3:44 PM Thomas Munro wrote: > > > > On Tue, Apr 21, 2020 at 2:05 PM Thomas Munro wrote: > > > As before, these two apply on top of Robert's patches (or at least his > > > 0001 and 0002). > > > > While trying to figure

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-21 Thread Dilip Kumar
On Tue, Apr 21, 2020 at 3:44 PM Thomas Munro wrote: > > On Tue, Apr 21, 2020 at 2:05 PM Thomas Munro wrote: > > As before, these two apply on top of Robert's patches (or at least his > > 0001 and 0002). > > While trying to figure out if Robert's 0003 patch was correct, I added > yet another patch

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-21 Thread Thomas Munro
On Tue, Apr 21, 2020 at 2:05 PM Thomas Munro wrote: > As before, these two apply on top of Robert's patches (or at least his > 0001 and 0002). While trying to figure out if Robert's 0003 patch was correct, I added yet another patch to this stack to test it. 0006 does basic xid map maintenance th

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-20 Thread Dilip Kumar
On Mon, Apr 20, 2020 at 11:31 PM Robert Haas wrote: > > On Mon, Apr 20, 2020 at 12:10 AM Dilip Kumar wrote: > > I have started reviewing these patches. I think, the fixes looks right to > > me. > > > > + LWLockAcquire(OldSnapshotTimeMapLock, LW_SHARED); > > + mapping->head_offset = oldSnapshotC

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-20 Thread Thomas Munro
On Mon, Apr 20, 2020 at 8:02 PM Dilip Kumar wrote: > On Mon, Apr 20, 2020 at 12:29 PM Thomas Munro wrote: > > I mean I want to verify that VACUUM or heap prune actually removed a > > tuple that was visible to an old snapshot. An idea I just had: maybe > > sto_using_select.spec should check the v

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-20 Thread Robert Haas
On Mon, Apr 20, 2020 at 12:10 AM Dilip Kumar wrote: > I have started reviewing these patches. I think, the fixes looks right to me. > > + LWLockAcquire(OldSnapshotTimeMapLock, LW_SHARED); > + mapping->head_offset = oldSnapshotControl->head_offset; > + mapping->head_timestamp = oldSnapshotControl-

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-20 Thread Dilip Kumar
On Mon, Apr 20, 2020 at 12:29 PM Thomas Munro wrote: > > On Mon, Apr 20, 2020 at 6:35 PM Dilip Kumar wrote: > > On Mon, Apr 20, 2020 at 11:24 AM Thomas Munro > > wrote: > > > > > > On Sat, Apr 18, 2020 at 9:27 PM Dilip Kumar wrote: > > > > On Sat, Apr 18, 2020 at 11:47 AM Thomas Munro > > >

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-20 Thread Thomas Munro
On Mon, Apr 20, 2020 at 6:35 PM Dilip Kumar wrote: > On Mon, Apr 20, 2020 at 11:24 AM Thomas Munro wrote: > > > > On Sat, Apr 18, 2020 at 9:27 PM Dilip Kumar wrote: > > > On Sat, Apr 18, 2020 at 11:47 AM Thomas Munro > > > wrote: > > Is this an improvement? I realise that there is still nothi

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-19 Thread Dilip Kumar
On Mon, Apr 20, 2020 at 11:24 AM Thomas Munro wrote: > > On Sat, Apr 18, 2020 at 9:27 PM Dilip Kumar wrote: > > On Sat, Apr 18, 2020 at 11:47 AM Thomas Munro > > wrote: > > > I think I found another bug in MaintainOldSnapshotTimeMapping(): if > > > you make time jump by more than old_snapshot_t

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-19 Thread Thomas Munro
On Sat, Apr 18, 2020 at 9:27 PM Dilip Kumar wrote: > On Sat, Apr 18, 2020 at 11:47 AM Thomas Munro wrote: > > I think I found another bug in MaintainOldSnapshotTimeMapping(): if > > you make time jump by more than old_snapshot_threshold in one go, then > > the map gets cleared and then no early p

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-19 Thread Dilip Kumar
On Thu, Apr 16, 2020 at 10:12 PM Robert Haas wrote: > > Hi, > > I'm starting a new thread for this, because the recent discussion of > problems with old_snapshot_threshold[1] touched on a lot of separate > issues, and I think it will be too confusing if we discuss all of them > on one thread. Atta

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-18 Thread Andres Freund
Hi, On 2020-04-17 14:12:44 +1200, Thomas Munro wrote: > What about a contrib function that lets you clobber > oldSnapshotControl->current_timestamp? It looks like all times in > this system come ultimately from GetSnapshotCurrentTimestamp(), which > uses that variable to make sure that time never

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-18 Thread Dilip Kumar
On Sat, Apr 18, 2020 at 11:47 AM Thomas Munro wrote: > > On Fri, Apr 17, 2020 at 2:12 PM Thomas Munro wrote: > > What about a contrib function that lets you clobber > > oldSnapshotControl->current_timestamp? It looks like all times in > > this system come ultimately from GetSnapshotCurrentTimest

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-17 Thread Thomas Munro
On Fri, Apr 17, 2020 at 2:12 PM Thomas Munro wrote: > What about a contrib function that lets you clobber > oldSnapshotControl->current_timestamp? It looks like all times in > this system come ultimately from GetSnapshotCurrentTimestamp(), which > uses that variable to make sure that time never g

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-16 Thread Thomas Munro
On Fri, Apr 17, 2020 at 5:46 AM Andres Freund wrote: > On 2020-04-16 13:34:39 -0400, Robert Haas wrote: > > On Thu, Apr 16, 2020 at 1:14 PM Andres Freund wrote: > > > I still think we need a way to test this without waiting for hours to > > > hit various edge cases. You argued against a fixed bin

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-16 Thread Andres Freund
Hi, On 2020-04-16 13:34:39 -0400, Robert Haas wrote: > On Thu, Apr 16, 2020 at 1:14 PM Andres Freund wrote: > > I still think we need a way to test this without waiting for hours to > > hit various edge cases. You argued against a fixed binning of > > old_snapshot_threshold/100 arguing its too co

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-16 Thread Robert Haas
On Thu, Apr 16, 2020 at 1:14 PM Andres Freund wrote: > I still think we need a way to test this without waiting for hours to > hit various edge cases. You argued against a fixed binning of > old_snapshot_threshold/100 arguing its too coarse. How about a 1000 or > so? For 60 days, the current max f

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-16 Thread Andres Freund
Hi, On 2020-04-16 12:41:55 -0400, Robert Haas wrote: > I'm starting a new thread for this, because the recent discussion of > problems with old_snapshot_threshold[1] touched on a lot of separate > issues, and I think it will be too confusing if we discuss all of them > on one thread. Attached are

fixing old_snapshot_threshold's time->xid mapping

2020-04-16 Thread Robert Haas
Hi, I'm starting a new thread for this, because the recent discussion of problems with old_snapshot_threshold[1] touched on a lot of separate issues, and I think it will be too confusing if we discuss all of them on one thread. Attached are three patches. 0001 makes oldSnapshotControl "extern" ra