Re: Bugs in TOAST handling, OID assignment and redo recovery

On 2018-Apr-11, Peter Eisentraut wrote: > On 4/10/18 06:29, Pavan Deolasee wrote: > > One of our 2ndQuadrant support customers recently reported a sudden rush > > of TOAST errors post a crash recovery, nearly causing an outage. Most > > errors read like this: > > > > ERROR: unexpected chunk

Re: Bugs in TOAST handling, OID assignment and redo recovery

On 4/18/18 02:37, Pavan Deolasee wrote: > While I agree that we should clean it up, I wonder if changing error > text would be a good idea. These errors are being reported by a very > long time and if we change the text, we might forget the knowledge about > the past reports. It's presumably

Re: Bugs in TOAST handling, OID assignment and redo recovery

On Wed, Apr 18, 2018 at 12:07:52PM +0530, Pavan Deolasee wrote: > > > On Thu, Apr 12, 2018 at 5:53 AM, Peter Eisentraut < > peter.eisentr...@2ndquadrant.com> wrote: > > On 4/10/18 06:29, Pavan Deolasee wrote: > > One of our 2ndQuadrant support customers recently reported a sudden rush >

Re: Bugs in TOAST handling, OID assignment and redo recovery

On Thu, Apr 12, 2018 at 5:53 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 4/10/18 06:29, Pavan Deolasee wrote: > > One of our 2ndQuadrant support customers recently reported a sudden rush > > of TOAST errors post a crash recovery, nearly causing an outage. Most > > errors

Re: Bugs in TOAST handling, OID assignment and redo recovery

Pavan Deolasee writes: > On Thu, Apr 12, 2018 at 1:58 AM, Tom Lane wrote: >> So while looking at this, it suddenly occurred to me that probing with >> SnapshotDirty isn't that safe for regular (non-TOAST) Oid assignment >> either. > Yeah it occurred

Re: Bugs in TOAST handling, OID assignment and redo recovery

On Thu, Apr 12, 2018 at 5:21 AM, Tom Lane wrote: > Michael Paquier writes: > > I have not really checked this thread in details, but one thing that > > strikes me is that it would be rather easy to add a TAP test based on > > the initial script that

Re: Bugs in TOAST handling, OID assignment and redo recovery

On Thu, Apr 12, 2018 at 1:58 AM, Tom Lane wrote: > So while looking at this, it suddenly occurred to me that probing with > SnapshotDirty isn't that safe for regular (non-TOAST) Oid assignment > either. > Yeah it occurred to me as well, but when I looked at the code, I

Re: Bugs in TOAST handling, OID assignment and redo recovery

On 4/10/18 06:29, Pavan Deolasee wrote: > One of our 2ndQuadrant support customers recently reported a sudden rush > of TOAST errors post a crash recovery, nearly causing an outage. Most > errors read like this: > > ERROR: unexpected chunk number 0 (expected 1) for toast value While

Re: Bugs in TOAST handling, OID assignment and redo recovery

Michael Paquier writes: > I have not really checked this thread in details, but one thing that > strikes me is that it would be rather easy to add a TAP test based on > the initial script that Pavan has sent. Would that be worth testing > cycles or not? I doubt it --- that

Re: Bugs in TOAST handling, OID assignment and redo recovery

On Wed, Apr 11, 2018 at 04:28:33PM -0400, Tom Lane wrote: > I propose therefore that the right fix does not require an API change > for GetNewOidWithIndex: we can just make it use SnapshotAny all the > time. I have not really checked this thread in details, but one thing that strikes me is that

Re: Bugs in TOAST handling, OID assignment and redo recovery

So while looking at this, it suddenly occurred to me that probing with SnapshotDirty isn't that safe for regular (non-TOAST) Oid assignment either. SnapshotDirty will consider a row dead the instant the deleting transaction has committed, but it may remain visible to other transactions for awhile

Re: Bugs in TOAST handling, OID assignment and redo recovery

On 11 April 2018 at 19:57, Tom Lane wrote: > Pavan Deolasee writes: >> Ok. I propose attached patches, more polished this time. > > I'll take these, unless some other committer is hot to do so? Please go ahead. -- Simon Riggs

Re: Bugs in TOAST handling, OID assignment and redo recovery

Pavan Deolasee writes: > Ok. I propose attached patches, more polished this time. I'll take these, unless some other committer is hot to do so? regards, tom lane

Re: Bugs in TOAST handling, OID assignment and redo recovery

On Wed, Apr 11, 2018 at 8:20 PM, Tom Lane wrote: > Pavan Deolasee writes: > > Or may be we simply err on the side of caution and scan the toast table > > with SnapshotAny while looking for a duplicate? That might prevent us > from > > reusing an OID

Re: Bugs in TOAST handling, OID assignment and redo recovery

Pavan Deolasee writes: > Or may be we simply err on the side of caution and scan the toast table > with SnapshotAny while looking for a duplicate? That might prevent us from > reusing an OID for a known-dead tuple, but should save us a second index > scan and still work.

Re: Bugs in TOAST handling, OID assignment and redo recovery

On Tue, Apr 10, 2018 at 7:24 PM, Pavan Deolasee wrote: > Hi Heikki, > > On Tue, Apr 10, 2018 at 7:07 PM, Heikki Linnakangas > wrote: > >> >>> >> It would seem more straightforward to add a snapshot parameter to >> GetNewOidWithIndex(), so that the this

Re: Bugs in TOAST handling, OID assignment and redo recovery

Hi Heikki, On Tue, Apr 10, 2018 at 7:07 PM, Heikki Linnakangas wrote: > >> > It would seem more straightforward to add a snapshot parameter to > GetNewOidWithIndex(), so that the this one caller could pass SnapshotToast, > while others pass SnapshotDirty. With your patch, you

Re: Bugs in TOAST handling, OID assignment and redo recovery

On 10/04/18 13:29, Pavan Deolasee wrote: Hello, One of our 2ndQuadrant support customers recently reported a sudden rush of TOAST errors post a crash recovery, nearly causing an outage. Most errors read like this: ERROR: unexpected chunk number 0 (expected 1) for toast value While we

Re: Bugs in TOAST handling, OID assignment and redo recovery

Am Dienstag, den 10.04.2018, 15:59 +0530 schrieb Pavan Deolasee: > One of our 2ndQuadrant support customers recently reported a sudden > rush of > TOAST errors post a crash recovery, nearly causing an outage. Most > errors > read like this: > > ERROR: unexpected chunk number 0 (expected 1) for

Bugs in TOAST handling, OID assignment and redo recovery

Hello, One of our 2ndQuadrant support customers recently reported a sudden rush of TOAST errors post a crash recovery, nearly causing an outage. Most errors read like this: ERROR: unexpected chunk number 0 (expected 1) for toast value While we could bring back the cluster to normal quickly