孔凡深(云梳) wrote:
> Hi, Antonin. I am more interested in zheap. Recently reviewing the patch you
> submitted.
> When I use pg_undodump-tool to dump the undo page chunk, I found that some
> chunk header is abnormal.
> After reading the relevant codes in
>
Hi,
On Thu, Nov 25, 2021 at 10:00 PM Antonin Houska wrote:
>
> So it should be ok if the temporary undo is managed and discarded by
> individual backends. Patch 0005 of the new series tries to do that.
The cfbot reports that at least the 001 patch doesn't apply anymore:
; >
> > > > >A transaction creates a temporary table, does some (many) changes
> > > > > and then
> > > > >gets rolled back. The undo records are being applied and it takes
> > > > > some
> > > > > tim
Thomas Munro wrote:
> On Wed, Sep 29, 2021 at 8:18 AM Antonin Houska wrote:
> > I'm just trying to use the existing infrastructure: the effect of DROP TABLE
> > also appear to be performed by the checkpointer. However I don't know why
> > the
> > unlinks need to be performed by the
On Wed, Sep 29, 2021 at 8:18 AM Antonin Houska wrote:
> I'm just trying to use the existing infrastructure: the effect of DROP TABLE
> also appear to be performed by the checkpointer. However I don't know why the
> unlinks need to be performed by the checkpointer.
For DROP TABLE, we leave an
Dmitry Dolgov <9erthali...@gmail.com> wrote:
> > On Tue, Sep 21, 2021 at 10:07:55AM +0200, Dmitry Dolgov wrote:
> > On Tue, 21 Sep 2021 09:00 Antonin Houska, wrote:
> >
> > > Dmitry Dolgov <9erthali...@gmail.com> wrote:
> > >
> > > > Yep, makes sense, thanks. I have few more questions:
> > > >
>
> On Tue, Sep 21, 2021 at 10:07:55AM +0200, Dmitry Dolgov wrote:
> On Tue, 21 Sep 2021 09:00 Antonin Houska, wrote:
>
> > Dmitry Dolgov <9erthali...@gmail.com> wrote:
> >
> > > Yep, makes sense, thanks. I have few more questions:
> > >
> > > * The use case with orphaned files is working somewhat
le, does some (many) changes
> > > > and then
> > > >gets rolled back. The undo records are being applied and it takes
> > > > some
> > > >time. Since XID of the transaction did not affect
> > > > oldestFullXidHaving
On Mon, Sep 27, 2021 at 7:43 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> Although the postgres core probably does not raise FATAL errors too often (OOM
> conditions seem to be the typical cause), I'm still not enthusiastic about
> idea that the undo feature turns such errors into PANIC.
Amit Kapila wrote:
> On Fri, Sep 24, 2021 at 4:44 PM Antonin Houska wrote:
> >
> > Amit Kapila wrote:
> >
> > > On Mon, Sep 20, 2021 at 10:24 AM Antonin Houska wrote:
> > > >
> > > > Amit Kapila wrote:
> > > >
> > > > > On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com>
>
On Fri, Sep 24, 2021 at 4:44 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> > On Mon, Sep 20, 2021 at 10:24 AM Antonin Houska wrote:
> > >
> > > Amit Kapila wrote:
> > >
> > > > On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > On
Amit Kapila wrote:
> On Mon, Sep 20, 2021 at 10:24 AM Antonin Houska wrote:
> >
> > Amit Kapila wrote:
> >
> > > On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com>
> > > wrote:
> > > >
> > > > > On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote:
> > > >
> > > >
affect
> > > oldestFullXidHavingUndo,
> > >the XID can disappear from the CLOG due to truncation.
> > >
> >
> > By above do you mean to say that in zheap code, we don't consider XIDs
> > that operate on temp table/undo for oldestFullXidHavingUndo?
>
On Mon, Sep 20, 2021 at 10:24 AM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> > On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com> wrote:
> > >
> > > > On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote:
> > >
> > > * What happened with the idea of abandoning
On Tue, 21 Sep 2021 09:00 Antonin Houska, wrote:
> Dmitry Dolgov <9erthali...@gmail.com> wrote:
>
> > Yep, makes sense, thanks. I have few more questions:
> >
> > * The use case with orphaned files is working somewhat differently after
> > the rebase on the latest master, do you observe it as
Dmitry Dolgov <9erthali...@gmail.com> wrote:
> > On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote:
> > Antonin Houska wrote:
> >
> > > Dmitry Dolgov <9erthali...@gmail.com> wrote:
> >
> > > > * By throwing at the patchset `make installcheck` I'm getting from time
> > > > to time
>
> > oldestFullXidHavingUndo,
> >the XID can disappear from the CLOG due to truncation.
> >
>
> By above do you mean to say that in zheap code, we don't consider XIDs
> that operate on temp table/undo for oldestFullXidHavingUndo?
I was referring to the code
Amit Kapila wrote:
> On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com> wrote:
> >
> > > On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote:
> >
> > * What happened with the idea of abandoning discard worker for the sake
> > of simplicity? From what I see
On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com> wrote:
>
> > On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote:
>
> * What happened with the idea of abandoning discard worker for the sake
> of simplicity? From what I see limiting everything to foreground undo
>
> On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote:
> Antonin Houska wrote:
>
> > Dmitry Dolgov <9erthali...@gmail.com> wrote:
>
> > > * By throwing at the patchset `make installcheck` I'm getting from time
> > > to time
> > > and error on the restart:
> > >
> > > TRAP:
On Thu, Sep 9, 2021 at 8:33 PM Antonin Houska wrote:
>
> The cfbot complained that the patch series no longer applies, so I've rebased
> it and also tried to make sure that the other flags become green.
>
> One particular problem was that pg_upgrade complained that "live undo data"
> remains in
> On Wed, Jun 30, 2021 at 07:41:16PM +0200, Antonin Houska wrote:
> Antonin Houska wrote:
>
> > tsunakawa.ta...@fujitsu.com wrote:
> >
> > > I'm crawling like a snail to read the patch set. Below are my first set
> > > of review comments, which are all minor.
> >
> > Thanks.
>
> I've added the
On Wed, Jun 30, 2021 at 11:10 PM Antonin Houska wrote:
>
> Antonin Houska wrote:
>
> > tsunakawa.ta...@fujitsu.com wrote:
> >
> > > I'm crawling like a snail to read the patch set. Below are my first set
> > > of review comments, which are all minor.
> >
> > Thanks.
>
> I've added the patch
Antonin Houska wrote:
> tsunakawa.ta...@fujitsu.com wrote:
>
> > I'm crawling like a snail to read the patch set. Below are my first set of
> > review comments, which are all minor.
>
> Thanks.
I've added the patch to the upcoming CF [1], so it possibly gets more review
and makes some
tsunakawa.ta...@fujitsu.com wrote:
> I'm crawling like a snail to read the patch set. Below are my first set of
> review comments, which are all minor.
Thanks. I will check your comments when I'll be preparing the next version of
the patch.
--
Antonin Houska
Web:
g_dump and pg_dumpall also output all data by
default, that seems how PostgreSQL commands behave.
(12)
+ startsegendseg
startseg and endseg are not described.
(13)
+Undo files backing undo logs in the default tablespace are stored under
...
+Undo log files contain standard page headers as des
On Wed, Feb 3, 2021 at 2:45 PM tsunakawa.ta...@fujitsu.com
wrote:
>
> From: Antonin Houska
> > not really an "ordinary patch".
> >
> > [1]
> > https://www.postgresql.org/message-id/CA%2BhUKG%2BMpzRsZFE7ChhR
> > q-Br5VYYi6mafVQ73Af7ahioWo5o8w%40mail.gmail.com
>
> I'm a bit interested in
From: Antonin Houska
> not really an "ordinary patch".
>
> [1]
> https://www.postgresql.org/message-id/CA%2BhUKG%2BMpzRsZFE7ChhR
> q-Br5VYYi6mafVQ73Af7ahioWo5o8w%40mail.gmail.com
I'm a bit interested in zheap-related topics. I'm reading this discussion to
see what I can do. (But this thread
Bruce Momjian wrote:
> On Fri, Jan 29, 2021 at 06:30:15PM +0100, Antonin Houska wrote:
> > Antonin Houska wrote:
> > > Well, on repeated run of the test I could also hit the first one. I could
> > > fix
> > > it and will post a new version of the patch (along with some other small
> > >
On Fri, Jan 29, 2021 at 06:30:15PM +0100, Antonin Houska wrote:
> Antonin Houska wrote:
> > Well, on repeated run of the test I could also hit the first one. I could
> > fix
> > it and will post a new version of the patch (along with some other small
> > changes) this week.
>
> Attached is the
Dmitry Dolgov <9erthali...@gmail.com> wrote:
> Thanks for the updated patch. As I've mentioned off the list I'm slowly
> looking through it with the intent to concentrate on undo progress
> tracking. But before I will post anything I want to mention couple of
> strange issues I see, otherwise I
> On Fri, Dec 04, 2020 at 10:22:42AM +0100, Antonin Houska wrote:
> Amit Kapila wrote:
>
> > On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
> > >
> > > Amit Kapila wrote:
> > >
> > > > On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
> > > >
> > > > If you want to track at undo
On Fri, Dec 4, 2020 at 1:50 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
>
> > The earlier version of the patch having all these ideas
> > implemented is attached
> > (Infrastructure-to-execute-pending-undo-actions and
> > Provide-interfaces-to-store-and-fetch-undo-records). The second one
Amit Kapila wrote:
> On Wed, Nov 25, 2020 at 8:00 PM Antonin Houska wrote:
> > I meant that AbortOutOfAnyTransaction should PANIC itself if it sees that
> > there is unapplied undo, so nothing changes for its callers. Do I still miss
> > something?
> >
>
> Adding PANIC in some generic
Amit Kapila wrote:
> On Wed, Nov 25, 2020 at 7:47 PM Antonin Houska wrote:
> >
> > Antonin Houska wrote:
> >
> > > Amit Kapila wrote:
> >
> > > > I think we also need to maintain oldestXidHavingUndo for CLOG
> > > > truncation and
> > > > transaction-wraparound. We can't allow CLOG
On Wed, Nov 25, 2020 at 7:47 PM Antonin Houska wrote:
>
> Antonin Houska wrote:
>
> > Amit Kapila wrote:
>
> > > I think we also need to maintain oldestXidHavingUndo for CLOG truncation
> > > and
> > > transaction-wraparound. We can't allow CLOG truncation for the transaction
> > > whose undo
On Wed, Nov 25, 2020 at 8:00 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> > On Wed, Nov 18, 2020 at 4:03 PM Antonin Houska wrote:
> > >
> > > Amit Kapila wrote:
> > >
> > > > On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
> > > > >
> > > > > Amit Kapila wrote:
> > > > >
> > > >
Amit Kapila wrote:
> On Wed, Nov 18, 2020 at 4:03 PM Antonin Houska wrote:
> >
> > Amit Kapila wrote:
> >
> > > On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
> > > >
> > > > Amit Kapila wrote:
> > > >
> > > > > On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska
> > > > > wrote:
> > > >
Antonin Houska wrote:
> Amit Kapila wrote:
> > I think we also need to maintain oldestXidHavingUndo for CLOG truncation and
> > transaction-wraparound. We can't allow CLOG truncation for the transaction
> > whose undo is not discarded as that could be required by some other
> > transaction.
>
On Wed, Nov 18, 2020 at 4:03 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> > On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
> > >
> > > Amit Kapila wrote:
> > >
> > > > On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
> > > > >
> > > > >
> > > > > No background undo
> > > >
Amit Kapila wrote:
> On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
> >
> > Amit Kapila wrote:
> >
> > > On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
> > > >
> > > >
> > > > No background undo
> > > > --
> > > >
> > > > Reduced complexity of the patch seems to
On Sun, Nov 15, 2020 at 11:24 AM Amit Kapila wrote:
>
> On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
> >
> > Amit Kapila wrote:
> >
> > > On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
> > > >
> > > >
> > > > No background undo
> > > > --
> > > >
> > > > Reduced
On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> > On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
> > >
> > >
> > > No background undo
> > > --
> > >
> > > Reduced complexity of the patch seems to be the priority at the moment.
> > > Amit
Amit Kapila wrote:
> On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
> >
> >
> > No background undo
> > --
> >
> > Reduced complexity of the patch seems to be the priority at the moment. Amit
> > suggested that cleanup of an orphaned relation file is simple enough to be
>
Thomas Munro wrote:
> On Thu, Nov 12, 2020 at 10:15 PM Antonin Houska wrote:
> I saw that -- great news! -- and have been meaning to write for a
> while. I think I am nearly ready to talk about it again.
I'm looking forward to it :-)
> 100% that it's worth trying to do something much
On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
>
>
> No background undo
> --
>
> Reduced complexity of the patch seems to be the priority at the moment. Amit
> suggested that cleanup of an orphaned relation file is simple enough to be
> done on foreground and I agree.
>
On Thu, Nov 12, 2020 at 10:15 PM Antonin Houska wrote:
> Thomas Munro wrote:
> > Thanks. We decided to redesign a couple of aspects of the undo
> > storage and record layers that this patch was intended to demonstrate,
> > and work on that is underway. More on that soon.
>
> As my boss
On Sat, Nov 30, 2019 at 9:25 PM Michael Paquier wrote:
> On Thu, Feb 07, 2019 at 07:35:31AM +0530, Andres Freund wrote:
> > It was JUST added ... :) thought I saw you reply on the other thread
> > about it, but I was wrong...
>
> Six months later without any activity, I am marking this entry as
>
On Thu, Feb 07, 2019 at 07:35:31AM +0530, Andres Freund wrote:
> It was JUST added ... :) thought I saw you reply on the other thread
> about it, but I was wrong...
Six months later without any activity, I am marking this entry as
returned with feedback. The latest patch set does not apply
On Thu, Nov 28, 2019 at 3:45 PM Michael Paquier wrote:
> On Tue, Sep 17, 2019 at 10:03:20AM +1200, Thomas Munro wrote:
> > Oops, right. So it should just be added to the if condition. Will do.
>
> It's been a couple of months and the discussion has stale. It seems
> also that the patch was
On Tue, Sep 17, 2019 at 10:03:20AM +1200, Thomas Munro wrote:
> Oops, right. So it should just be added to the if condition. Will do.
It's been a couple of months and the discussion has stale. It seems
also that the patch was waiting for an update. So I am marking it as
RwF for now. Please
On Mon, Sep 16, 2019 at 10:37 PM Robert Haas wrote:
>
> It seems to me that zheap went wrong in ending up with separate undo
> types for in-place and non-in-place updates. Why not just have ONE
> kind of undo record that describes an update, and allow that update to
> have either one TID or two
On Tue, Sep 17, 2019 at 3:09 AM Kuntal Ghosh wrote:
> On Mon, Sep 16, 2019 at 11:23 AM Thomas Munro wrote:
> > Agreed. I added a line to break out of that loop if !block->in_use.
> >
> I think we should skip the block if !block->in_use. Because, the undo
> buffer can be registered in a
On Mon, Sep 16, 2019 at 11:09 AM Kuntal Ghosh
wrote:
> Okay. In that case, we need to rethink the cases for multi-inserts and
> non-inlace updates both of which currently inserts multiple undo
> record corresponding to a single WAL record. For multi-inserts, it can
> be solved easily by moving
Hello Thomas,
On Mon, Sep 16, 2019 at 11:23 AM Thomas Munro wrote:
>
> > 1. In UndoLogAllocateInRecovery, when we find the current log number
> > from the list of registered blocks, we don't check whether the
> > block->in_use flag is true or not. In XLogResetInsertion, we just
> > reset in_use
On Mon, Sep 16, 2019 at 5:27 AM Kuntal Ghosh wrote:
> While testing zheap over undo apis, we've found the following
> issues/scenarios that might need some fixes/discussions:
Thanks!
> 1. In UndoLogAllocateInRecovery, when we find the current log number
> from the list of registered blocks, we
Hello Thomas,
While testing zheap over undo apis, we've found the following
issues/scenarios that might need some fixes/discussions:
1. In UndoLogAllocateInRecovery, when we find the current log number
from the list of registered blocks, we don't check whether the
block->in_use flag is true or
On 2019-Sep-06, vignesh C wrote:
> Hi Thomas,
>
> While testing one of the recovery scenarios I found one issue:
> FailedAssertion("!(logno == context->recovery_logno)
I marked this patch Waiting on Author.
--
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development,
Hi Thomas,
While testing one of the recovery scenarios I found one issue:
FailedAssertion("!(logno == context->recovery_logno)
The details of the same is mentioned below:
The context's try_location was not updated in
UndoLogAllocateInRecovery, in PrepareUndoInsert the try_location was
updated
On Fri, Aug 30, 2019 at 8:27 PM Kuntal Ghosh wrote:
> I'm getting the following assert failure while performing the recovery
> with the same.
> "TRAP: FailedAssertion("slot->meta.status == UNDO_LOG_STATUS_FULL",
> File: "undolog.c", Line: 997)"
>
> I found that we don't emit an WAL record when we
Hello Thomas,
I was doing some testing for the scenario where the undo written by a
transaction overflows to multiple undo logs. For that I've modified
the following macro:
#define UndoLogMaxSize (1024 * 1024) /* 1MB undo log size */
(I should have used the provided pg_force_switch_undo though
On Wed, Aug 21, 2019 at 4:34 PM Robert Haas wrote:
> ReleaseResourcesAndProcessUndo() is only supposed to be called after
> AbortTransaction(), and the additional steps it performs --
> AtCleanup_Portals() and AtEOXact_Snapshot() or alternatively
> AtSubCleanup_Portals -- are taken from
ody's attached, and
(3) nobody's attached but the last record might need a fixup. When we
start up, all undo logs are in state 3, and the discard worker runs
around and puts them into state 1. Subsequently, they alternate
between states 1 and 2 for as long as the system remains up. But if
as an
On Wed, Aug 21, 2019 at 4:44 AM Andres Freund wrote:
> On 2019-08-20 21:02:18 +1200, Thomas Munro wrote:
> > 1. Anyone is allowed to try to read or write data at any UndoRecPtr
> > that has been allocated, through the buffer pool (though you'd usually
> > want to check it with
On Thu, Aug 22, 2019 at 9:55 PM Andres Freund wrote:
>
> Hi
>
> On August 22, 2019 9:14:10 AM PDT, Dilip Kumar wrote:
> > But, those requests will
> >ultimately be used for collecting the record by the bulk fetch. So if
> >we are planning to change the bulk fetch to read forward then maybe we
>
On Thu, Aug 22, 2019 at 7:34 PM Robert Haas wrote:
>
> On Thu, Aug 22, 2019 at 1:34 AM Dilip Kumar wrote:
> > Yeah, we can handle the bulk fetch as you suggested and it will make
> > it a lot easier. But, currently while registering the undo request
> > (especially during the first pass) we
On Thu, Aug 22, 2019 at 12:54 AM Andres Freund wrote:
> But why? It makes a *lot* more sense to have it in the beginning. I
> don't think bulk-fetch really requires it to be in the end - we can
> still process records forward on a page-by-page basis.
There are two separate needs here: to be able
Hi
On August 22, 2019 9:14:10 AM PDT, Dilip Kumar wrote:
> But, those requests will
>ultimately be used for collecting the record by the bulk fetch. So if
>we are planning to change the bulk fetch to read forward then maybe we
>don't need the valid last undo record pointer because that we will
On Thu, Aug 22, 2019 at 9:21 PM Dilip Kumar wrote:
>
> On Thu, Aug 22, 2019 at 7:34 PM Robert Haas wrote:
> >
> > On Thu, Aug 22, 2019 at 1:34 AM Dilip Kumar wrote:
> > > Yeah, we can handle the bulk fetch as you suggested and it will make
> > > it a lot easier. But, currently while
On Thu, Aug 22, 2019 at 1:34 AM Dilip Kumar wrote:
> Yeah, we can handle the bulk fetch as you suggested and it will make
> it a lot easier. But, currently while registering the undo request
> (especially during the first pass) we need to compute the from_urecptr
> and the to_urecptr. And, for
On Thu, Aug 22, 2019 at 11:04 AM Dilip Kumar wrote:
>
> On Thu, Aug 22, 2019 at 10:24 AM Andres Freund wrote:
> >
> > Hi,
> >
> > On 2019-08-22 10:19:04 +0530, Dilip Kumar wrote:
> > > On Thu, Aug 22, 2019 at 9:58 AM Andres Freund wrote:
> > > >
> > > > Hi,
> > > >
> > > > On 2019-08-22
On Thu, Aug 22, 2019 at 10:24 AM Andres Freund wrote:
>
> Hi,
>
> On 2019-08-22 10:19:04 +0530, Dilip Kumar wrote:
> > On Thu, Aug 22, 2019 at 9:58 AM Andres Freund wrote:
> > >
> > > Hi,
> > >
> > > On 2019-08-22 09:51:22 +0530, Dilip Kumar wrote:
> > > > We can not know the complete size of
On Thu, Aug 22, 2019 at 9:58 AM Andres Freund wrote:
>
> Hi,
>
> On 2019-08-22 09:51:22 +0530, Dilip Kumar wrote:
> > We can not know the complete size of the record even by reading the
> > header because we have a payload that is variable part and payload
> > length are stored in the payload
Hi,
On 2019-08-22 10:19:04 +0530, Dilip Kumar wrote:
> On Thu, Aug 22, 2019 at 9:58 AM Andres Freund wrote:
> >
> > Hi,
> >
> > On 2019-08-22 09:51:22 +0530, Dilip Kumar wrote:
> > > We can not know the complete size of the record even by reading the
> > > header because we have a payload that
Hi,
On 2019-08-22 09:51:22 +0530, Dilip Kumar wrote:
> We can not know the complete size of the record even by reading the
> header because we have a payload that is variable part and payload
> length are stored in the payload header which again can be at random
> offset.
Wait, but that's just
On Wed, Aug 21, 2019 at 9:04 PM Robert Haas wrote:
>
> On Wed, Aug 21, 2019 at 3:55 AM Dilip Kumar wrote:
> > I have already attempted that part and I feel it is not making code
> > any simpler than what we have today. For packing, it's fine because I
> > can process all the member once and
On Wed, Aug 14, 2019 at 2:57 AM Andres Freund wrote:
> - My reading of the current xact.c integration is that it's not workable
> as is. Undo is executed outside of a valid transaction state,
> exceptions aren't properly undone, logic would need to be duplicated
> to a significant degree,
On August 21, 2019 8:36:34 AM PDT, Robert Haas wrote:
> We treat LWLockAcquire() as a no-fail operation in many
>places; in my opinion, that elog(ERROR) that we have for too many
>LWLocks should be changed to elog(PANIC) precisely because we do treat
>LWLockAcquire() as no-fail in lots of
On Wed, Aug 21, 2019 at 3:55 AM Dilip Kumar wrote:
> I have already attempted that part and I feel it is not making code
> any simpler than what we have today. For packing, it's fine because I
> can process all the member once and directly pack it into one memory
> chunk and I can insert that to
On Wed, Aug 21, 2019 at 6:38 AM Amit Kapila wrote:
> > FWIW, although I also thought of doing what you are describing here, I
> > think Andres's proposal is probably preferable, because it's simpler.
> > There's not really any reason why we can't take the buffer locks from
> > within the critical
On Tue, Aug 20, 2019 at 8:10 PM Robert Haas wrote:
>
> On Tue, Aug 20, 2019 at 2:42 AM Amit Kapila wrote:
> > > Well, my main point, which so far has largely been ignored, was that we
> > > may not acquire page locks when we still need to search for victim
> > > buffers later. If we don't need
On Tue, Aug 20, 2019 at 7:57 PM Robert Haas wrote:
>
> On Mon, Aug 19, 2019 at 2:04 AM Dilip Kumar wrote:
> > Currently, In UnpackedUndoRecord we store all members directly which
> > are set by the caller. We store pointers to some header which are
> > allocated internally by the undo layer and
On Tue, Aug 13, 2019 at 8:11 AM Robert Haas wrote:
> > We can probably check the fxid queue and error queue to get that
> > value. However, I am not sure if that is sufficient because incase we
> > perform the request in the foreground, it won't be present in queues.
>
> Oh, I forgot about that
On Mon, Aug 19, 2019 at 2:04 AM Dilip Kumar wrote:
> Currently, In UnpackedUndoRecord we store all members directly which
> are set by the caller. We store pointers to some header which are
> allocated internally by the undo layer and the caller need not worry
> about setting them. So now you
On Tue, Aug 20, 2019 at 5:02 AM Thomas Munro wrote:
> 3. UndoLogDiscard() uses DiscardBuffer() to invalidate any currently
> unpinned buffers, and marks as BM_DISCARDED any that happen to be
> pinned right now, so they can't be immediately invalidated. Such
> buffers are never written back and
On 2019-08-20 09:44:23 -0700, Andres Freund wrote:
> On 2019-08-20 21:02:18 +1200, Thomas Munro wrote:
> > Aside from code changes based on review (and I have more to come of
> > those), the attached experimental patchset (also at
> > https://github.com/EnterpriseDB/zheap/tree/undo) has a new
>hold the maximum amount of undo needed. These buffers would be marked
> >as !BM_TAG_VALID | BUF_REFCOUNT_ONE.
> >
> > I assume that we'd make a) cheap by keeping it pinned for undo logs that
> > a backend is actively attached to. b) should only be ne
Hi,
On 2019-08-20 21:02:18 +1200, Thomas Munro wrote:
> Aside from code changes based on review (and I have more to come of
> those), the attached experimental patchset (also at
> https://github.com/EnterpriseDB/zheap/tree/undo) has a new protocol
> that, I hope, allows for better concurrency,
Hi,
On 2019-08-20 09:08:29 -0400, Robert Haas wrote:
> On Sat, Aug 17, 2019 at 1:28 PM Andres Freund wrote:
> > The primary one in the context here is that if we do *not* have to lock
> > the buffers all ahead of time, we can simplify the interface. We
> > certainly can't lock the buffers over
On Mon, Aug 19, 2019 at 8:22 AM Amit Kapila wrote:
> One point to remember in this regard is that we do need to modify the
> LSN in undo pages after writing WAL, so all the undo pages need to be
> locked by that time or we again need to take the lock on them.
Uh, but a big part of the point of
On Tue, Aug 20, 2019 at 2:42 AM Amit Kapila wrote:
> > Well, my main point, which so far has largely been ignored, was that we
> > may not acquire page locks when we still need to search for victim
> > buffers later. If we don't need to lock the pages up-front, but only do
> > so once we're
On Mon, Aug 19, 2019 at 5:16 PM Andres Freund wrote:
> Well, my main point, which so far has largely been ignored, was that we
> may not acquire page locks when we still need to search for victim
> buffers later. If we don't need to lock the pages up-front, but only do
> so once we're actually
On Sat, Aug 17, 2019 at 1:28 PM Andres Freund wrote:
> The primary one in the context here is that if we do *not* have to lock
> the buffers all ahead of time, we can simplify the interface. We
> certainly can't lock the buffers over IO (due to buffer reclaim) as
> we're doing right now, so we'd
o that we need to update (e.g. to
>update the next pointer)
> c) perform clock sweep etc to acquire (find or create) enough clean to
>hold the maximum amount of undo needed. These buffers would be marked
>as !BM_TAG_VALID | BUF_REFCOUNT_ONE.
>
> I assume that we'd make
On Tue, Aug 20, 2019 at 2:46 AM Andres Freund wrote:
>
> On 2019-08-19 17:52:24 +0530, Amit Kapila wrote:
> > On Sat, Aug 17, 2019 at 10:58 PM Andres Freund wrote:
> > >
> > > > Well, I don't understand why you're on about this. We've discussed it
> > > > a number of times but I'm still
On 2019-08-19 17:52:24 +0530, Amit Kapila wrote:
> On Sat, Aug 17, 2019 at 10:58 PM Andres Freund wrote:
> >
> > Hi,
> >
> > On 2019-08-17 12:05:21 -0400, Robert Haas wrote:
> > > On Wed, Aug 14, 2019 at 12:39 PM Andres Freund wrote:
> > > > > > Again, I think it's not ok to just assume you can
On Sat, Aug 17, 2019 at 10:58 PM Andres Freund wrote:
>
> Hi,
>
> On 2019-08-17 12:05:21 -0400, Robert Haas wrote:
> > On Wed, Aug 14, 2019 at 12:39 PM Andres Freund wrote:
> > > > > Again, I think it's not ok to just assume you can lock an essentially
> > > > > unbounded number of buffers. This
On Wed, Aug 14, 2019 at 10:35 PM Andres Freund wrote:
>
>
> > > - When reading an undo record, the whole stage of UnpackUndoData()
> > > reading data into a the UndoPackContext is omitted, reading directly
> > > into the UnpackedUndoRecord. That removes one further copy of the
> > >
On Sat, Aug 17, 2019 at 9:35 PM Robert Haas wrote:
>
> On Wed, Aug 14, 2019 at 12:39 PM Andres Freund wrote:
> > > > Again, I think it's not ok to just assume you can lock an essentially
> > > > unbounded number of buffers. This seems almost guaranteed to result in
> > > > deadlocks. And there's
solutely *NOT* obvious that that's
the explanation. And I don't think any of the locking sites actually has
comments explaining why the locks are acquired at that time (in fact,
IIRC until the review some even only mentioned pinning, not locking).
> > > Suppose you insert one rec
1 - 100 of 346 matches
Mail list logo
