* Andres Freund (and...@2ndquadrant.com) wrote:
> > It helps in that once we have the lock, things aren't changing under us.
> > The closer we can keep that to when the transaction starts, the better..
>
> If you look at my example the timing where we take the snapshot isn't
> the problem. While w
On 2013-05-06 13:07:17 -0400, Tom Lane wrote:
> I'm afraid that this is institutionalizing a design deficiency in
> pg_dump; namely that it takes its snapshot before acquiring locks.
I have suggested this before, but if pg_dump would use SELECT FOR SHARE
in the queries it uses to build DDL it woul
On Tue, May 7, 2013 at 10:02 AM, Dimitri Fontaine wrote:
> Rather than take some locks, you can now prevent the database objects
> from changing with an event trigger. pg_dump could install that event
> trigger in a preparing transaction, then do its work as currently, then
> when done either remo
On 2013-05-07 16:50:52 +0100, Greg Stark wrote:
> What's the worst case for using an old snapshot? If I try to access a
> table that doesn't exist any longer I'll get an error. That doesn't
> really seem that bad for the use case I described. It's worse for the
> full table dump but for an explicit
On 2013-05-07 11:01:48 -0400, Stephen Frost wrote:
> * Andres Freund (and...@2ndquadrant.com) wrote:
> > On 2013-05-07 08:54:54 -0400, Stephen Frost wrote:
> > > Agreed- but it also isn't currently possible to make it any smaller.
> >
> > Uh. Why not? I think this is what needs to be fixed instea
Robert Haas writes:
> On Mon, May 6, 2013 at 1:07 PM, Tom Lane wrote:
>> I'm afraid that this is institutionalizing a design deficiency in
>> pg_dump; namely that it takes its snapshot before acquiring locks.
>> Ideally that would happen the other way around. I don't have a good
>> idea how we c
On Mon, May 6, 2013 at 1:07 PM, Tom Lane wrote:
>> Exported snapshots allow you to coordinate a number of actions
>> together, so they all see a common view of the database. So this patch
>> allows a very general approach to this, much more so than pg_dump
>> allows currently since the exact timin
Greg Stark writes:
> [ ideas about dumping some past state of a table ]
> If I try to access a table whose schema has changed then I might use
> the wrong tupledesc and see rows that don't decode properly. That
> would be a disaster. Can we protect against that by noticing that the
> pg_class ro
Greg,
* Greg Stark (st...@mit.edu) wrote:
> One natural way to do it would be to make an option to pg_dump which
> caused it to do all the normal pre-dump things it would normally do,
> then export a snapshot and wait for the user. (Alternately it could
> even create a prepared transaction which i
On Tue, May 7, 2013 at 1:54 PM, Stephen Frost wrote:
> I believe the point that Tom is making is that we shouldn't paint
> ourselves into a corner by letting users provide old snapshots to
> pg_dump which haven't acquired any of the necessary locks. The goal, at
> least as I read it, is to come u
* Andres Freund (and...@2ndquadrant.com) wrote:
> On 2013-05-07 08:54:54 -0400, Stephen Frost wrote:
> > Agreed- but it also isn't currently possible to make it any smaller.
>
> Uh. Why not? I think this is what needs to be fixed instead of making
> the hole marginally smaller elsewhere.
If we'r
Hi,
On 2013-05-07 08:54:54 -0400, Stephen Frost wrote:
> * Andres Freund (and...@2ndquadrant.com) wrote:
> > > All of which I
> > > think I agree with, but I don't agree with the conclusion that this
> > > larger window is somehow acceptable because there's a very small window
> > > (one which can
* Simon Riggs (si...@2ndquadrant.com) wrote:
> On 7 May 2013 01:18, Stephen Frost wrote:
> > * Simon Riggs (si...@2ndquadrant.com) wrote:
> >> If anybody really wanted to fix pg_dump, they could do. If that was so
> >> important, why block this patch, but allow parallel pg_dump to be
> >> committe
* Andres Freund (and...@2ndquadrant.com) wrote:
> > All of which I
> > think I agree with, but I don't agree with the conclusion that this
> > larger window is somehow acceptable because there's a very small window
> > (one which can't be made any smaller, today..) which exists today.
>
> The wind
On 7 May 2013 01:18, Stephen Frost wrote:
> * Simon Riggs (si...@2ndquadrant.com) wrote:
>> If anybody really wanted to fix pg_dump, they could do. If that was so
>> important, why block this patch, but allow parallel pg_dump to be
>> committed without it?
>
> Because parallel pg_dump didn't make
On 2013-05-06 21:07:36 -0400, Stephen Frost wrote:
> * Andres Freund (and...@2ndquadrant.com) wrote:
> > On 2013-05-06 20:18:26 -0400, Stephen Frost wrote:
> > > Because parallel pg_dump didn't make the problem any *worse*..? This
> > > does. The problem existed before parallel pg_dump.
> >
> >
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 05/07/2013 06:37 AM, Joe Conway wrote:
> On 05/06/2013 03:00 PM, Stephen Frost wrote:
> > For example, I'm not sure that we need more information in the
> > WAL.. What we need is a way to tell VACUUM to skip over 'recently
> > modified' records and
* Andres Freund (and...@2ndquadrant.com) wrote:
> On 2013-05-06 20:18:26 -0400, Stephen Frost wrote:
> > Because parallel pg_dump didn't make the problem any *worse*..? This
> > does. The problem existed before parallel pg_dump.
>
> Yes, it did.
That's not entirely clear- are you agreeing with
On 2013-05-07 02:53:16 +0200, Andres Freund wrote:
> A rather useful feature has to fix a bug in pg_dump which a) exists for
> ages b) has yet to be reported to the lists c) is rather complicated to
> fix and quite possibly requires proper snapshots for internals?
Just to clarify: I think this wor
On 2013-05-06 20:18:26 -0400, Stephen Frost wrote:
> Simon,
>
> * Simon Riggs (si...@2ndquadrant.com) wrote:
> > If anybody really wanted to fix pg_dump, they could do. If that was so
> > important, why block this patch, but allow parallel pg_dump to be
> > committed without it?
> Because paralle
Simon,
* Simon Riggs (si...@2ndquadrant.com) wrote:
> If anybody really wanted to fix pg_dump, they could do. If that was so
> important, why block this patch, but allow parallel pg_dump to be
> committed without it?
Because parallel pg_dump didn't make the problem any *worse*..? This
does. The
On 6 May 2013 22:13, Tom Lane wrote:
> Simon Riggs writes:
>> It does *not* pass in a raw snapshot. All it does is to allow pg_dump
>> to use an API that is already exposed by the backend for this very
>> purpose, one that has been in Postgres since 9.2.
>> http://www.postgresql.org/docs/devel/st
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 05/06/2013 03:00 PM, Stephen Frost wrote:
> For example, I'm not sure that we need more information in the
> WAL.. What we need is a way to tell VACUUM to skip over 'recently
> modified' records and not mark them as dead until some time has
> passed
* Greg Stark (st...@mit.edu) wrote:
> On Mon, May 6, 2013 at 10:02 PM, Simon Riggs wrote:
> > At the database level, it rolls back the whole kaboodle. Not what I
> > meant at all and I would expect people to start twitching at the
> > prospect.
>
> I think it would be pretty sweet but we don't ha
On Mon, May 6, 2013 at 10:02 PM, Simon Riggs wrote:
> At the database level, it rolls back the whole kaboodle. Not what I
> meant at all and I would expect people to start twitching at the
> prospect.
I think it would be pretty sweet but we don't have the infrastructure
for it. We would need to r
Simon,
* Simon Riggs (si...@2ndquadrant.com) wrote:
> On 6 May 2013 19:35, Stephen Frost wrote:
> > It certainly sounds interesting and I like the idea of it, but perhaps
> > we need a different mechanism than just passing in a raw snapshot, to
> > address the concerns that Tom raised.
>
> It do
Simon Riggs writes:
> It does *not* pass in a raw snapshot. All it does is to allow pg_dump
> to use an API that is already exposed by the backend for this very
> purpose, one that has been in Postgres since 9.2.
> http://www.postgresql.org/docs/devel/static/functions-admin.html#FUNCTIONS-SNAPSHOT
On 6 May 2013 19:48, Greg Stark wrote:
> On Mon, May 6, 2013 at 6:58 PM, Simon Riggs wrote:
>> In any case, "flashback database" is one of the most requested
>> features I know of... the ability to dump the database as it appeared
>> in the past *after* that point has passed.
>
> Fwiw that's not
On 6 May 2013 19:35, Stephen Frost wrote:
> It certainly sounds interesting and I like the idea of it, but perhaps
> we need a different mechanism than just passing in a raw snapshot, to
> address the concerns that Tom raised.
It does *not* pass in a raw snapshot. All it does is to allow pg_dump
On Mon, May 6, 2013 at 6:58 PM, Simon Riggs wrote:
> In any case, "flashback database" is one of the most requested
> features I know of... the ability to dump the database as it appeared
> in the past *after* that point has passed.
Fwiw that's not what flashback database does. It rolls back the
On 2013-05-06 14:35:14 -0400, Stephen Frost wrote:
> * Andres Freund (and...@2ndquadrant.com) wrote:
> > Its rather useful if you e.g. want to instantiate a new replica without
> > rebuilding pg_dump/pg_restore's capabilities wrt. ordering, parallelism,
> > separating initial data load from index c
* Andres Freund (and...@2ndquadrant.com) wrote:
> Its rather useful if you e.g. want to instantiate a new replica without
> rebuilding pg_dump/pg_restore's capabilities wrt. ordering, parallelism,
> separating initial data load from index creation and all that. Which
> already has been incompletely
On 6 May 2013 18:07, Tom Lane wrote:
> Or in short: -1 for the very concept of letting the user control
> pg_dump's snapshot.
That API is already exposed, so not sure why you say this now? This
has been in PG since early in 9.2, about 2 years ago.
In any case, "flashback database" is one of the
On 2013-05-06 13:07:17 -0400, Tom Lane wrote:
> Simon Riggs writes:
> > On 6 May 2013 16:02, Andrew Dunstan wrote:
> >> On 05/06/2013 10:56 AM, Simon Riggs wrote:
> >>> This overrides the internally generated snapshot in parallel pg_dump.
>
> >> Could you be a bit more expansive about the use cas
Simon Riggs writes:
> On 6 May 2013 16:02, Andrew Dunstan wrote:
>> On 05/06/2013 10:56 AM, Simon Riggs wrote:
>>> This overrides the internally generated snapshot in parallel pg_dump.
>> Could you be a bit more expansive about the use case, please?
> Exported snapshots allow you to coordinate
On 6 May 2013 16:02, Andrew Dunstan wrote:
>
> On 05/06/2013 10:56 AM, Simon Riggs wrote:
>>
>> Patch to allow pg_dump to use a snapshot exported with an explicit
>> pg_export_snapshot() for when precise timing of the snapshot is
>> important.
>>
>> This overrides the internally generated snapshot
On 05/06/2013 10:56 AM, Simon Riggs wrote:
Patch to allow pg_dump to use a snapshot exported with an explicit
pg_export_snapshot() for when precise timing of the snapshot is
important.
This overrides the internally generated snapshot in parallel pg_dump.
Could you be a bit more expansive
Patch to allow pg_dump to use a snapshot exported with an explicit
pg_export_snapshot() for when precise timing of the snapshot is
important.
This overrides the internally generated snapshot in parallel pg_dump.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development
38 matches
Mail list logo
