Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-22 Thread Jim Nasby
On 2/26/14, 9:15 AM, Simon Riggs wrote: On 26 February 2014 13:38, Andres Freund wrote: >Hi, > >On 2014-02-26 07:32:45 +, Simon Riggs wrote: >> >* This definitely should include isolationtester tests actually >> > performing concurrent ALTER TABLEs. All that's currently there is >> > t

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-22 Thread Simon Riggs
On 21 March 2014 16:11, Simon Riggs wrote: >>> + * Be careful to ensure this function is called for Tables and Indexes >>> only. >>> + * It is not currently safe to be called for Views because security_barrier >>> + * is listed as an option and so would be allowed to be set at a level >>> lower

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-22 Thread Simon Riggs
On 21 March 2014 23:36, Tom Lane wrote: > Simon Riggs writes: >> On 21 March 2014 20:58, Noah Misch wrote: >>> It's not the behavior I would choose for a new product, but I can't see >>> benefits sufficient to overturn previous decisions to keep it. > >> Speechless > > The key argument for not "

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Tom Lane
Simon Riggs writes: > On 21 March 2014 20:58, Noah Misch wrote: >> It's not the behavior I would choose for a new product, but I can't see >> benefits sufficient to overturn previous decisions to keep it. > Speechless The key argument for not "fixing" this is that it would break existing pg_dum

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Simon Riggs
On 21 March 2014 20:58, Noah Misch wrote: > On Fri, Mar 21, 2014 at 06:53:27PM +, Simon Riggs wrote: >> On 21 March 2014 17:49, Noah Misch wrote: >> >> >> > alter table information_schema.triggers set (security_barrier = true); >> >> >> >> I find it hard to justify why we accept such a stat

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Noah Misch
On Fri, Mar 21, 2014 at 06:53:27PM +, Simon Riggs wrote: > On 21 March 2014 17:49, Noah Misch wrote: > > >> > alter table information_schema.triggers set (security_barrier = true); > >> > >> I find it hard to justify why we accept such a statement. Surely its a > >> bug when the named table

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Simon Riggs
On 21 March 2014 03:45, Noah Misch wrote: >> + * Note that Hot Standby only knows about AccessExclusiveLocks on the master >> + * so any changes that might affect SELECTs running on standbys need to use >> + * AccessExclusiveLocks even if you think a lesser lock would do, unless you >> + * have a

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Simon Riggs
On 21 March 2014 17:49, Noah Misch wrote: >> >> + * Be careful to ensure this function is called for Tables and Indexes >> >> only. >> >> + * It is not currently safe to be called for Views because >> >> security_barrier >> >> + * is listed as an option and so would be allowed to be set at a le

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Noah Misch
On Fri, Mar 21, 2014 at 04:11:12PM +, Simon Riggs wrote: > On 21 March 2014 03:45, Noah Misch wrote: > > On Sat, Mar 08, 2014 at 11:14:30AM +, Simon Riggs wrote: > Thanks for the review. I'll respond to each point on a later email but > looks nothing much major, apart from the point raise

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Simon Riggs
On 21 March 2014 03:45, Noah Misch wrote: > On Sat, Mar 08, 2014 at 11:14:30AM +, Simon Riggs wrote: >> On 7 March 2014 09:04, Simon Riggs wrote: >> > The right thing to do here is to not push to the extremes. If we mess >> > too much with the ruleutil stuff it will just be buggy. A more >> >

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-20 Thread Noah Misch
On Sat, Mar 08, 2014 at 11:14:30AM +, Simon Riggs wrote: > On 7 March 2014 09:04, Simon Riggs wrote: > > The right thing to do here is to not push to the extremes. If we mess > > too much with the ruleutil stuff it will just be buggy. A more > > considered analysis in a later release is requir

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-20 Thread Vik Fearing
On 03/18/2014 11:39 AM, Simon Riggs wrote: > On 8 March 2014 11:14, Simon Riggs wrote: >> On 7 March 2014 09:04, Simon Riggs wrote: >> >>> The right thing to do here is to not push to the extremes. If we mess >>> too much with the ruleutil stuff it will just be buggy. A more >>> considered analys

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-19 Thread Simon Riggs
On 18 March 2014 21:21, Noah Misch wrote: > On Tue, Mar 18, 2014 at 10:39:03AM +, Simon Riggs wrote: >> On 8 March 2014 11:14, Simon Riggs wrote: >> > Implemented in attached patch, v22 > >> > I commend this patch to you for final review; I would like to commit >> > this in a few days. >> >>

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-18 Thread Noah Misch
On Tue, Mar 18, 2014 at 10:39:03AM +, Simon Riggs wrote: > On 8 March 2014 11:14, Simon Riggs wrote: > > Implemented in attached patch, v22 > > I commend this patch to you for final review; I would like to commit > > this in a few days. > > I'm planning to commit this today at 1500UTC barrin

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-18 Thread Simon Riggs
On 8 March 2014 11:14, Simon Riggs wrote: > On 7 March 2014 09:04, Simon Riggs wrote: > >> The right thing to do here is to not push to the extremes. If we mess >> too much with the ruleutil stuff it will just be buggy. A more >> considered analysis in a later release is required for a full and >

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-09 Thread Vik Fearing
On 03/06/2014 10:47 AM, Simon Riggs wrote: > On 5 March 2014 09:28, Simon Riggs wrote: > >> So that returns us to solving the catalog consistency problem in >> pg_dump and similar applications. > No answer, guys, and time is ticking away here. Sorry, I've accumulated several days of backlog (and

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-07 Thread Simon Riggs
On 6 March 2014 22:43, Noah Misch wrote: > On Tue, Mar 04, 2014 at 06:50:17PM +0100, Andres Freund wrote: >> On 2014-03-04 11:40:10 -0500, Tom Lane wrote: >> > Robert Haas writes: > I think this is all too >> > late for 9.4, though. >> > >> > I agree with the feeling that a meaningful fix for pg_

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-06 Thread Simon Riggs
On 6 March 2014 22:43, Noah Misch wrote: > Good analysis. The hazards arise when pg_dump uses one of the ruleutils.c > deparse worker functions. Ah, good. We're thinking along the same lines. I was already working on this analysis also. I'll complete mine and then compare notes. > One thing no

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-06 Thread Noah Misch
On Tue, Mar 04, 2014 at 06:50:17PM +0100, Andres Freund wrote: > On 2014-03-04 11:40:10 -0500, Tom Lane wrote: > > Robert Haas writes: > I think this is all too > > late for 9.4, though. > > > > I agree with the feeling that a meaningful fix for pg_dump isn't going > > to get done for 9.4. So th

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-06 Thread Simon Riggs
On 5 March 2014 09:28, Simon Riggs wrote: > So that returns us to solving the catalog consistency problem in > pg_dump and similar applications. No answer, guys, and time is ticking away here. I'd like to get a communal solution to this rather than just punting the whole patch. If we have to s

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-05 Thread Simon Riggs
On 4 March 2014 21:37, Tom Lane wrote: > Robert Haas writes: >> On Tue, Mar 4, 2014 at 2:39 PM, Simon Riggs wrote: >>> Your earlier claim that the dump is inconsistent just isn't accurate. >>> We now have MVCC catalogs, so any dump is going to see a perfectly >>> consistent set of data plus DDL.

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Tom Lane
Andres Freund writes: > On 2014-03-04 16:37:48 -0500, Tom Lane wrote: >> However, it seems possible that we could have a mode in which a read-only >> session did all its catalog fetches according to the transaction snapshot. >> That would get us to a situation where the backend-internal lookups th

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-04 Thread Tom Lane
Greg Stark writes: > On Tue, Mar 4, 2014 at 8:08 PM, Tom Lane wrote: >> CREATE INDEX CONCURRENTLY, otoh, already did break pg_dump, >> and we had to hack things to fix it; see commit >> 683abc73dff549e94555d4020dae8d02f32ed78b. > Well pg_dump was only broken in that there was a new catalog state

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Andres Freund
On 2014-03-04 16:37:48 -0500, Tom Lane wrote: > However, it seems possible that we could have a mode in which a read-only > session did all its catalog fetches according to the transaction snapshot. > That would get us to a situation where the backend-internal lookups that > ruleutils relies on wou

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-04 Thread Greg Stark
On Tue, Mar 4, 2014 at 8:08 PM, Tom Lane wrote: > CREATE INDEX CONCURRENTLY, otoh, already did break pg_dump, > and we had to hack things to fix it; see commit > 683abc73dff549e94555d4020dae8d02f32ed78b. Well pg_dump was only broken in that there was a new catalog state to deal with. But the comm

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Andres Freund
On 2014-03-04 14:29:31 -0800, Josh Berkus wrote: > On 03/04/2014 11:43 AM, Andres Freund wrote: > > On March 4, 2014 8:39:55 PM CET, Simon Riggs wrote: > >> I was going to add an option to increase lock level, but I can't see > >> why you'd want it even. The dumps are consistent... > > > > Mvcc s

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Josh Berkus
On 03/04/2014 11:43 AM, Andres Freund wrote: > On March 4, 2014 8:39:55 PM CET, Simon Riggs wrote: >> I was going to add an option to increase lock level, but I can't see >> why you'd want it even. The dumps are consistent... > > Mvcc scans only guarantee that individual scans are consistent, not

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Tom Lane
Robert Haas writes: > On Tue, Mar 4, 2014 at 2:39 PM, Simon Riggs wrote: >> Your earlier claim that the dump is inconsistent just isn't accurate. >> We now have MVCC catalogs, so any dump is going to see a perfectly >> consistent set of data plus DDL. OK the catalogs may change AFTER the >> snaps

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Robert Haas
On Tue, Mar 4, 2014 at 2:39 PM, Simon Riggs wrote: > On 4 March 2014 16:27, Tom Lane wrote: >> Alvaro Herrera writes: >>> One concern is schema changes that make a dump unrestorable, for >>> instance if there's a foreign key relationship between tables A and B, >> >> Yeah. Ideally, what pg_dump

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-04 Thread Tom Lane
Andres Freund writes: > On 2014-03-04 11:40:10 -0500, Tom Lane wrote: >> I don't care for (2). I'd like to have lock strength reduction as >> much as anybody, but it can't come at the price of reduction of >> reliability. > I am sorry, but I think this is vastly overstating the scope of the > pg

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Andres Freund
On March 4, 2014 8:39:55 PM CET, Simon Riggs wrote: >On 4 March 2014 16:27, Tom Lane wrote: >> Alvaro Herrera writes: >>> One concern is schema changes that make a dump unrestorable, for >>> instance if there's a foreign key relationship between tables A and >B, >> >> Yeah. Ideally, what pg_dum

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Simon Riggs
On 4 March 2014 16:27, Tom Lane wrote: > Alvaro Herrera writes: >> One concern is schema changes that make a dump unrestorable, for >> instance if there's a foreign key relationship between tables A and B, > > Yeah. Ideally, what pg_dump would produce would be a consistent snapshot > of the data

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-04 Thread Andres Freund
On 2014-03-04 11:40:10 -0500, Tom Lane wrote: > Robert Haas writes: > I think this is all too > late for 9.4, though. > > I agree with the feeling that a meaningful fix for pg_dump isn't going > to get done for 9.4. So that leaves us with the alternatives of (1) > put off the lock-strength-reduc

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Atri Sharma
On Tue, Mar 4, 2014 at 10:20 PM, Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Stephen Frost writes: > > > I don't have too much of an issue with the above, but I would like to > > > have us figure out a solution to the deadlock problem with parallel > > > pg_dump. The issue

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Alvaro Herrera
Tom Lane escribió: > I'd like to have lock strength reduction as much as anybody, but it > can't come at the price of reduction of reliability. Can we have at least a cut-down version of it? If we can just reduce the lock level required for ALTER TABLE / VALIDATE, that would be an enormous impro

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > I don't have too much of an issue with the above, but I would like to > > have us figure out a solution to the deadlock problem with parallel > > pg_dump. The issue arises when pg_dump gets an AccessShareLock and then > > another

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Tom Lane
Stephen Frost writes: > I don't have too much of an issue with the above, but I would like to > have us figure out a solution to the deadlock problem with parallel > pg_dump. The issue arises when pg_dump gets an AccessShareLock and then > another process attempts to acquire an AccessExclusiveLoc

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Tom Lane
Robert Haas writes: > I think this is all too late for 9.4, though. I agree with the feeling that a meaningful fix for pg_dump isn't going to get done for 9.4. So that leaves us with the alternatives of (1) put off the lock-strength-reduction patch for another year; (2) push it anyway and accept

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Yeah. Ideally, what pg_dump would produce would be a consistent snapshot > of the database state as of its transaction snapshot time. We have always > had that guarantee so far as user data was concerned, but it's been shaky > (and getting worse) so far as

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Tom Lane
Alvaro Herrera writes: > One concern is schema changes that make a dump unrestorable, for > instance if there's a foreign key relationship between tables A and B, Yeah. Ideally, what pg_dump would produce would be a consistent snapshot of the database state as of its transaction snapshot time.

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Robert Haas
On Tue, Mar 4, 2014 at 10:17 AM, Alvaro Herrera wrote: > One possible idea would be to create a new lock level which conflicts > with DDL changes but not with regular operation including dumps; so it > wouldn't self-conflict but it would conflict with ShareUpdateExclusive. > pg_dump would acquire

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Atri Sharma
On Tue, Mar 4, 2014 at 8:19 PM, Stephen Frost wrote: > * Atri Sharma (atri.j...@gmail.com) wrote: > > If its not the case, the user should be more careful about when he is > > scheduling backups to so that they dont conflict with DDL changes. > > I'm not following this as closely as I'd like to,

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Alvaro Herrera
Stephen Frost escribió: > * Atri Sharma (atri.j...@gmail.com) wrote: > > If its not the case, the user should be more careful about when he is > > scheduling backups to so that they dont conflict with DDL changes. > > I'm not following this as closely as I'd like to, but I wanted to voice > my opi

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Stephen Frost
* Atri Sharma (atri.j...@gmail.com) wrote: > If its not the case, the user should be more careful about when he is > scheduling backups to so that they dont conflict with DDL changes. I'm not following this as closely as I'd like to, but I wanted to voice my opinion that this is just not acceptabl

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Simon Riggs
On 4 March 2014 12:18, Robert Haas wrote: > On Tue, Mar 4, 2014 at 6:57 AM, Simon Riggs wrote: >> The main impact I see is that this would block VACUUM while pg_dump runs. >> >> But then, while pg_dump runs VACUUM is ineffective anyway so perhaps >> that is no bad thing. > > Well, a vacuum that's

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Atri Sharma
I'd really like to see us find a way to apply some version of this > patch. I was in favor of the concept 3 years ago when we did this the > first time, and I've subsequently done quite a bit of work (viz., MVCC > catalog snapshots) to eliminate the main objection that was raised at > that time.

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Robert Haas
On Tue, Mar 4, 2014 at 6:57 AM, Simon Riggs wrote: > The main impact I see is that this would block VACUUM while pg_dump runs. > > But then, while pg_dump runs VACUUM is ineffective anyway so perhaps > that is no bad thing. Well, a vacuum that's already running when pg_dump starts up may be doing

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Simon Riggs
On 4 March 2014 09:31, Simon Riggs wrote: > On 4 March 2014 08:39, Atri Sharma wrote: >> >>> >>> Good points. >>> >>> In most cases, DDL is applied manually after careful thought, so >>> people seldom dump at the same time they upgrade the database. This is >>> especially true for pg_dump since i

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Atri Sharma
> > If its not the case, the user should be more careful about when he is > > scheduling backups to so that they dont conflict with DDL changes. > > That is most certainly the wise choice. > > > I am not too comfortable with exposing the locking type to the user. That > > may be just me though. > >

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Simon Riggs
On 4 March 2014 08:39, Atri Sharma wrote: > >> >> Good points. >> >> In most cases, DDL is applied manually after careful thought, so >> people seldom dump at the same time they upgrade the database. This is >> especially true for pg_dump since it captures the logical definition >> of tables. So m

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Atri Sharma
> > Good points. > > In most cases, DDL is applied manually after careful thought, so > people seldom dump at the same time they upgrade the database. This is > especially true for pg_dump since it captures the logical definition > of tables. So most people will be happy with the default locking, b

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Simon Riggs
On 4 March 2014 01:07, Andres Freund wrote: > On 2014-03-03 19:15:27 -0500, Tom Lane wrote: >> Noah Misch writes: >> > Just to be clear, that list is not a commentary on the particular patch at >> > hand. Those are merely the kinds of regressions to look for in a patch >> > affecting this area o

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Tom Lane
Andres Freund writes: > On 2014-03-03 20:32:13 -0500, Tom Lane wrote: >> You're missing the point entirely if you think pg_dump recreates >> everything client-side. > No, I am not obviously not thinking that. What I mean is that the things > that actually change their locking requirement in the

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Andres Freund
On 2014-03-03 20:32:13 -0500, Tom Lane wrote: > > Afair (I really haven't rechecked) all the actions that have a changed > > locklevels affect things that pg_dump recreates clientside, using a > > repeatable read snapshot, so there shouldn't be much change there? > > You're missing the point entir

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Tom Lane
Andres Freund writes: > On 2014-03-03 19:15:27 -0500, Tom Lane wrote: >> This greatly >> ameliorates the snapshot-skew problems that arise from its habit of doing >> some things for itself and other things via backend-internal functions >> (which historically used SnapshotNow and now use a fresh M

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Andres Freund
On 2014-03-03 19:15:27 -0500, Tom Lane wrote: > Noah Misch writes: > > Just to be clear, that list is not a commentary on the particular patch at > > hand. Those are merely the kinds of regressions to look for in a patch > > affecting this area of the code. > > A complaint on pgsql-bugs just now

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Tom Lane
Noah Misch writes: > Just to be clear, that list is not a commentary on the particular patch at > hand. Those are merely the kinds of regressions to look for in a patch > affecting this area of the code. A complaint on pgsql-bugs just now reminded me of a specific area that needs to be looked at

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Noah Misch
On Mon, Mar 03, 2014 at 07:19:45PM +, Simon Riggs wrote: > On 3 March 2014 18:57, Noah Misch wrote: > > On Mon, Mar 03, 2014 at 10:19:55AM -0500, Robert Haas wrote: > >> On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs wrote: > >> > Removing SELECT privilege while running a SELECT would be a diff

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Simon Riggs
On 3 March 2014 18:57, Noah Misch wrote: > On Mon, Mar 03, 2014 at 10:19:55AM -0500, Robert Haas wrote: >> On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs wrote: >> > Removing SELECT privilege while running a SELECT would be a different >> > matter. This is all a matter of definition; we can make u

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Noah Misch
On Mon, Mar 03, 2014 at 03:43:46PM +, Simon Riggs wrote: > The question is are there any specific areas of concern here? If not, > then we commit because we've done a lot of work on it and at the > moment the balance is high benefit to users against a non-specific > feeling of risk. > > @Noah

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Noah Misch
On Mon, Mar 03, 2014 at 10:19:55AM -0500, Robert Haas wrote: > On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs wrote: > > Removing SELECT privilege while running a SELECT would be a different > > matter. This is all a matter of definition; we can make up any rules > > we like. Doing so is IMHO a sep

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Robert Haas
On Mon, Mar 3, 2014 at 11:29 AM, Simon Riggs wrote: > On 3 March 2014 16:06, Robert Haas wrote: >> On Sun, Mar 2, 2014 at 4:50 AM, Simon Riggs wrote: >>> v20 includes slightly re-ordered checks in GetLockLevel, plus more >>> detailed comments on each group of subcommands. >>> >>> Also corrects g

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Simon Riggs
On 3 March 2014 16:06, Robert Haas wrote: > On Sun, Mar 2, 2014 at 4:50 AM, Simon Riggs wrote: >> v20 includes slightly re-ordered checks in GetLockLevel, plus more >> detailed comments on each group of subcommands. >> >> Also corrects grammar as noted by Vik. >> >> Plus adds an example of usage

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Simon Riggs
On 3 March 2014 15:53, Tom Lane wrote: > Simon Riggs writes: >> On 3 March 2014 15:19, Robert Haas wrote: >>> What I'm >>> really concerned about is whether there are other things like the >>> SnapshotNow issues that can cause stuff to halt and catch fire. I >>> don't know whether there are or

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Robert Haas
On Sun, Mar 2, 2014 at 4:50 AM, Simon Riggs wrote: > v20 includes slightly re-ordered checks in GetLockLevel, plus more > detailed comments on each group of subcommands. > > Also corrects grammar as noted by Vik. > > Plus adds an example of usage to the docs. This patch contains a one line change

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Tom Lane
Simon Riggs writes: > On 3 March 2014 15:19, Robert Haas wrote: >> What I'm >> really concerned about is whether there are other things like the >> SnapshotNow issues that can cause stuff to halt and catch fire. I >> don't know whether there are or are not, but that's my concern. > Of course it

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Simon Riggs
On 3 March 2014 15:19, Robert Haas wrote: > On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs wrote: > What I'm > really concerned about is whether there are other things like the > SnapshotNow issues that can cause stuff to halt and catch fire. I > don't know whether there are or are not, but that'

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Robert Haas
On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs wrote: > Removing SELECT privilege while running a SELECT would be a different > matter. This is all a matter of definition; we can make up any rules > we like. Doing so is IMHO a separate patch and not something to hold > up the main patch. So I thin

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-01 Thread Vik Fearing
On 03/01/2014 12:06 PM, Simon Riggs wrote: > On 27 February 2014 08:48, Simon Riggs wrote: >> On 26 February 2014 15:25, Andres Freund wrote: >>> On 2014-02-26 15:15:00 +, Simon Riggs wrote: On 26 February 2014 13:38, Andres Freund wrote: > Hi, > > On 2014-02-26 07:32:45 +00

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-27 Thread Simon Riggs
On 26 February 2014 15:25, Andres Freund wrote: > On 2014-02-26 15:15:00 +, Simon Riggs wrote: >> On 26 February 2014 13:38, Andres Freund wrote: >> > Hi, >> > >> > On 2014-02-26 07:32:45 +, Simon Riggs wrote: >> >> > * This definitely should include isolationtester tests actually >> >> >

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-27 Thread Simon Riggs
On 26 February 2014 15:25, Andres Freund wrote: >> >> > * Why does ChangeOwner need AEL? >> >> >> >> Ownership affects privileges, which includes SELECTs, hence AEL. >> > >> > So? >> >> That reply could be added to any post. Please explain your concern. > > I don't understand why that means it ne

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-27 Thread Simon Riggs
On 26 February 2014 07:32, Simon Riggs wrote: >> * Are you sure AlterConstraint is generally safe without an AEL? It >> should be safe to change whether something is deferred, but not >> necessarily whether it's deferrable? > > We change the lock levels for individual commands. This patch pro

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-26 Thread Andres Freund
On 2014-02-26 15:15:00 +, Simon Riggs wrote: > On 26 February 2014 13:38, Andres Freund wrote: > > Hi, > > > > On 2014-02-26 07:32:45 +, Simon Riggs wrote: > >> > * This definitely should include isolationtester tests actually > >> > performing concurrent ALTER TABLEs. All that's current

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-26 Thread Simon Riggs
On 26 February 2014 13:38, Andres Freund wrote: > Hi, > > On 2014-02-26 07:32:45 +, Simon Riggs wrote: >> > * This definitely should include isolationtester tests actually >> > performing concurrent ALTER TABLEs. All that's currently there is >> > tests that the locklevel isn't too high, b

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-26 Thread Andres Freund
Hi, On 2014-02-26 07:32:45 +, Simon Riggs wrote: > > * This definitely should include isolationtester tests actually > > performing concurrent ALTER TABLEs. All that's currently there is > > tests that the locklevel isn't too high, but not that it actually works. > > There is no concurren

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-25 Thread Simon Riggs
On 24 February 2014 16:01, Andres Freund wrote: > Hi, > > I took a quick peek at this, and noticed the following things: > * I am pretty sure this patch doesn't compile anymore after the latest > set of releases. Refreshed to v18, will post shortly. > * This definitely should include isolation

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-24 Thread Andres Freund
Hi, I took a quick peek at this, and noticed the following things: * I am pretty sure this patch doesn't compile anymore after the latest set of releases. * This definitely should include isolationtester tests actually performing concurrent ALTER TABLEs. All that's currently there is tests t

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-29 Thread Simon Riggs
On 29 January 2014 05:43, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jan 28, 2014 at 12:36 PM, Alvaro Herrera >> wrote: >>> Honestly, I would prefer that we push a patch that has been thoroughly >>> reviewed and in which we have more confidence, so that we can push >>> without a GUC. This

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 28, 2014 at 12:36 PM, Alvaro Herrera > wrote: >> Honestly, I would prefer that we push a patch that has been thoroughly >> reviewed and in which we have more confidence, so that we can push >> without a GUC. This means mark in CF as needs-review, then some other

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Robert Haas
On Tue, Jan 28, 2014 at 12:36 PM, Alvaro Herrera wrote: > Bruce Momjian escribió: >> > I have no problem removing the parameter if required to. In that case, >> > I would like to leave the parameter in until mid beta, to allow >> > greater certainty. > > Uhm. If we remove a GUC during beta we don

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Simon Riggs
On 28 January 2014 17:21, Heikki Linnakangas wrote: > I don't understand why anyone would want to turn this feature off, ie. > require stronger locks than necessary for a DDL change. Nobody would *want* to turn it off. They might need to, as explained. > If we're not confident that the patch is

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Andres Freund
On 2014-01-27 15:25:22 -0500, Robert Haas wrote: > On Mon, Jan 27, 2014 at 12:58 PM, Simon Riggs wrote: > > This version adds a GUC called ddl_exclusive_locks which allows us to > > keep the 9.3 behaviour if we wish it. Some people may be surprised > > that their programs don't wait in the same pl

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Simon Riggs
On 28 January 2014 17:15, Bruce Momjian wrote: > On Tue, Jan 28, 2014 at 04:36:39PM +, Simon Riggs wrote: >> For me, reducing the strength of DDL locking is a major change in >> RDBMS behaviour that could both delight and surprise our users. Maybe >> a few actually depend upon the locking beha

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Tom Lane
Alvaro Herrera writes: > Honestly, I would prefer that we push a patch that has been thoroughly > reviewed and in which we have more confidence, so that we can push > without a GUC. This means mark in CF as needs-review, then some other > developer reviews it and marks it as ready-for-committer.

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Alvaro Herrera
Bruce Momjian escribió: > > I have no problem removing the parameter if required to. In that case, > > I would like to leave the parameter in until mid beta, to allow > > greater certainty. Uhm. If we remove a GUC during beta we don't need to force an initdb. At worst we will need to keep a no-o

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Bruce Momjian
On Tue, Jan 28, 2014 at 07:30:47PM +0200, Heikki Linnakangas wrote: > On 01/28/2014 07:26 PM, Bruce Momjian wrote: > >On Tue, Jan 28, 2014 at 07:21:50PM +0200, Heikki Linnakangas wrote: > I have no problem removing the parameter if required to. In that case, > I would like to leave the para

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Heikki Linnakangas
On 01/28/2014 07:26 PM, Bruce Momjian wrote: On Tue, Jan 28, 2014 at 07:21:50PM +0200, Heikki Linnakangas wrote: I have no problem removing the parameter if required to. In that case, I would like to leave the parameter in until mid beta, to allow greater certainty. In any case, I would wish to

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Bruce Momjian
On Tue, Jan 28, 2014 at 07:21:50PM +0200, Heikki Linnakangas wrote: > >>I have no problem removing the parameter if required to. In that case, > >>I would like to leave the parameter in until mid beta, to allow > >>greater certainty. In any case, I would wish to retain as a minimum an > >>extern bo

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Heikki Linnakangas
On 01/28/2014 07:15 PM, Bruce Momjian wrote: On Tue, Jan 28, 2014 at 04:36:39PM +, Simon Riggs wrote: For me, reducing the strength of DDL locking is a major change in RDBMS behaviour that could both delight and surprise our users. Maybe a few actually depend upon the locking behaviour, mayb

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Bruce Momjian
On Tue, Jan 28, 2014 at 04:36:39PM +, Simon Riggs wrote: > For me, reducing the strength of DDL locking is a major change in > RDBMS behaviour that could both delight and surprise our users. Maybe > a few actually depend upon the locking behaviour, maybe. After some > years of various people lo

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Simon Riggs
On 28 January 2014 14:55, Bruce Momjian wrote: > On Mon, Jan 27, 2014 at 08:57:26PM +, Simon Riggs wrote: >> We get the new behaviour by default and I expect we'll be very happy with it. >> >> A second thought is that if we have problems of some kind in the field >> as a result of the new lock

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Bruce Momjian
On Mon, Jan 27, 2014 at 08:57:26PM +, Simon Riggs wrote: > We get the new behaviour by default and I expect we'll be very happy with it. > > A second thought is that if we have problems of some kind in the field > as a result of the new lock modes then we will be able to turn them > off. I'm h

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Simon Riggs
On 27 January 2014 20:47, Tom Lane wrote: > Peter Geoghegan writes: >> On Mon, Jan 27, 2014 at 12:25 PM, Robert Haas wrote: >>> I haven't reviewed the patch, but -1 for adding a GUC. > >> I'm pretty surprised that it's been suggested that some people might >> prefer AccessExclusiveLocks. Why wou

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Simon Riggs
On 27 January 2014 20:35, Peter Geoghegan wrote: > On Mon, Jan 27, 2014 at 12:25 PM, Robert Haas wrote: >> I haven't reviewed the patch, but -1 for adding a GUC. > > I'm pretty surprised that it's been suggested that some people might > prefer AccessExclusiveLocks. Why would anyone prefer that?

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Tom Lane
Peter Geoghegan writes: > On Mon, Jan 27, 2014 at 12:25 PM, Robert Haas wrote: >> I haven't reviewed the patch, but -1 for adding a GUC. > I'm pretty surprised that it's been suggested that some people might > prefer AccessExclusiveLocks. Why would anyone prefer that? For one thing, so they can

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Peter Geoghegan
On Mon, Jan 27, 2014 at 12:25 PM, Robert Haas wrote: > I haven't reviewed the patch, but -1 for adding a GUC. I'm pretty surprised that it's been suggested that some people might prefer AccessExclusiveLocks. Why would anyone prefer that? -- Peter Geoghegan -- Sent via pgsql-hackers mailing

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Robert Haas
On Mon, Jan 27, 2014 at 12:58 PM, Simon Riggs wrote: > On 24 January 2014 08:33, Simon Riggs wrote: >> On 24 January 2014 07:08, Peter Geoghegan wrote: >>> On Wed, Jan 15, 2014 at 6:57 AM, Simon Riggs wrote: v15 to fix the above problem. >>> >> v16 attached > > v17 attached > > This versio

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Simon Riggs
On 27 January 2014 17:58, Simon Riggs wrote: > On 24 January 2014 08:33, Simon Riggs wrote: >> On 24 January 2014 07:08, Peter Geoghegan wrote: >>> On Wed, Jan 15, 2014 at 6:57 AM, Simon Riggs wrote: v15 to fix the above problem. >>> >> v16 attached > > v17 attached Frostbite... -- Sim

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Simon Riggs
On 24 January 2014 08:33, Simon Riggs wrote: > On 24 January 2014 07:08, Peter Geoghegan wrote: >> On Wed, Jan 15, 2014 at 6:57 AM, Simon Riggs wrote: >>> v15 to fix the above problem. >> > v16 attached v17 attached This version adds a GUC called ddl_exclusive_locks which allows us to keep the

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-24 Thread Simon Riggs
On 24 January 2014 07:08, Peter Geoghegan wrote: > On Wed, Jan 15, 2014 at 6:57 AM, Simon Riggs wrote: >> v15 to fix the above problem. > > Minor quibble: I get a compiler warning with the patch applied. > "relcache.c: In function 'RememberToFreeTupleDescAtEOX': > relcache.c:2317:3: warning: ISO

  1   2   3   >