Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-03-06 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 26 February 2015 at 09:50, Dean Rasheed wrote: > > On 26 February 2015 at 05:43, Stephen Frost wrote: > >> I wonder if there are some documentation updates which need to be done > >> for this also? I'm planning to look as I vauguely recall me

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-03-06 Thread Dean Rasheed
On 26 February 2015 at 09:50, Dean Rasheed wrote: > On 26 February 2015 at 05:43, Stephen Frost wrote: >> I wonder if there are some documentation updates which need to be done >> for this also? I'm planning to look as I vauguely recall mentioning the >> ordering of operations somewhere along th

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-03-02 Thread Dean Rasheed
On 26 February 2015 at 09:27, Dean Rasheed wrote: > I think this should probably be committed as 2 separate patches... On closer inspection, the 2 parts are interrelated since the new self-join test that tests the inheritance planner changes also requires the rewriter changes, otherwise it falls

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-02-26 Thread Dean Rasheed
On 26 February 2015 at 05:43, Stephen Frost wrote: > Dean, > > * Dean Rasheed (dean.a.rash...@gmail.com) wrote: >> Attached is a patch to make RLS checks run before attempting to >> insert/update any data rather than afterwards. > > Excellent, this I really like and it's a pretty straight-forward

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-02-26 Thread Dean Rasheed
On 26 February 2015 at 05:41, Stephen Frost wrote: > Dean, > > * Dean Rasheed (dean.a.rash...@gmail.com) wrote: >> Here's an updated patch with a new test for this bug. I've been >> developing the fixes for these RLS issues as one big patch, but I >> suppose it would be easy to split up, if that's

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-02-25 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > Attached is a patch to make RLS checks run before attempting to > insert/update any data rather than afterwards. Excellent, this I really like and it's a pretty straight-forward change. I wonder if there are some documentation updates which

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-02-25 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > Here's an updated patch with a new test for this bug. I've been > developing the fixes for these RLS issues as one big patch, but I > suppose it would be easy to split up, if that's preferred. Thanks for working on all of this! I've brough

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-19 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > Attached is a patch to make RLS checks run before attempting to > insert/update any data rather than afterwards. Excellent, many thanks! > In the end I decided not to create a new structure for RLS checks > because most of the code that ha

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-19 Thread Dean Rasheed
On 10 January 2015 at 15:12, Stephen Frost wrote: > * Dean Rasheed (dean.a.rash...@gmail.com) wrote: >> Currently we're applying RLS CHECKs after the INSERT or UPDATE, like >> WITH CHECK OPTIONs on views. The SQL spec says that WITH CHECK OPTIONs >> on views have to be applied after the INSERT/UPD

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-14 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: > Turns out it wasn't as simple as that. prepend_row_security_policies() > really could get called multiple times for the same RTE, because the > call to query_tree_walker() at the end of fireRIRrules() would descend > into the just-added quals again

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-14 Thread Dean Rasheed
On 14 January 2015 at 08:43, Dean Rasheed wrote: > On 12 January 2015 at 14:24, Stephen Frost wrote: >> Interesting, thanks for the work! I had been suspicious that there was >> an issue with the recursion handling. >> > > So continuing to review the RLS code, I spotted the following in > prepen

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-14 Thread Dean Rasheed
On 12 January 2015 at 14:24, Stephen Frost wrote: > Interesting, thanks for the work! I had been suspicious that there was > an issue with the recursion handling. > So continuing to review the RLS code, I spotted the following in prepend_row_security_policies(): /* * We may end up gett

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-12 Thread Dean Rasheed
On 12 January 2015 at 14:24, Stephen Frost wrote: > Looking at the regression tests a bit though, I notice that this removes > the lower-level LockRows.. There had been much discussion about that > last spring which I believe you were a part of..? I specifically recall > discussing it with Craig

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-12 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 10 January 2015 at 21:38, Dean Rasheed wrote: > > OK, I'll take a look at it. I started reading the existing code that > > expands RLS policies and spotted a couple of bugs that should be fixed > > first:- > > > > 1). In prepend_row_secu

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-12 Thread Dean Rasheed
On 10 January 2015 at 21:38, Dean Rasheed wrote: > OK, I'll take a look at it. I started reading the existing code that > expands RLS policies and spotted a couple of bugs that should be fixed > first:- > > 1). In prepend_row_security_policies(), defaultDeny should be > initialised to false at the

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-10 Thread Dean Rasheed
On 10 January 2015 at 15:12, Stephen Frost wrote: >> NOTE: If we do change RLS CHECKs to be executed prior to modifying any >> data, that's potentially a change that could be made independently of >> the UPSERT patch. We should also probably then stop referring to them >> as WITH CHECK OPTIONs in

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-10 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 9 January 2015 at 20:26, Stephen Frost wrote: > > Where this leaves me, at least, is feeling like we should always apply > > the INSERT WITH CHECK policy, then if there is a conflict, check the > > UPDATE USING policy and throw an error

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-10 Thread Dean Rasheed
On 9 January 2015 at 20:26, Stephen Frost wrote: > Where this leaves me, at least, is feeling like we should always apply > the INSERT WITH CHECK policy, then if there is a conflict, check the > UPDATE USING policy and throw an error if the row isn't visible but > otherwise perform the UPDATE and

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Stephen Frost
* Peter Geoghegan (p...@heroku.com) wrote: > On Fri, Jan 9, 2015 at 1:09 PM, Stephen Frost wrote: > > Therefore, > > I'm not sure that I see the point in checking the INSERT tuple against > > the UPDATE policy. > > I guess it wouldn't be hard to modify the struct WithCheckOption to > store the cm

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Peter Geoghegan
On Fri, Jan 9, 2015 at 1:09 PM, Stephen Frost wrote: > To flip it around a bit, I don't think we can avoid checking the > *resulting* tuple from the UPDATE against the UPDATE policy. We can avoid it - by not updating. What I'm suggesting is that an enforcement occurs that is more or less equivale

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Stephen Frost
* Peter Geoghegan (p...@heroku.com) wrote: > On Fri, Jan 9, 2015 at 12:26 PM, Stephen Frost wrote: > > Where this leaves me, at least, is feeling like we should always apply > > the INSERT WITH CHECK policy, then if there is a conflict, check the > > UPDATE USING policy and throw an error if the r

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Peter Geoghegan
On Fri, Jan 9, 2015 at 12:53 PM, Stephen Frost wrote: > I'm not sure how that would work exactly though, since the tuple the > UPDATE results in might be different from what the INSERT has, as Dean > pointed out. The INSERT tuple might even pass the UPDATE policy where > the resulting tuple from

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Peter Geoghegan
On Fri, Jan 9, 2015 at 12:26 PM, Stephen Frost wrote: > Where this leaves me, at least, is feeling like we should always apply > the INSERT WITH CHECK policy, then if there is a conflict, check the > UPDATE USING policy and throw an error if the row isn't visible but > otherwise perform the UPDATE

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Stephen Frost
Peter, * Peter Geoghegan (p...@heroku.com) wrote: > On Fri, Jan 9, 2015 at 2:22 AM, Dean Rasheed wrote: > > Whoa, hang on. I think you're being a bit quick to dismiss that > > example. Why shouldn't I want an upsert where the majority of the > > table columns follow the usual "make it so" pattern

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Peter Geoghegan
On Fri, Jan 9, 2015 at 2:22 AM, Dean Rasheed wrote: > Whoa, hang on. I think you're being a bit quick to dismiss that > example. Why shouldn't I want an upsert where the majority of the > table columns follow the usual "make it so" pattern of an upsert, but > there is also this kind of audit colum

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Stephen Frost
Dean, Peter, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 9 January 2015 at 08:49, Peter Geoghegan wrote: > > On Fri, Jan 9, 2015 at 12:19 AM, Dean Rasheed > > wrote: > >> I was trying to think up an example where you might actually have > >> different INSERT and UPDATE policies, and t

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Dean Rasheed
On 9 January 2015 at 08:49, Peter Geoghegan wrote: > On Fri, Jan 9, 2015 at 12:19 AM, Dean Rasheed > wrote: >> I was trying to think up an example where you might actually have >> different INSERT and UPDATE policies, and the best I can think of is >> some sort of mod_count column where you have

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Peter Geoghegan
On Fri, Jan 9, 2015 at 12:19 AM, Dean Rasheed wrote: > I was trying to think up an example where you might actually have > different INSERT and UPDATE policies, and the best I can think of is > some sort of mod_count column where you have an INSERT CHECK > (mod_count = 0) and an UPDATE CHECK (mod_

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Dean Rasheed
On 9 January 2015 at 00:49, Stephen Frost wrote: > Peter, > > * Peter Geoghegan (p...@heroku.com) wrote: >> For column level privileges, you wouldn't expect to only get an error >> about not having the relevant update permissions at runtime, when the >> update path happens to be taken. And so it i

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-08 Thread Stephen Frost
Peter, * Peter Geoghegan (p...@heroku.com) wrote: > For column level privileges, you wouldn't expect to only get an error > about not having the relevant update permissions at runtime, when the > update path happens to be taken. And so it is for RLS. Right, that's the precedent we should be consi

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Peter Geoghegan
On Wed, Jan 7, 2015 at 5:33 PM, David Fetter wrote: > Same database, same constraints, different outcome. You're missing the point, I think. UPSERT means that the user doesn't care about whether or not a given tuple proposed for insertion was handled with an insert or an update. If you have disti

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread David Fetter
On Wed, Jan 07, 2015 at 05:18:58PM -0800, Peter Geoghegan wrote: > On Wed, Jan 7, 2015 at 5:16 PM, David Fetter wrote: > > There's precedent. Unique constraints, for example. > > I don't see that as any kind of precedent. In the part you sliced off, Stephen described a situation where the conte

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Peter Geoghegan
On Wed, Jan 7, 2015 at 5:16 PM, David Fetter wrote: > There's precedent. Unique constraints, for example. I don't see that as any kind of precedent. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.post

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread David Fetter
On Wed, Jan 07, 2015 at 03:01:20PM -0500, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: > > On Wed, Jan 7, 2015 at 4:04 AM, Dean Rasheed > > wrote: > > > I think the policies applied should depend on the path taken, so if it > > > does an INSERT, then only the INSERT CHECK p

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Peter Geoghegan
On Wed, Jan 7, 2015 at 12:26 PM, Stephen Frost wrote: > I agree with that up to a point- this case feels more like a POLA > violation though rather than a run-of-the-mill "you need to test all > that." I'm a bit worried this might lead to cases where users can't use > UPSERT because they don't kn

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Peter Geoghegan
On Wed, Jan 7, 2015 at 12:01 PM, Stephen Frost wrote: > Other databases have this capability and have triggers and at least one > ends up firing both INSERT and UPDATE triggers, with many complaints > from users about how that ends up making the performance suck. Perhaps > we could use that as a

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jan 7, 2015 at 3:01 PM, Stephen Frost wrote: > > * Robert Haas (robertmh...@gmail.com) wrote: > >> On Wed, Jan 7, 2015 at 4:04 AM, Dean Rasheed > >> wrote: > >> > I think the policies applied should depend on the path taken, so if it > >> >

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Robert Haas
On Wed, Jan 7, 2015 at 3:01 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Wed, Jan 7, 2015 at 4:04 AM, Dean Rasheed >> wrote: >> > I think the policies applied should depend on the path taken, so if it >> > does an INSERT, then only the INSERT CHECK policy should

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jan 7, 2015 at 4:04 AM, Dean Rasheed wrote: > > I think the policies applied should depend on the path taken, so if it > > does an INSERT, then only the INSERT CHECK policy should be applied > > (after the insert), but if it ends up doing an U

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 6 January 2015 at 20:44, Peter Geoghegan wrote: > > Another issue that I see is that there is only one resultRelation > > between the INSERT and UPDATE. That means that without some additional > > special measure, both UPDATE and INSERT "WITH C

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Robert Haas
On Wed, Jan 7, 2015 at 4:04 AM, Dean Rasheed wrote: > I think the policies applied should depend on the path taken, so if it > does an INSERT, then only the INSERT CHECK policy should be applied > (after the insert), but if it ends up doing an UPDATE, I would expect > the UPDATE USING policy to be

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Dean Rasheed
On 6 January 2015 at 20:44, Peter Geoghegan wrote: > Another issue that I see is that there is only one resultRelation > between the INSERT and UPDATE. That means that without some additional > special measure, both UPDATE and INSERT "WITH CHECK ( ... ) " options > are applied regardless of whethe

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-06 Thread Peter Geoghegan
Another issue that I see is that there is only one resultRelation between the INSERT and UPDATE. That means that without some additional special measure, both UPDATE and INSERT "WITH CHECK ( ... ) " options are applied regardless of whether the INSERT path was taken, or the UPDATE path. Maybe that'

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-06 Thread Peter Geoghegan
On Tue, Jan 6, 2015 at 9:39 AM, Robert Haas wrote: > I think the INSERT .. ON CONFLICT UPDATE shouldn't be able to attempt > an update unless the UPDATE policies of the table are such that a > regular UPDATE would find the affected row. The post-image of the row > needs to satisfy any UPDATE CHEC

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-06 Thread Robert Haas
On Mon, Jan 5, 2015 at 12:54 PM, Peter Geoghegan wrote: > The patch that implements INSERT ... ON CONFLICT UPDATE has support > and tests for per-column privileges (which are not relevant to the > IGNORE variant, AFAICT). However, RLS support is another thing > entirely. It has not been properly t

[HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-05 Thread Peter Geoghegan
The patch that implements INSERT ... ON CONFLICT UPDATE has support and tests for per-column privileges (which are not relevant to the IGNORE variant, AFAICT). However, RLS support is another thing entirely. It has not been properly thought out, and unlike per-column privileges requires careful con