Re: Greatest Common Divisor

2020-01-25 Thread Dean Rasheed
On Mon, 20 Jan 2020 at 08:04, Vik Fearing wrote: > > On 20/01/2020 08:44, Dean Rasheed wrote: > >> > > I see this has been marked RFC. I'll take it, > Committed with some adjustments, mostly cosmetic but a couple more substantive: The code to guard against a floating poi

pgsql: Add functions gcd() and lcm() for integer and numeric types.

2020-01-25 Thread Dean Rasheed
Add functions gcd() and lcm() for integer and numeric types. These compute the greatest common divisor and least common multiple of a pair of numbers using the Euclidean algorithm. Vik Fearing, reviewed by Fabien Coelho. Discussion:

Re: Greatest Common Divisor

2020-01-20 Thread Dean Rasheed
On Mon, 20 Jan 2020 at 19:04, Alvaro Herrera wrote: > > On 2020-Jan-20, Dean Rasheed wrote: > > > + > > +greatest common divisor the largest positive number that > > +divides both inputs with no remainder; returns > > 0 if

Re: Greatest Common Divisor

2020-01-20 Thread Dean Rasheed
On Mon, 20 Jan 2020 at 18:52, Vik Fearing wrote: > > On 20/01/2020 11:28, Dean Rasheed wrote: > > > > + > > +least common multiple the smallest strictly positive number > > +that is an integer multiple of both inputs; returns > > 0

Re: Greatest Common Divisor

2020-01-20 Thread Dean Rasheed
Looking at the docs, I think it's worth going a little further than just saying what the acronyms stand for -- especially since the behaviour for zero inputs is an implementation choice (albeit the most common one). I propose the following: + +greatest common divisor the largest

Re: Greatest Common Divisor

2020-01-19 Thread Dean Rasheed
On Tue, 7 Jan 2020 at 12:31, Tom Lane wrote: > > Dean Rasheed writes: > > Do we actually need the smallint versions of these functions? > > Doubt it. It'd be fairly hard even to call those, since e.g. "42" > is an int not a smallint. > I see this has been mar

Re: Errors when update a view with conditional-INSTEAD rules

2020-01-17 Thread Dean Rasheed
On Fri, 17 Jan 2020 at 06:14, Pengzhou Tang wrote: > > I am wondering whether a simple auto-updatable view can have a conditional > update instead rule. Well, the decision reached in [1] was that we wouldn't allow that. We could decide to allow it now as a new feature enhancement, but it

Re: Errors when update a view with conditional-INSTEAD rules

2020-01-14 Thread Dean Rasheed
On Tue, 7 Jan 2020 at 11:00, Dean Rasheed wrote: > > Here's a patch along those lines. Yes, it's a little more code > duplication, but I think it's worth it for the more detailed error. > There was no previous regression test coverage of this case so I added > some (all ot

pgsql: Make rewriter prevent auto-updates on views with conditional INS

2020-01-14 Thread Dean Rasheed
Make rewriter prevent auto-updates on views with conditional INSTEAD rules. A view with conditional INSTEAD rules and no unconditional INSTEAD rules or INSTEAD OF triggers is not auto-updatable. Previously we relied on a check in the executor to catch this, but that's problematic since the

pgsql: Make rewriter prevent auto-updates on views with conditional INS

2020-01-14 Thread Dean Rasheed
Make rewriter prevent auto-updates on views with conditional INSTEAD rules. A view with conditional INSTEAD rules and no unconditional INSTEAD rules or INSTEAD OF triggers is not auto-updatable. Previously we relied on a check in the executor to catch this, but that's problematic since the

pgsql: Make rewriter prevent auto-updates on views with conditional INS

2020-01-14 Thread Dean Rasheed
Make rewriter prevent auto-updates on views with conditional INSTEAD rules. A view with conditional INSTEAD rules and no unconditional INSTEAD rules or INSTEAD OF triggers is not auto-updatable. Previously we relied on a check in the executor to catch this, but that's problematic since the

pgsql: Make rewriter prevent auto-updates on views with conditional INS

2020-01-14 Thread Dean Rasheed
Make rewriter prevent auto-updates on views with conditional INSTEAD rules. A view with conditional INSTEAD rules and no unconditional INSTEAD rules or INSTEAD OF triggers is not auto-updatable. Previously we relied on a check in the executor to catch this, but that's problematic since the

pgsql: Make rewriter prevent auto-updates on views with conditional INS

2020-01-14 Thread Dean Rasheed
Make rewriter prevent auto-updates on views with conditional INSTEAD rules. A view with conditional INSTEAD rules and no unconditional INSTEAD rules or INSTEAD OF triggers is not auto-updatable. Previously we relied on a check in the executor to catch this, but that's problematic since the

pgsql: Make rewriter prevent auto-updates on views with conditional INS

2020-01-14 Thread Dean Rasheed
Make rewriter prevent auto-updates on views with conditional INSTEAD rules. A view with conditional INSTEAD rules and no unconditional INSTEAD rules or INSTEAD OF triggers is not auto-updatable. Previously we relied on a check in the executor to catch this, but that's problematic since the

pgsql: Make rewriter prevent auto-updates on views with conditional INS

2020-01-14 Thread Dean Rasheed
Make rewriter prevent auto-updates on views with conditional INSTEAD rules. A view with conditional INSTEAD rules and no unconditional INSTEAD rules or INSTEAD OF triggers is not auto-updatable. Previously we relied on a check in the executor to catch this, but that's problematic since the

Re: Greatest Common Divisor

2020-01-07 Thread Dean Rasheed
Do we actually need the smallint versions of these functions? I would have thought that automatic casting would take care of any cases that need smallints, and I can't believe that there's any performance benefit to be had that's worth maintaining the extra code. Regards, Dean

Re: Errors when update a view with conditional-INSTEAD rules

2020-01-07 Thread Dean Rasheed
On Sat, 4 Jan 2020 at 18:12, Dean Rasheed wrote: > > On Sat, 4 Jan 2020 at 17:13, Tom Lane wrote: > > > > Dean Rasheed writes: > > > That included a change to rewriteTargetListIU() to prevent it from > > > adding dummy targetlist entries for unassigned-to

Re: Allow WHEN in INSTEAD OF triggers

2020-01-06 Thread Dean Rasheed
On Sat, 28 Dec 2019 at 16:45, David Fetter wrote: > > On Sat, Dec 28, 2019 at 12:12:30AM -0300, Alvaro Herrera wrote: > > On 2019-Dec-28, David Fetter wrote: > > > > > While noodling around with an upcoming patch to remove user-modifiable > > > RULEs, I noticed that WHEN conditions were

Re: [Proposal] Global temporary tables

2020-01-06 Thread Dean Rasheed
On Mon, 6 Jan 2020 at 11:01, Tomas Vondra wrote: > > On Mon, Jan 06, 2020 at 01:04:15PM +0800, 曾文旌(义从) wrote: > > >2 We feel that gtt needs to maintain statistics, but there is no > >agreement on what it will be done. > > > > I certainly agree GTT needs to maintain statistics, otherwise it'll

Re: Greatest Common Divisor

2020-01-04 Thread Dean Rasheed
On Sat, 4 Jan 2020 at 17:55, Vik Fearing wrote: > On 04/01/2020 10:37, Dean Rasheed wrote: > > > > BTW, there is actually no need to restrict the inputs to integral > > values because GCD is something that has a perfectly natural extension > > to floating point

Re: Errors when update a view with conditional-INSTEAD rules

2020-01-04 Thread Dean Rasheed
On Sat, 4 Jan 2020 at 17:13, Tom Lane wrote: > > Dean Rasheed writes: > > That included a change to rewriteTargetListIU() to prevent it from > > adding dummy targetlist entries for unassigned-to attributes for > > auto-updatable views, in case they are no

Re: Errors when update a view with conditional-INSTEAD rules

2020-01-04 Thread Dean Rasheed
On Tue, 3 Dec 2019 at 11:06, Pengzhou Tang wrote: > > Hi Hackers, > > I hit an error when updating a view with conditional INSTEAD OF rules, the > reproduce steps are list below: > > CREATE TABLE t1(a int, b int); > CREATE TABLE t2(a int, b int); > CREATE VIEW v1 AS SELECT * FROM t1 where b >

Re: Greatest Common Divisor

2020-01-04 Thread Dean Rasheed
On Sat, 4 Jan 2020 at 09:37, Dean Rasheed wrote: > > Well Vik has now provided a numeric implementation and it doesn't > appear to be too much code. > BTW, I did a bit of research into the efficiency of Euclid's algorithm. It's actually quite interesting: It turns out that th

Re: Greatest Common Divisor

2020-01-04 Thread Dean Rasheed
On Thu, 2 Jan 2020 at 15:13, Tom Lane wrote: > > Stephen Frost writes: > > * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > >> I'm not objecting to adding it, I'm just curious. In fact, I think > >> that if we do add this, then we should probably add lcm() at the

Re: Greatest Common Divisor

2020-01-02 Thread Dean Rasheed
Out of curiosity, what was the original use-case for this? I'm not objecting to adding it, I'm just curious. In fact, I think that if we do add this, then we should probably add lcm() at the same time, since handling its overflow cases is sufficiently non-trivial to justify not requiring users to

Re: BF failure: could not open relation with OID XXXX while querying pg_views

2019-09-15 Thread Dean Rasheed
On Sun, 15 Sep 2019 at 12:20, Tomas Vondra wrote: > > On Sun, Sep 15, 2019 at 11:27:19AM +0100, Dean Rasheed wrote: > >On Sun, 15 Sep 2019 at 11:11, Tomas Vondra > >wrote: > >> > >> On Sun, Sep 15, 2019 at 10:16:30AM +0100, Dean Rasheed wrote: > >>

pgsql: Fix intermittent self-test failures caused by the stats_ext test

2019-09-15 Thread Dean Rasheed
that unsafe. This led to intermittent failures of the rules test on the buildfarm, although I wasn't able to reproduce that locally. Fix by creating the view in a different schema. Tomas Vondra and Dean Rasheed, report and diagnosis by Thomas Munro. Discussion: https://postgr.es/m/ca

pgsql: Fix intermittent self-test failures caused by the stats_ext test

2019-09-15 Thread Dean Rasheed
that unsafe. This led to intermittent failures of the rules test on the buildfarm, although I wasn't able to reproduce that locally. Fix by creating the view in a different schema. Tomas Vondra and Dean Rasheed, report and diagnosis by Thomas Munro. Discussion: https://postgr.es/m/ca

Re: BF failure: could not open relation with OID XXXX while querying pg_views

2019-09-15 Thread Dean Rasheed
On Sun, 15 Sep 2019 at 11:11, Tomas Vondra wrote: > > On Sun, Sep 15, 2019 at 10:16:30AM +0100, Dean Rasheed wrote: > > > >Ah sorry, I missed this thread before. As author of that commit, it's > >really on me to fix it, and the cause seems pretty clear-cut, so I'll >

Re: Multivariate MCV list vs. statistics target

2019-08-01 Thread Dean Rasheed
On Thu, 1 Aug 2019 at 11:30, Tomas Vondra wrote: > > I'll move it to the next CF. Aside from the issues pointed by Kyotaro-san > in his review, I still haven't made my mind about whether to base the use > statistics targets set for the attributes. That's what we're doing now, > but I'm not sure

Re: ANALYZE: ERROR: tuple already updated by self

2019-07-29 Thread Dean Rasheed
On Sun, 28 Jul 2019 at 11:15, Tomas Vondra wrote: > > Attached is a patch fixing the error by not building extended stats for > the inh=true case (as already proposed in this thread). That's about the > simplest way to resolve this issue for v12. It should add a simple > regression test too, I

Re: Choosing values for multivariate MCV lists

2019-07-01 Thread Dean Rasheed
On Sat, 29 Jun 2019 at 14:01, Tomas Vondra wrote: > > >However, it looks like the problem is with mcv_list_items()'s use > >of %f to convert to text, which is pretty ugly. > > >>>There's one issue with the signature, though - currently the function > >>>returns null flags as bool

Re: pgsql: Drop test user when done with it.

2019-06-25 Thread Dean Rasheed
On Mon, 24 Jun 2019 at 17:37, Tom Lane wrote: > > Drop test user when done with it. > > Commit d7f8d26d9 added a test case that created a user, but forgot > to drop it again. This is no good; for one thing, it causes repeated > "make installcheck" runs to fail. > Ah, I see .. yes, my bad.

Re: Choosing values for multivariate MCV lists

2019-06-24 Thread Dean Rasheed
On Mon, 24 Jun 2019 at 00:42, Tomas Vondra wrote: > > On Sun, Jun 23, 2019 at 10:23:19PM +0200, Tomas Vondra wrote: > >On Sun, Jun 23, 2019 at 08:48:26PM +0100, Dean Rasheed wrote: > >>On Sat, 22 Jun 2019 at 15:10, Tomas Vondra > >>wrote: >

Re: Choosing values for multivariate MCV lists

2019-06-23 Thread Dean Rasheed
On Sat, 22 Jun 2019 at 15:10, Tomas Vondra wrote: > One annoying thing I noticed is that the base_frequency tends to end up > being 0, most likely due to getting too small. It's a bit strange, though, > because with statistic target set to 10k the smallest frequency for a > single column is

Re: Multivariate MCV stats can leak data to unprivileged users

2019-06-23 Thread Dean Rasheed
On Mon, 13 May 2019 at 23:36, Tomas Vondra wrote: > > On Fri, May 10, 2019 at 10:19:44AM +0100, Dean Rasheed wrote: > >While working on 1aebfbea83c, I noticed that the new multivariate MCV > >stats feature suffers from the same problem, and also the original > >

pgsql: Add security checks to the multivariate MCV estimation code.

2019-06-23 Thread Dean Rasheed
to the current user, even if they have table or column privileges. Thus we further insist that the operator be leakproof in this case. Dean Rasheed, reviewed by Tomas Vondra. Discussion: https://postgr.es/m/CAEZATCUhT9rt7Ui=Vdx4N==VV5XOK5dsXfnGgVOz_JhAicB=z...@mail.gmail.com Branch -- master Details

Re: Choosing values for multivariate MCV lists

2019-06-21 Thread Dean Rasheed
On Thu, 20 Jun 2019 at 23:35, Tomas Vondra wrote: > > On Thu, Jun 20, 2019 at 06:55:41AM +0100, Dean Rasheed wrote: > > >I'm not sure it's easy to justify ordering by Abs(freq-base_freq)/freq > >though, because that would seem likely to put too much weight on the > &

Re: Multivariate MCV list vs. statistics target

2019-06-21 Thread Dean Rasheed
On Thu, 20 Jun 2019 at 23:12, Tomas Vondra wrote: > > On Thu, Jun 20, 2019 at 08:08:44AM +0100, Dean Rasheed wrote: > >On Tue, 18 Jun 2019 at 22:34, Tomas Vondra > >wrote: > >> > >> So I'm thinking we should allow tweaking the statistics fo

Re: Choosing values for multivariate MCV lists

2019-06-19 Thread Dean Rasheed
On Tue, 18 Jun 2019 at 21:59, Tomas Vondra wrote: > > The current implementation of multi-column MCV lists (added in this > cycle) uses a fairly simple algorithm to pick combinations to include in > the MCV list. We just compute a minimum number of occurences, and then > include all entries

Re: Multivariate MCV stats can leak data to unprivileged users

2019-06-10 Thread Dean Rasheed
On Thu, 6 Jun 2019 at 21:33, Tomas Vondra wrote: > > Hi, > > Attached are three patches tweaking the stats - two were already posted > in this thread, the third one is just updating docs. > > 1) 0001 - split pg_statistic_ext into definition + data > > This is pretty much the patch Dean posted

Re: Multivariate MCV stats can leak data to unprivileged users

2019-05-20 Thread Dean Rasheed
On Mon, 20 May 2019 at 14:32, Tom Lane wrote: > > Dean Rasheed writes: > > On Sun, 19 May 2019 at 23:45, Tomas Vondra > > wrote: > >> Oh, right. It still has the disadvantage that it obfuscates the actual > >> data stored in the pg_stats_ext_data (or whate

Re: Multivariate MCV stats can leak data to unprivileged users

2019-05-20 Thread Dean Rasheed
On Sun, 19 May 2019 at 23:45, Tomas Vondra wrote: > > Oh, right. It still has the disadvantage that it obfuscates the actual > data stored in the pg_stats_ext_data (or whatever would it be called), > so e.g. functions would have to do additional checks to make sure it > actually is the right

Re: Multivariate MCV stats can leak data to unprivileged users

2019-05-19 Thread Dean Rasheed
On Sun, 19 May 2019 at 15:28, Tom Lane wrote: > > > I wonder ... another way we could potentially do this is > > > create table pg_statistic_ext_data( > > stxoid oid, -- OID of owning pg_statistic_ext entry > > stxkind char, -- what kind of data > > stxdata bytea -- the data, in some

Re: Multivariate MCV stats can leak data to unprivileged users

2019-05-19 Thread Dean Rasheed
On Sun, 19 May 2019 at 00:48, Stephen Frost wrote: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Tomas Vondra writes: > > > > > I think we have four options - rework it before beta1, rework it after > > > beta1, rework it in PG13 and leave it as it is now. > > > > Yup, that's about what the

Re: Multivariate MCV stats can leak data to unprivileged users

2019-05-18 Thread Dean Rasheed
On Sat, 18 May 2019 at 16:13, Tom Lane wrote: > > Dean Rasheed writes: > > On the other hand, pg_dump relies on pg_statistic_ext to work out > > which extended statistics objects to dump. If we were to change that > > to use pg_stats_ext, then a user dumpin

Re: Multivariate MCV stats can leak data to unprivileged users

2019-05-18 Thread Dean Rasheed
On Sat, 18 May 2019 at 10:11, Dean Rasheed wrote: > > On Fri, 17 May 2019 at 21:29, Andres Freund wrote: > > > > On 2019-05-16 14:28:03 +0100, Dean Rasheed wrote: > > > 5). Some columns from pg_statistic_ext have to be made visible for > > > psql \d to

Re: Multivariate MCV stats can leak data to unprivileged users

2019-05-18 Thread Dean Rasheed
On Fri, 17 May 2019 at 21:29, Andres Freund wrote: > > On 2019-05-16 14:28:03 +0100, Dean Rasheed wrote: > > 5). Some columns from pg_statistic_ext have to be made visible for > > psql \d to work. Basically, it needs to be able to query for the > > existence of extended st

Re: Multivariate MCV stats can leak data to unprivileged users

2019-05-16 Thread Dean Rasheed
On Mon, 13 May 2019 at 23:36, Tomas Vondra wrote: > > Yeah. I suggest we add a simple pg_stats_ext view, similar to pg_stats. > It would: > > (1) translate the schema / relation / attribute names > > I don't see why translating column indexes to names would be fiddly. > It's a matter of

Multivariate MCV stats can leak data to unprivileged users

2019-05-10 Thread Dean Rasheed
While working on 1aebfbea83c, I noticed that the new multivariate MCV stats feature suffers from the same problem, and also the original problems that were fixed in e2d4ef8de8 and earlier --- namely that a user can see values in the MCV lists that they shouldn't see (values from tables that they

pgsql: Fix security checks for selectivity estimation functions with RL

2019-05-09 Thread Dean Rasheed
the leakproofness check will only be skipped if there are no securityQuals and the user has table or column privileges on the table -- i.e., only if we know that the user has access to all the data in the column. Back-patch to 9.5 where RLS was added. Dean Rasheed, reviewed by Jonathan Katz and Stephen

pgsql: Fix security checks for selectivity estimation functions with RL

2019-05-09 Thread Dean Rasheed
the leakproofness check will only be skipped if there are no securityQuals and the user has table or column privileges on the table -- i.e., only if we know that the user has access to all the data in the column. Back-patch to 9.5 where RLS was added. Dean Rasheed, reviewed by Jonathan Katz and Stephen

pgsql: Fix security checks for selectivity estimation functions with RL

2019-05-09 Thread Dean Rasheed
the leakproofness check will only be skipped if there are no securityQuals and the user has table or column privileges on the table -- i.e., only if we know that the user has access to all the data in the column. Back-patch to 9.5 where RLS was added. Dean Rasheed, reviewed by Jonathan Katz and Stephen

pgsql: Fix security checks for selectivity estimation functions with RL

2019-05-09 Thread Dean Rasheed
the leakproofness check will only be skipped if there are no securityQuals and the user has table or column privileges on the table -- i.e., only if we know that the user has access to all the data in the column. Back-patch to 9.5 where RLS was added. Dean Rasheed, reviewed by Jonathan Katz and Stephen

pgsql: Fix security checks for selectivity estimation functions with RL

2019-05-09 Thread Dean Rasheed
the leakproofness check will only be skipped if there are no securityQuals and the user has table or column privileges on the table -- i.e., only if we know that the user has access to all the data in the column. Back-patch to 9.5 where RLS was added. Dean Rasheed, reviewed by Jonathan Katz and Stephen

pgsql: Use checkAsUser for selectivity estimator checks, if it's set.

2019-05-06 Thread Dean Rasheed
safeguards controlling access to pg_statistic for direct table access (in which case checkAsUser won't be set) and for security barrier views, because of the nearby checks on rte->security_barrier and rte->securityQuals. Back-patch to all supported branches because e2d4ef8de8 was. Dean R

pgsql: Use checkAsUser for selectivity estimator checks, if it's set.

2019-05-06 Thread Dean Rasheed
safeguards controlling access to pg_statistic for direct table access (in which case checkAsUser won't be set) and for security barrier views, because of the nearby checks on rte->security_barrier and rte->securityQuals. Back-patch to all supported branches because e2d4ef8de8 was. Dean R

pgsql: Use checkAsUser for selectivity estimator checks, if it's set.

2019-05-06 Thread Dean Rasheed
safeguards controlling access to pg_statistic for direct table access (in which case checkAsUser won't be set) and for security barrier views, because of the nearby checks on rte->security_barrier and rte->securityQuals. Back-patch to all supported branches because e2d4ef8de8 was. Dean R

pgsql: Use checkAsUser for selectivity estimator checks, if it's set.

2019-05-06 Thread Dean Rasheed
safeguards controlling access to pg_statistic for direct table access (in which case checkAsUser won't be set) and for security barrier views, because of the nearby checks on rte->security_barrier and rte->securityQuals. Back-patch to all supported branches because e2d4ef8de8 was. Dean R

pgsql: Use checkAsUser for selectivity estimator checks, if it's set.

2019-05-06 Thread Dean Rasheed
safeguards controlling access to pg_statistic for direct table access (in which case checkAsUser won't be set) and for security barrier views, because of the nearby checks on rte->security_barrier and rte->securityQuals. Back-patch to all supported branches because e2d4ef8de8 was. Dean R

pgsql: Use checkAsUser for selectivity estimator checks, if it's set.

2019-05-06 Thread Dean Rasheed
safeguards controlling access to pg_statistic for direct table access (in which case checkAsUser won't be set) and for security barrier views, because of the nearby checks on rte->security_barrier and rte->securityQuals. Back-patch to all supported branches because e2d4ef8de8 was. Dean R

Re: BUG #15708: RLS 'using' running as wrong user when called from a view

2019-04-29 Thread Dean Rasheed
On Mon, 29 Apr 2019 at 04:56, Daurnimator wrote: > > On Wed, 27 Mar 2019 at 23:46, Dean Rasheed wrote: > > On second thoughts, it actually needs to be in > > get_row_security_policies(), after making copies of the quals from the > > policies, otherwise it would be scrib

Multivariate MCV lists -- pg_mcv_list_items() seems to be broken

2019-04-15 Thread Dean Rasheed
I just noticed the following: CREATE TABLE foo (a int, b int); INSERT INTO foo SELECT x/10, x/100 FROM generate_series(1, 100) x; CREATE STATISTICS foo_s ON a,b FROM foo; ANALYSE foo; SELECT pg_mcv_list_items(stxmcv) from pg_statistic_ext WHERE stxname = 'foo_s'; which fails with ERROR: cache

pgsql: Perform RLS subquery checks as the right user when going via a v

2019-04-02 Thread Dean Rasheed
Perform RLS subquery checks as the right user when going via a view. When accessing a table with RLS via a view, the RLS checks are performed as the view owner. However, the code neglected to propagate that to any subqueries in the RLS checks. Fix that by calling setRuleCheckAsUser() for all RLS

pgsql: Perform RLS subquery checks as the right user when going via a v

2019-04-02 Thread Dean Rasheed
Perform RLS subquery checks as the right user when going via a view. When accessing a table with RLS via a view, the RLS checks are performed as the view owner. However, the code neglected to propagate that to any subqueries in the RLS checks. Fix that by calling setRuleCheckAsUser() for all RLS

pgsql: Perform RLS subquery checks as the right user when going via a v

2019-04-02 Thread Dean Rasheed
Perform RLS subquery checks as the right user when going via a view. When accessing a table with RLS via a view, the RLS checks are performed as the view owner. However, the code neglected to propagate that to any subqueries in the RLS checks. Fix that by calling setRuleCheckAsUser() for all RLS

pgsql: Perform RLS subquery checks as the right user when going via a v

2019-04-02 Thread Dean Rasheed
Perform RLS subquery checks as the right user when going via a view. When accessing a table with RLS via a view, the RLS checks are performed as the view owner. However, the code neglected to propagate that to any subqueries in the RLS checks. Fix that by calling setRuleCheckAsUser() for all RLS

pgsql: Perform RLS subquery checks as the right user when going via a v

2019-04-02 Thread Dean Rasheed
Perform RLS subquery checks as the right user when going via a view. When accessing a table with RLS via a view, the RLS checks are performed as the view owner. However, the code neglected to propagate that to any subqueries in the RLS checks. Fix that by calling setRuleCheckAsUser() for all RLS

Re: BUG #15708: RLS 'using' running as wrong user when called from a view

2019-03-27 Thread Dean Rasheed
On Mon, 25 Mar 2019 at 20:27, Stephen Frost wrote: > > * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > > > It looks like the best place to fix it is in > > get_policies_for_relation(), since that's where all the policies to be > > applied for a given RTE are pull

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-26 Thread Dean Rasheed
On Tue, 26 Mar 2019 at 11:59, Dean Rasheed wrote: > > On Mon, 25 Mar 2019 at 23:36, Tomas Vondra > wrote: > > > > Attached is an updated patch... > > I just looked through the latest set of changes and I have a couple of > additional review comments: > I

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-26 Thread Dean Rasheed
On Mon, 25 Mar 2019 at 23:36, Tomas Vondra wrote: > > Attached is an updated patch, fixing all the issues pointed out so far. > Unless there are some objections, I plan to commit the 0001 part by the > end of this CF. Part 0002 is a matter for PG13, as previously agreed. > Yes, I think that's

Re: BUG #15708: RLS 'using' running as wrong user when called from a view

2019-03-24 Thread Dean Rasheed
On Thu, 21 Mar 2019 at 00:39, PG Bug reporting form wrote: > > This fails, seemingly because the RLS on 'bar' is being checked by alice, > instead of the view owner bob: > Yes I agree, that appears to be a bug. The subquery in the RLS policy should be checked as the view owner -- i.e., we need

Re: regr_slope returning NULL

2019-03-24 Thread Dean Rasheed
On Sun, 24 Mar 2019 at 08:01, Steve Baldwin wrote: > > Thanks Tom, > > I've tried this on 11.2 (OS X 10.14.3, installed locally) and 10.6 (AWS RDS) > instances with identical results. The values you show are identical to those > returned by Oracle so that's great but why am I seeing different

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-24 Thread Dean Rasheed
On Sun, 24 Mar 2019 at 00:17, David Rowley wrote: > > On Sun, 24 Mar 2019 at 12:41, Tomas Vondra > wrote: > > > > On 3/21/19 4:05 PM, David Rowley wrote: > > > > 29. Looking at the tests I see you're testing that you get bad > > > estimates without extended stats. That does not really seem

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-17 Thread Dean Rasheed
On Sat, 16 Mar 2019 at 23:44, Tomas Vondra wrote: > > > > 16). This regression test fails for me: > > > > @@ -654,11 +654,11 @@ > > -- check change of unrelated column type does not reset the MCV statistics > > ALTER TABLE mcv_lists ALTER COLUMN d TYPE VARCHAR(64); > > SELECT * FROM

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-17 Thread Dean Rasheed
On Sat, 16 Mar 2019 at 23:44, Tomas Vondra wrote: > > > 28). I just spotted the 1MB limit on the serialised MCV list size. I > > think this is going to be too limiting. For example, if the stats > > target is at its maximum of 1, that only leaves around 100 bytes > > for each item's values,

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-17 Thread Dean Rasheed
On Sat, 16 Mar 2019 at 23:44, Tomas Vondra wrote: > > > 21). For consistency with other bms_ functions, I think the name of > > the Bitmapset argument for bms_member_index() should just be called > > "a". Nitpicking, I'd also put bms_member_index() immediately after > > bms_is_member() in the

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-16 Thread Dean Rasheed
On Fri, 15 Mar 2019 at 00:06, Tomas Vondra wrote: > ... attached patch ... Some more review comments, carrying on from where I left off: 16). This regression test fails for me: @@ -654,11 +654,11 @@ -- check change of unrelated column type does not reset the MCV statistics ALTER TABLE

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-16 Thread Dean Rasheed
On Fri, 15 Mar 2019 at 00:06, Tomas Vondra wrote: > I've noticed an annoying thing when modifying type of column not > included in a statistics... > > That is, we don't remove the statistics, but the estimate still changes. > But that's because the ALTER TABLE also resets reltuples/relpages: > >

Re: pgsql: Add support for hyperbolic functions, as well as log10().

2019-03-14 Thread Dean Rasheed
On Thu, 14 Mar 2019 at 04:41, Tom Lane wrote: > > Dean Rasheed writes: > > I'm amazed that jacana's asinh() returned -0 for an input of +0. > > Even more amusingly, it returns NaN for acosh('infinity'), cf > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jacana=2019

Re: pgsql: Add support for hyperbolic functions, as well as log10().

2019-03-13 Thread Dean Rasheed
On Wed, 13 Mar 2019, 21:56 Tom Lane, wrote: > > Of these, probably the least bad is #3, even though it might require > a few rounds of experimentation to find the best extra_float_digits > setting to use. I'll go try it without any roundoff, just to see > what the raw results look like ... >

Re: [Patch] Log10 and hyperbolic functions for SQL:2016 compliance

2019-03-11 Thread Dean Rasheed
On Sun, 3 Feb 2019 at 15:12, Tom Lane wrote: > > Andrew Gierth writes: > > The spec doesn't require the inverse functions (asinh, acosh, atanh), > > but surely there is no principled reason to omit them? > > +1 --- AFAICS, the C library has offered all six since C89. > +1 for including the

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-11 Thread Dean Rasheed
On Sun, 10 Mar 2019 at 22:28, David Rowley wrote: > > On Mon, 11 Mar 2019 at 06:36, Tomas Vondra > wrote: > > > > On 3/9/19 7:33 PM, Dean Rasheed wrote: > > > I wonder if it's possible to write smaller, more targeted tests. > > > Currently "stats_

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-11 Thread Dean Rasheed
On Sun, 10 Mar 2019 at 17:36, Tomas Vondra wrote: > On 3/10/19 2:09 PM, Dean Rasheed wrote: > > 14). The attnums Bitmapset passed to > > statext_is_compatible_clause_internal() is an input/output argument > > that it updates. That should probably be documented. Whe

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-10 Thread Dean Rasheed
On Sun, 10 Mar 2019 at 13:09, Dean Rasheed wrote: > Here are some more comments: > One more thing --- the comment for statext_clauselist_selectivity() says: * So (simple_selectivity - base_selectivity) may be seen as a correction for * the part not covered by the MCV list. That's not

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-10 Thread Dean Rasheed
On Sat, 9 Mar 2019 at 18:33, Dean Rasheed wrote: > > On Thu, 28 Feb 2019 at 19:56, Tomas Vondra > wrote: > > Attached is an updated version of this patch series. > > Here are some random review comments. I'll add more later, but I'm out > of energy for today. > Here

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-09 Thread Dean Rasheed
On Thu, 28 Feb 2019 at 19:56, Tomas Vondra wrote: > Attached is an updated version of this patch series. Here are some random review comments. I'll add more later, but I'm out of energy for today. 1). src/test/regress/expected/type_sanity.out has bit-rotted. 2). Duplicate OIDs (3425). 3). It

pgsql: Further fixing for multi-row VALUES lists for updatable views.

2019-03-03 Thread Dean Rasheed
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE,

pgsql: Further fixing for multi-row VALUES lists for updatable views.

2019-03-03 Thread Dean Rasheed
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE,

pgsql: Further fixing for multi-row VALUES lists for updatable views.

2019-03-03 Thread Dean Rasheed
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE,

pgsql: Further fixing for multi-row VALUES lists for updatable views.

2019-03-03 Thread Dean Rasheed
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE,

pgsql: Further fixing for multi-row VALUES lists for updatable views.

2019-03-03 Thread Dean Rasheed
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE,

pgsql: Further fixing for multi-row VALUES lists for updatable views.

2019-03-03 Thread Dean Rasheed
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE,

Re: Row Level Security − leakproof-ness and performance implications

2019-02-28 Thread Dean Rasheed
On Thu, 28 Feb 2019 at 14:13, Robert Haas wrote: > A wild idea might be to let > proleakproof take on three values: yes, no, and maybe. When 'maybe' > functions are involved, we tell them whether or not the current query > involves any security barriers, and if so they self-censor. > Does

Re: BUG #15623: Inconsistent use of default for updatable view

2019-02-28 Thread Dean Rasheed
On Thu, 28 Feb 2019 at 07:47, Amit Langote wrote: > > +if (attrno == 0) > +elog(ERROR, "Cannot set value in column %d to > DEFAULT", i); > > Maybe: s/Cannot/cannot/g > Ah yes, you're right. That is the convention. > +Assert(list_length(sublist) ==

Re: BUG #15623: Inconsistent use of default for updatable view

2019-02-27 Thread Dean Rasheed
On Tue, 12 Feb 2019 at 10:33, Dean Rasheed wrote: > Here's an updated patch ... So I pushed that. However, ... Playing around with it some more, I realised that whilst this appeared to fix the reported problem, it exposes another issue which is down to the interaction betw

Re: INSERT ... OVERRIDING USER VALUE vs GENERATED ALWAYS identity columns

2019-02-25 Thread Dean Rasheed
On Mon, 25 Feb 2019 at 12:47, Peter Eisentraut wrote: > > - and the column in new rows will automatically have values from the > > - sequence assigned to it. > > + and new rows in the column will automatically have values from the > > + sequence assigned to them. > > The "it"

INSERT ... OVERRIDING USER VALUE vs GENERATED ALWAYS identity columns

2019-02-22 Thread Dean Rasheed
So I started looking into the bug noted in [1], but before getting to multi-row inserts, I concluded that the current single-row behaviour isn't spec-compliant. In particular, Syntax Rule 11b of section 14.11 says that an INSERT statement on a GENERATED ALWAYS identity column must specify an

pgsql: Fix DEFAULT-handling in multi-row VALUES lists for updatable vie

2019-02-20 Thread Dean Rasheed
Fix DEFAULT-handling in multi-row VALUES lists for updatable views. INSERT ... VALUES for a single VALUES row is implemented differently from a multi-row VALUES list, which causes inconsistent behaviour in the way that DEFAULT items are handled. In particular, when inserting into an

pgsql: Fix DEFAULT-handling in multi-row VALUES lists for updatable vie

2019-02-20 Thread Dean Rasheed
Fix DEFAULT-handling in multi-row VALUES lists for updatable views. INSERT ... VALUES for a single VALUES row is implemented differently from a multi-row VALUES list, which causes inconsistent behaviour in the way that DEFAULT items are handled. In particular, when inserting into an

<    2   3   4   5   6   7   8   9   10   11   >