Re: Allowing extensions to supply operator-/function-specific info

2019-03-05 Thread Tom Lane
Paul Ramsey writes: >> On Mar 5, 2019, at 3:56 PM, Tom Lane wrote: >> Then you're at least missing adequate tests for the 3-arg functions... >> 3 args with the index column second will not work as this stands. > Some of the operators are indifferent to order (&&, overlaps) and others are > not

Re: Allowing extensions to supply operator-/function-specific info

2019-03-05 Thread Paul Ramsey
> On Mar 5, 2019, at 3:56 PM, Tom Lane wrote: > > Paul Ramsey writes: >> On Mar 5, 2019, at 3:26 PM, Tom Lane wrote: >>> Hm, I think your addition of this bit is wrong: >>> >>> +/* >>> +* Arguments were swapped to put the index value on the >>> +

Re: Allowing extensions to supply operator-/function-specific info

2019-03-05 Thread Tom Lane
Paul Ramsey writes: > On Mar 5, 2019, at 3:26 PM, Tom Lane wrote: >> Hm, I think your addition of this bit is wrong: >> >> +/* >> +* Arguments were swapped to put the index value on the >> +* left, so we need the commutated operator for

Re: Allowing extensions to supply operator-/function-specific info

2019-03-05 Thread Paul Ramsey
> On Mar 5, 2019, at 3:26 PM, Tom Lane wrote: > > Paul Ramsey writes: >> Thanks for the patch, I’ve applied and smoothed and taken your advice on >> schema-qualified lookups as well. > > Hm, I think your addition of this bit is wrong: > > +/* > +* Arg

Re: Allowing extensions to supply operator-/function-specific info

2019-03-05 Thread Tom Lane
Paul Ramsey writes: > Thanks for the patch, I’ve applied and smoothed and taken your advice on > schema-qualified lookups as well. Hm, I think your addition of this bit is wrong: +/* +* Arguments were swapped to put the index value on the +

Re: Allowing extensions to supply operator-/function-specific info

2019-03-05 Thread Paul Ramsey
> On Mar 4, 2019, at 4:22 PM, Tom Lane wrote: > > Paul Ramsey writes: >>> On Mar 4, 2019, at 2:52 PM, Tom Lane wrote: >>> BTW, if you'd like me to review the code you added for this, I'd be happy >>> to do so. I've never looked at PostGIS' innards, but probably I can make >>> sense of the c

Re: Allowing extensions to supply operator-/function-specific info

2019-03-04 Thread Tom Lane
Paul Ramsey writes: >> On Mar 4, 2019, at 2:52 PM, Tom Lane wrote: >> BTW, if you'd like me to review the code you added for this, I'd be happy >> to do so. I've never looked at PostGIS' innards, but probably I can make >> sense of the code for this despite that. > I would be ecstatic for a rev

Re: Allowing extensions to supply operator-/function-specific info

2019-03-04 Thread Paul Ramsey
> On Mar 4, 2019, at 2:52 PM, Tom Lane wrote: > > Paul Ramsey writes: >> Gotcha, done and now have an implementation that passes all our regression >> tests. > > Very cool! So the next step, I guess, is to address your original problem > by cranking up the cost estimates for these functions

Re: Allowing extensions to supply operator-/function-specific info

2019-03-04 Thread Tom Lane
Paul Ramsey writes: > Gotcha, done and now have an implementation that passes all our regression > tests. Very cool! So the next step, I guess, is to address your original problem by cranking up the cost estimates for these functions --- have you tried that yet? In principle you should be able

Re: Allowing extensions to supply operator-/function-specific info

2019-03-04 Thread Paul Ramsey
> On Mar 4, 2019, at 1:13 PM, Tom Lane wrote: > > Paul Ramsey writes: >> I had what seemed to be working code except for a couple rare cases, >> but when I fixed those cases it turned out that I had a major problem: >> building a OP expression works fine, but building a >> OP expression r

Re: Allowing extensions to supply operator-/function-specific info

2019-03-04 Thread Tom Lane
Paul Ramsey writes: > I had what seemed to be working code except for a couple rare cases, > but when I fixed those cases it turned out that I had a major problem: > building a OP expression works fine, but building a > OP expression returns me an error. Yup, you're not supposed to do that.

Re: Allowing extensions to supply operator-/function-specific info

2019-03-04 Thread Paul Ramsey
So I am getting much closer to a working implementation in PostGIS, but have just run into an issue which I am assuming is my misunderstanding something... https://github.com/pramsey/postgis/blob/92268c94f3aa1fc63a2941f2b451be15b28662cf/postgis/gserialized_supportfn.c#L287 I had what seemed to be

Re: Allowing extensions to supply operator-/function-specific info

2019-02-27 Thread Tom Lane
Paul Ramsey writes: > I added three indexes to my test table: > CREATE INDEX foo_g_gist_x ON foo USING GIST (g); > CREATE INDEX foo_g_gist_nd_x ON foo USING GIST (g gist_geometry_ops); > CREATE INDEX foo_g_spgist_x ON foo USING SPGIST (g); > They all support the overlaps (&&) operator. > So

Re: Allowing extensions to supply operator-/function-specific info

2019-02-27 Thread Paul Ramsey
> On Feb 27, 2019, at 3:40 PM, Tom Lane wrote: > >> Variable SupportRequestCost is very exciting, but given that variable cost >> is usually driven by the complexity of arguments, what kind of argument is >> the SupportRequestCost call fed during the planning stage? Constant >> arguments are

Re: Allowing extensions to supply operator-/function-specific info

2019-02-27 Thread Tom Lane
Paul Ramsey writes: > The documentation says that a support function should have a signature > "supportfn(internal) returns internal”, but doesn’t say which (if any) > annotations should be provided. IMMUTABLE? PARALLEL SAFE? STRICT? None? All? It doesn't matter much given that these things are

Re: Allowing extensions to supply operator-/function-specific info

2019-02-27 Thread Paul Ramsey
A few more questions… The documentation says that a support function should have a signature "supportfn(internal) returns internal”, but doesn’t say which (if any) annotations should be provided. IMMUTABLE? PARALLEL SAFE? STRICT? None? All? Variable SupportRequestCost is very exciting, but give

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Tom Lane
Paul Ramsey writes: >> On Feb 26, 2019, at 2:19 PM, Tom Lane wrote: >> In most cases, multiple matching arguments are going to lead to >> failure to construct any useful index condition, because your >> comparison value has to be a pseudoconstant (ie, not a variable >> from the same table, so in

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Paul Ramsey
> On Feb 26, 2019, at 2:19 PM, Tom Lane wrote: > > In most cases, multiple matching arguments are going to lead to > failure to construct any useful index condition, because your > comparison value has to be a pseudoconstant (ie, not a variable > from the same table, so in both of the above exa

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Tom Lane
Paul Ramsey writes: > On Feb 26, 2019, at 2:19 PM, Tom Lane wrote: >> What's the query look like exactly? The other two calls will occur >> anyway, but SupportRequestIndexCondition depends on the function >> call's placement. > select geos_intersects_new(g, 'POINT(0 0)') from foo; Right, so th

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Paul Ramsey
> On Feb 26, 2019, at 2:19 PM, Tom Lane wrote: > >> I have >> created a table (foo) a geometry column (g) and an index (GIST on >> foo(g)) and am running a query against foo using a noop function with >> a support function bound to it. > >> The support function is called, twice, once in >> T_S

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Tom Lane
Paul Ramsey writes: > New line of questioning: under what conditions will the support > function be called in a T_SupportRequestIndexCondition mode? It'll be called if the target function appears at top level of a WHERE or JOIN condition and any one of the function's arguments syntactically match

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Paul Ramsey
On Mon, Feb 25, 2019 at 4:09 PM Tom Lane wrote: > > Paul Ramsey writes: > > On Mon, Feb 25, 2019 at 3:01 PM Tom Lane wrote: > >> It's whichever one the index column's opclass belongs to. Basically what > >> you're trying to do here is verify whether the index will support the > >> optimization

Re: Allowing extensions to supply operator-/function-specific info

2019-02-25 Thread Tom Lane
Paul Ramsey writes: > On Mon, Feb 25, 2019 at 3:01 PM Tom Lane wrote: >> It's whichever one the index column's opclass belongs to. Basically what >> you're trying to do here is verify whether the index will support the >> optimization you want to perform. > * If I have tbl1.geom > * and I have

Re: Allowing extensions to supply operator-/function-specific info

2019-02-25 Thread Paul Ramsey
On Mon, Feb 25, 2019 at 3:01 PM Tom Lane wrote: > > Looking at the examples, they are making use of the opfamily that > > comes in SupportRequestIndexCondition.opfamily. > > That opfamily Oid is the first one in the IndexOptInfo.opfamily array. > > Here's where my thread of understanding fails to

Re: Allowing extensions to supply operator-/function-specific info

2019-02-25 Thread Tom Lane
Paul Ramsey writes: > So... trying to figure out how to use SupportRequestIndexCondition to > convert a call to Intersects() in to a call that also has the operator > && as well. OK. > Looking at the examples, they are making use of the opfamily that > comes in SupportRequestIndexCondition.opfam

Re: Allowing extensions to supply operator-/function-specific info

2019-02-25 Thread Paul Ramsey
On Mon, Jan 28, 2019 at 9:51 AM Tom Lane wrote: > is people like PostGIS, who already cleared that bar. I hope that > we'll soon have a bunch of examples, like those in the 0004 patch, > that people can look at to see how to do things in this area. I see > no reason to believe it'll be all that

Re: Allowing extensions to supply operator-/function-specific info

2019-01-29 Thread Tom Lane
Simon Riggs writes: > On Tue, 29 Jan 2019 at 09:55, Tom Lane wrote: >> I have no particular >> interest in working on that right now, because it doesn't respond to >> what I understand PostGIS' need to be, and there are only so many >> hours in the day. But maybe it could be made workable in the

Re: Allowing extensions to supply operator-/function-specific info

2019-01-28 Thread Simon Riggs
On Tue, 29 Jan 2019 at 09:55, Tom Lane wrote: > Simon Riggs writes: > > On Sun, 27 Jan 2019 at 19:17, Tom Lane wrote: > >> ... I don't > >> know whether that would satisfy your concern, because I'm not clear > >> on what your concern is. > > > To be able to extract indexable clauses where none

Re: Allowing extensions to supply operator-/function-specific info

2019-01-28 Thread Tom Lane
Simon Riggs writes: > On Sun, 27 Jan 2019 at 19:17, Tom Lane wrote: >> ... I don't >> know whether that would satisfy your concern, because I'm not clear >> on what your concern is. > To be able to extract indexable clauses where none existed before. That's a pretty vague statement, because it

Re: Allowing extensions to supply operator-/function-specific info

2019-01-28 Thread Simon Riggs
On Sun, 27 Jan 2019 at 19:17, Tom Lane wrote: > > * Allow a normal term to match a functional index, e.g. WHERE x = > > 'abcdefgh' => WHERE substr(x, 1 , 5) = 'abcde' AND x = 'abcdefgh' > > I'm a bit confused about what you think this example means. I do > intend to work on letting extensions d

Re: Allowing extensions to supply operator-/function-specific info

2019-01-28 Thread Tom Lane
Robert Haas writes: > On Sat, Jan 26, 2019 at 12:35 PM Tom Lane wrote: >> Attached is an 0004 that makes a stab at providing some intelligence >> for unnest() and the integer cases of generate_series(). > That looks awesome. > I'm somewhat dubious about whole API. It's basically -- if you have

Re: Allowing extensions to supply operator-/function-specific info

2019-01-28 Thread Robert Haas
On Sat, Jan 26, 2019 at 12:35 PM Tom Lane wrote: > Attached is an 0004 that makes a stab at providing some intelligence > for unnest() and the integer cases of generate_series(). That looks awesome. I'm somewhat dubious about whole API. It's basically -- if you have a problem and a PhD in Postg

Re: Allowing extensions to supply operator-/function-specific info

2019-01-27 Thread Tom Lane
Simon Riggs writes: > On Sun, 20 Jan 2019 at 23:48, Tom Lane wrote: >> What I'm envisioning therefore is that we allow an auxiliary function ... > Does this help with these cases? > * Allow a set returning function to specify number of output rows, in cases > where that is variable and dependen

Re: Allowing extensions to supply operator-/function-specific info

2019-01-27 Thread Simon Riggs
On Sun, 20 Jan 2019 at 23:48, Tom Lane wrote: > What I'm envisioning therefore is that we allow an auxiliary function to > be attached to any operator or function that can provide functionality > like this, and that we set things up so that the set of tasks that > such functions can perform can

Re: Allowing extensions to supply operator-/function-specific info

2019-01-26 Thread Tom Lane
I wrote: > There's a considerable amount of follow-up work that ought to happen > now to make use of these capabilities for places that have been > pain points in the past, such as generate_series() and unnest(). > But I haven't touched that yet. Attached is an 0004 that makes a stab at providing

Allowing extensions to supply operator-/function-specific info

2019-01-20 Thread Tom Lane
Over in the thread at [1], we realized that PostGIS has been thrashing around trying to fake its way to having "special index operators", ie a way to automatically convert WHERE clauses into lossy index quals. That's existed in a non-extensible way inside indxpath.c for twenty years come July. Sin