Re: RFC: Extension Packaging & Lookup

2024-10-29 Thread Paul Ramsey
> On Oct 29, 2024, at 10:55 AM, David E. Wheeler wrote: > >>> Relative rpaths as I have seen them are relative to the executable or >>> library in which they are defined (as far as I know, I’m not a dylib expert >>> by any stretch). The implication is that extension.so >>>

Re: RFC: Extension Packaging & Lookup

2024-10-29 Thread Paul Ramsey
> On Oct 29, 2024, at 10:09 AM, David E. Wheeler wrote: > > On Oct 29, 2024, at 12:23, Paul Ramsey wrote: > >> Thanks for this, David, > > 🤘🏻 > >> This of course is the area that worries the heck out of me, as someone with >> extensions that inclu

Re: RFC: Extension Packaging & Lookup

2024-10-29 Thread Paul Ramsey
Thanks for this, David, > On Oct 28, 2024, at 3:19 PM, David E. Wheeler wrote: > > ## Challenge: Third Party Dependencies This of course is the area that worries the heck out of me, as someone with extensions that includes not just single system dependencies but long chains of them (depending

Pg17 Crash in Planning (Arrays + Casting + UDF)

2024-10-09 Thread Paul Ramsey
Hackers, This extremely odd case [2] came in via a report using a lot of PostGIS functions, but it can be reconfigured into a pure-PostgreSQL crasher [1]. CREATE TABLE n (i integer); CREATE OR REPLACE FUNCTION add(integer) RETURNS integer AS 'SELECT 2 * $1 + 4 * $1' LANGUAGE 'sql' I

Re: daitch_mokotoff module

2023-02-07 Thread Paul Ramsey
> On Feb 7, 2023, at 6:47 AM, Dag Lem wrote: > > I just went by to check the status of the patch, and I noticed that > you've added yourself as reviewer earlier - great! > > Please tell me if there is anything I can do to help bring this across > the finish line. Honestly, I had set it to Re

Re: daitch_mokotoff module

2023-01-12 Thread Paul Ramsey
> On Jan 12, 2023, at 7:30 AM, Dag Lem wrote: > > Paul Ramsey writes: > >> On Mon, Jan 2, 2023 at 2:03 PM Dag Lem wrote: >> >>> I also improved on the documentation example (using Full Text Search). >>> AFAIK you can't make general queries li

Re: daitch_mokotoff module

2023-01-11 Thread Paul Ramsey
On Mon, Jan 2, 2023 at 2:03 PM Dag Lem wrote: > I also improved on the documentation example (using Full Text Search). > AFAIK you can't make general queries like that using arrays, however in > any case I must admit that text arrays seem like more natural building > blocks than space delimited t

Re: [PATCH] random_normal function

2023-01-10 Thread Paul Ramsey
On Tue, Jan 10, 2023 at 6:34 AM Tom Lane wrote: > > Dean Rasheed writes: > > I double-checked the one-in-a-billion claim, and it looks about right > > for each test. > > Thanks for double-checking my arithmetic. > > > The rest looks good to me, except there's a random non-ASCII character > > inst

Re: [PATCH] random_normal function

2022-12-15 Thread Paul Ramsey
> On Dec 14, 2022, at 9:17 PM, Michael Paquier wrote: > > On Tue, Dec 13, 2022 at 03:51:11PM -0800, Paul Ramsey wrote: >> Clearing up one CI failure. > > +-- normal values converge on stddev == 2.0 > +SELECT round(stddev(random_normal(2, 2))) > + FROM generate_seri

Re: [PATCH] random_normal function

2022-12-13 Thread Paul Ramsey
> On Dec 9, 2022, at 3:20 PM, Paul Ramsey wrote: > > > >> On Dec 9, 2022, at 9:17 AM, Paul Ramsey wrote: >> >> >>> On Dec 8, 2022, at 8:29 PM, Michael Paquier wrote: >>> >>> On Thu, Dec 08, 2022 at 04:44:56PM -0800, Pa

Re: [PATCH] random_normal function

2022-12-09 Thread Paul Ramsey
> On Dec 9, 2022, at 9:17 AM, Paul Ramsey wrote: > > >> On Dec 8, 2022, at 8:29 PM, Michael Paquier wrote: >> >> On Thu, Dec 08, 2022 at 04:44:56PM -0800, Paul Ramsey wrote: >>> Final tme, with fixes from cirrusci. >> >> Well, why not.

Re: [PATCH] random_normal function

2022-12-09 Thread Paul Ramsey
> On Dec 9, 2022, at 11:10 AM, Paul Ramsey wrote: > > > >> On Dec 9, 2022, at 11:01 AM, Joe Conway wrote: >> >> On 12/9/22 13:51, Paul Ramsey wrote: >>>> On Dec 9, 2022, at 10:39 AM, Mark Dilger >>>> wrote: >>>>>

Re: [PATCH] random_normal function

2022-12-09 Thread Paul Ramsey
> On Dec 9, 2022, at 11:01 AM, Joe Conway wrote: > > On 12/9/22 13:51, Paul Ramsey wrote: >>> On Dec 9, 2022, at 10:39 AM, Mark Dilger >>> wrote: >>>> On Dec 8, 2022, at 1:53 PM, Paul Ramsey wrote: >>>> Just a utility function to generat

Re: [PATCH] random_normal function

2022-12-09 Thread Paul Ramsey
> On Dec 9, 2022, at 10:39 AM, Mark Dilger wrote: > >> On Dec 8, 2022, at 1:53 PM, Paul Ramsey wrote: >> >> Just a utility function to generate random numbers from a normal >> distribution. I find myself doing this several times a year, and I am >

Re: [PATCH] random_normal function

2022-12-09 Thread Paul Ramsey
> On Dec 8, 2022, at 8:29 PM, Michael Paquier wrote: > > On Thu, Dec 08, 2022 at 04:44:56PM -0800, Paul Ramsey wrote: >> Final tme, with fixes from cirrusci. > > Well, why not. Seems like you would use that a lot with PostGIS. > > #include /*

Re: [PATCH] random_normal function

2022-12-08 Thread Paul Ramsey
> On Dec 8, 2022, at 3:25 PM, Paul Ramsey wrote: > >> >> Revised patch attached. > > And again, because I think I missed one change in the last one. > > Final tme, with fixes from cirrusci. random_normal_04.patch Description: Binary data

Re: [PATCH] random_normal function

2022-12-08 Thread Paul Ramsey
> > Revised patch attached. And again, because I think I missed one change in the last one. random_normal_03.patch Description: Binary data

Re: [PATCH] random_normal function

2022-12-08 Thread Paul Ramsey
> On Dec 8, 2022, at 2:46 PM, Justin Pryzby wrote: > > I guess make_interval is a typo ? > > This is causing it to fail tests: > http://cfbot.cputube.org/paul-ramsey.html > Yep, dumb typo, thanks! This bot is amazeballs, thank you! P.

Re: [PATCH] random_normal function

2022-12-08 Thread Paul Ramsey
> On Dec 8, 2022, at 2:40 PM, David G. Johnston > wrote: > > On Thu, Dec 8, 2022 at 2:53 PM Paul Ramsey wrote: > > random_normal(stddev float8 DEFAULT 1.0, mean float8 DEFAULT 0.0) > > Any particular justification for placing stddev before mean? A brief survey &

[PATCH] random_normal function

2022-12-08 Thread Paul Ramsey
Just a utility function to generate random numbers from a normal distribution. I find myself doing this several times a year, and I am sure I must not be the only one. random_normal(stddev float8 DEFAULT 1.0, mean float8 DEFAULT 0.0) random_normal_01.patch Description: Binary data

Re: Lambda expressions (was Re: BUG #15471)

2022-10-06 Thread Paul Ramsey
On Tue, Oct 30, 2018 at 3:20 PM Andres Freund wrote: > > Hi, > > On 2018-10-30 16:54:45 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2018-10-30 16:23:37 -0400, Tom Lane wrote: > > >> Well, a Lambda expression is not something that can be optimized away > > >> (unless perhaps you can

Re: Datum values consistency within one query

2020-04-03 Thread Paul Ramsey
> On Apr 2, 2020, at 4:30 PM, Tom Lane wrote: > > Paul Ramsey writes: >> Getting the datum value is really fast, so I can have a cache that >> keeps the latest detoasted object around, and update it when the datum >> changes, and store the cache information in the

Datum values consistency within one query

2020-04-02 Thread Paul Ramsey
Imagine a function that was going to take a table of, say, images, and so something to them over and over, like: SELECT pixel_count(img), clr_name, img_name FROM images img CROSS JOIN colors clr When you run this function, you find that a great amount of time is being spend in the decompres

Re: [postgis-devel] About EXTENSION from UNPACKAGED on PostgreSQL 13

2020-02-26 Thread Paul Ramsey
OK, well, what PostGIS needs is the ability for 'ALTER EXTENSION …. UPDATE foo’ to end up with two extensions in the end, ‘foo’ and ‘foo_new’. That’s what’s happening in the 2.x -> 3 upgrade process, as ‘postgis’ becomes ‘postgis’ and ‘postgis_raster’. Presumably 15 years out from the 1.x -> 2

Re: [proposal] de-TOAST'ing using a iterator

2019-09-25 Thread Paul Ramsey
> On Sep 23, 2019, at 9:45 AM, Alvaro Herrera wrote: > > Paul Ramsey, do you have opinions to share about this patch? I think > PostGIS might benefit from it. Thread starts here: I like the idea a great deal, but actually PostGIS is probably neutral on it: we generally want

Re: Optimize partial TOAST decompression

2019-07-02 Thread Paul Ramsey
On Mon, Jul 1, 2019 at 6:46 AM Binguo Bao wrote: > > Andrey Borodin 于2019年6月29日周六 下午9:48写道: >> I've took a look into the code. >> I think we should extract function for computation of max_compressed_size >> and put it somewhere along with pglz code. Just in case something will >> change somethi

Re: Compressed TOAST Slicing

2019-04-09 Thread Paul Ramsey
> On Apr 9, 2019, at 10:09 AM, Andrey Borodin wrote: > > He advised me to use algorithm that splits copied regions into smaller > non-overlapping subregions with exponentially increasing size. > > while (off <= len) > { >memcpy(dp, dp - off, off); >len -= off; >dp += off; >off

Re: Compressed TOAST Slicing

2019-03-19 Thread Paul Ramsey
> On Mar 19, 2019, at 4:47 AM, Stephen Frost wrote: > > Greetings, > > * Paul Ramsey (pram...@cleverelephant.ca) wrote: >>> On Mar 18, 2019, at 7:34 AM, Robert Haas wrote: >>> +1. I think Paul had it right originally. >> >> In that spirit,

Re: Compressed TOAST Slicing

2019-03-18 Thread Paul Ramsey
> On Mar 18, 2019, at 7:34 AM, Robert Haas wrote: > > On Mon, Mar 18, 2019 at 10:14 AM Tom Lane wrote: >> Stephen Frost writes: >>> * Andres Freund (and...@anarazel.de) wrote: I don't think that should stop us from breaking the API. You've got to do quite low level stuff to need pgl

Re: Compressed TOAST Slicing

2019-03-13 Thread Paul Ramsey
> On Mar 13, 2019, at 9:32 AM, Andrey Borodin wrote: > > > >> 13 марта 2019 г., в 21:05, Paul Ramsey >> написал(а): >> >> Here is a new (final?) patch ... >> >> > > This check > > @@ -744,6 +748,8 @@ p

Re: Compressed TOAST Slicing

2019-03-13 Thread Paul Ramsey
On Mar 13, 2019, at 8:25 AM, Paul Ramsey <pram...@cleverelephant.ca> wrote:On Mar 13, 2019, at 3:09 AM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote:On 3/13/19 3:19 AM, Michael Paquier wrote:On Tue, Mar 12, 2019 at 07:01:17PM -0700, Andres Freund wrote:I don't think this

Re: Compressed TOAST Slicing

2019-03-13 Thread Paul Ramsey
> On Mar 13, 2019, at 3:09 AM, Tomas Vondra > wrote: > > On 3/13/19 3:19 AM, Michael Paquier wrote: >> On Tue, Mar 12, 2019 at 07:01:17PM -0700, Andres Freund wrote: >>> I don't think this is even close to popular enough to incur the >>> maybe of a separate function / more complicated interfa

Re: Compressed TOAST Slicing

2019-03-12 Thread Paul Ramsey
> On Mar 11, 2019, at 10:22 PM, Andrey Borodin wrote: > > Hi! > >> 21 февр. 2019 г., в 23:50, Paul Ramsey >> написал(а): >> >> Merci! Attached are updated patches. >> > > As noted before, patches are extremely useful. > So, I'v

Re: Compressed TOAST Slicing

2019-03-12 Thread Paul Ramsey
> On Mar 12, 2019, at 9:45 AM, Paul Ramsey wrote: > > > >> On Mar 12, 2019, at 9:13 AM, Andres Freund wrote: >> >> On 2019-03-12 14:42:14 +0900, Michael Paquier wrote: >>> On Mon, Mar 11, 2019 at 08:38:56PM +, Regina Obe wrote: >>>>

Re: Compressed TOAST Slicing

2019-03-12 Thread Paul Ramsey
> On Mar 12, 2019, at 9:13 AM, Andres Freund wrote: > > On 2019-03-12 14:42:14 +0900, Michael Paquier wrote: >> On Mon, Mar 11, 2019 at 08:38:56PM +, Regina Obe wrote: >>> I tested on windows mingw64 (as of a week ago) and confirmed the >>> patch applies cleanly and significantly faster fo

Re: Compressed TOAST Slicing

2019-03-12 Thread Paul Ramsey
> On Mar 11, 2019, at 10:42 PM, Michael Paquier wrote: > > On Mon, Mar 11, 2019 at 08:38:56PM +, Regina Obe wrote: >> I tested on windows mingw64 (as of a week ago) and confirmed the >> patch applies cleanly and significantly faster for left, substr >> tests than head. > > int32 > pglz_d

Re: Allowing extensions to supply operator-/function-specific info

2019-03-05 Thread Paul Ramsey
> On Mar 5, 2019, at 3:56 PM, Tom Lane wrote: > > Paul Ramsey writes: >> On Mar 5, 2019, at 3:26 PM, Tom Lane wrote: >>> Hm, I think your addition of this bit is wrong: >>> >>> +/* >>> +* A

Re: Allowing extensions to supply operator-/function-specific info

2019-03-05 Thread Paul Ramsey
> On Mar 5, 2019, at 3:26 PM, Tom Lane wrote: > > Paul Ramsey writes: >> Thanks for the patch, I’ve applied and smoothed and taken your advice on >> schema-qualified lookups as well. > > Hm, I think your ad

Re: Allowing extensions to supply operator-/function-specific info

2019-03-05 Thread Paul Ramsey
> On Mar 4, 2019, at 4:22 PM, Tom Lane wrote: > > Paul Ramsey writes: >>> On Mar 4, 2019, at 2:52 PM, Tom Lane wrote: >>> BTW, if you'd like me to review the code you added for this, I'd be happy >>> to do so. I've never looked at PostGIS

Re: Allowing extensions to supply operator-/function-specific info

2019-03-04 Thread Paul Ramsey
> On Mar 4, 2019, at 2:52 PM, Tom Lane wrote: > > Paul Ramsey writes: >> Gotcha, done and now have an implementation that passes all our regression >> tests. > > Very cool! So the next step, I guess, is to address your original problem > by cranking u

Re: Allowing extensions to supply operator-/function-specific info

2019-03-04 Thread Paul Ramsey
> On Mar 4, 2019, at 1:13 PM, Tom Lane wrote: > > Paul Ramsey writes: >> I had what seemed to be working code except for a couple rare cases, >> but when I fixed those cases it turned out that I had a major problem: >> building a OP expression works fine, but bu

Re: Allowing extensions to supply operator-/function-specific info

2019-03-04 Thread Paul Ramsey
So I am getting much closer to a working implementation in PostGIS, but have just run into an issue which I am assuming is my misunderstanding something... https://github.com/pramsey/postgis/blob/92268c94f3aa1fc63a2941f2b451be15b28662cf/postgis/gserialized_supportfn.c#L287 I had what seemed to be

Re: Allowing extensions to supply operator-/function-specific info

2019-02-27 Thread Paul Ramsey
> On Feb 27, 2019, at 3:40 PM, Tom Lane wrote: > >> Variable SupportRequestCost is very exciting, but given that variable cost >> is usually driven by the complexity of arguments, what kind of argument is >> the SupportRequestCost call fed during the planning stage? Constant >> arguments are

Re: Allowing extensions to supply operator-/function-specific info

2019-02-27 Thread Paul Ramsey
A few more questions… The documentation says that a support function should have a signature "supportfn(internal) returns internal”, but doesn’t say which (if any) annotations should be provided. IMMUTABLE? PARALLEL SAFE? STRICT? None? All? Variable SupportRequestCost is very exciting, but give

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Paul Ramsey
> On Feb 26, 2019, at 2:19 PM, Tom Lane wrote: > > In most cases, multiple matching arguments are going to lead to > failure to construct any useful index condition, because your > comparison value has to be a pseudoconstant (ie, not a variable > from the same table, so in both of the above exa

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Paul Ramsey
> On Feb 26, 2019, at 2:19 PM, Tom Lane wrote: > >> I have >> created a table (foo) a geometry column (g) and an index (GIST on >> foo(g)) and am running a query against foo using a noop function with >> a support function bound to it. > >> The support function is called, twice, once in >> T_S

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Paul Ramsey
On Mon, Feb 25, 2019 at 4:09 PM Tom Lane wrote: > > Paul Ramsey writes: > > On Mon, Feb 25, 2019 at 3:01 PM Tom Lane wrote: > >> It's whichever one the index column's opclass belongs to. Basically what > >> you're trying to do here is verify whether

Re: Allowing extensions to supply operator-/function-specific info

2019-02-25 Thread Paul Ramsey
On Mon, Feb 25, 2019 at 3:01 PM Tom Lane wrote: > > Looking at the examples, they are making use of the opfamily that > > comes in SupportRequestIndexCondition.opfamily. > > That opfamily Oid is the first one in the IndexOptInfo.opfamily array. > > Here's where my thread of understanding fails to

Re: Allowing extensions to supply operator-/function-specific info

2019-02-25 Thread Paul Ramsey
On Mon, Jan 28, 2019 at 9:51 AM Tom Lane wrote: > is people like PostGIS, who already cleared that bar. I hope that > we'll soon have a bunch of examples, like those in the 0004 patch, > that people can look at to see how to do things in this area. I see > no reason to believe it'll be all that

Re: Compressed TOAST Slicing

2019-02-21 Thread Paul Ramsey
On Wed, Feb 20, 2019 at 1:12 PM Stephen Frost wrote: > > * Paul Ramsey (pram...@cleverelephant.ca) wrote: > > On Wed, Feb 20, 2019 at 10:50 AM Daniel Verite > > wrote: > > > > > > What about starts_with(string, prefix)? > > Thanks, I'll add that. &

Re: Compressed TOAST Slicing

2019-02-20 Thread Paul Ramsey
On Wed, Feb 20, 2019 at 10:50 AM Daniel Verite wrote: > > Paul Ramsey wrote: > > > Oddly enough, I couldn't find many/any things that were sensitive to > > left-end decompression. The only exception is "LIKE this%" which > > clearly would be hel

Re: Compressed TOAST Slicing

2019-02-20 Thread Paul Ramsey
> On Feb 20, 2019, at 10:37 AM, Simon Riggs wrote: > > -1, I think this is blowing up the complexity of a already useful patch, > even though there's no increase in complexity due to the patch proposed > here. I totally get wanting incremental decompression for jsonb, but I > don't see why Paul

Re: Compressed TOAST Slicing

2019-02-19 Thread Paul Ramsey
On Sat, Feb 16, 2019 at 7:25 AM Simon Riggs wrote: > Could we get an similarly optimized implementation of -> operator for JSONB > as well? > Are there any other potential uses? Best to fix em all up at once and then > move on to other things. Thanks. Oddly enough, I couldn't find many/any thi

Re: Changing SQL Inlining Behaviour (or...?)

2019-02-05 Thread Paul Ramsey
> On Feb 5, 2019, at 11:07 AM, Tom Lane wrote: > > Paul Ramsey writes: >> Hokay… I’ve read through the patch set, applied it and built it, all works. >> Am starting to try a test implementation in PostGIS land. Robert’s comment >> about “PostgreSQL magic”

Re: Changing SQL Inlining Behaviour (or...?)

2019-02-05 Thread Paul Ramsey
> On Feb 3, 2019, at 3:47 PM, Tom Lane wrote: > > I wrote: >> I've posted some preliminary design ideas at >> https://www.postgresql.org/message-id/15193.1548028...@sss.pgh.pa.us >> and >> https://www.postgresql.org/message-id/15289.1548028...@sss.pgh.pa.us >> While there's a nontrivial amount

Re: Changing SQL Inlining Behaviour (or...?)

2019-01-21 Thread Paul Ramsey
> On Jan 21, 2019, at 3:27 PM, Andres Freund wrote: > > Hi, > > On 2019-01-21 15:21:29 -0800, Paul Ramsey wrote: >> As a practical matter, most of the exact-test functions have a >> preamble that checks the bbox, so in the seqscan case having the >> oper

Re: Changing SQL Inlining Behaviour (or...?)

2019-01-21 Thread Paul Ramsey
> On Jan 21, 2019, at 3:17 PM, Andres Freund wrote: > > Hi, > > On 2019-01-19 20:53:55 -0500, Tom Lane wrote: >> I wrote: >>> Paul Ramsey writes: >>>> Is there a rule of thumb we can use in costing our wrappers to ensure that >>>> th

Re: Changing SQL Inlining Behaviour (or...?)

2019-01-19 Thread Paul Ramsey
> On Jan 19, 2019, at 5:53 PM, Tom Lane wrote: > > I wrote: >> Paul Ramsey writes: >>> Is there a rule of thumb we can use in costing our wrappers to ensure that >>> they always inline? > >> Not really, which is a weak spot of this whole approach

Re: Changing SQL Inlining Behaviour (or...?)

2019-01-19 Thread Paul Ramsey
> On Jan 19, 2019, at 12:25 PM, Tom Lane wrote: > > Adam Brightwell writes: >> I've been working with Paul to create and test a patch (attached) that >> addresses Solution #2. This patch essentially modifies the inlining >> logic to compare the cost of the function with the total cost of the

Re: Changing SQL Inlining Behaviour (or...?)

2019-01-03 Thread Paul Ramsey
On Mon, Dec 31, 2018 at 2:23 PM Adam Brightwell < adam.brightw...@crunchydata.com> wrote: > > > * Solution #2 - Quick and dirty and invisible. Tom suggested a hack that > achieves the aims of #1 but without adding syntax to CREATE FUNCTION: have > the inlining logic look at the cost of the wrapper

Re: Dead code in toast_fetch_datum_slice?

2018-12-10 Thread Paul Ramsey
> On Dec 10, 2018, at 10:30 AM, Stephen Frost wrote: > > Greetings, > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: >> On 2018-Dec-10, Paul Ramsey wrote: >>> Your analysis looks correct to me, I'm pretty sure I had the same reaction, >>> fi

Re: Dead code in toast_fetch_datum_slice?

2018-12-10 Thread Paul Ramsey
On Fri, Dec 7, 2018 at 3:25 PM Stephen Frost wrote: > > > Perhaps I'm missing something, but in toast_fetch_datum_slice() there's: > > Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer)); > > Further, the only caller of this function today is > heap_tuple_untoast_attr_slice(), which does

Re: Compressed TOAST Slicing

2018-12-06 Thread Paul Ramsey
PG indentation is not followed here for n. I have attached updated patches that add the comment and adhere to the Pg variable declaration indentation styles, ATB! P -- Paul Ramsey http://crunchydata.com compressed-datum-slicing-left-20190103a.patch Description: Binary data compresse

Re: Convert MAX_SAOP_ARRAY_SIZE to new guc

2018-11-15 Thread Paul Ramsey
On Fri, Nov 9, 2018 at 1:32 PM James Coleman wrote: > Summary: > Create new guc array_optimization_size_limit and use it to replace > MAX_SAOP_ARRAY_SIZE in predtest.c. > > Status: > The attached patch applies cleanly to master, builds without error, > and passes tests locally. > Confirmed that

Re: Changing SQL Inlining Behaviour (or...?)

2018-11-13 Thread Paul Ramsey
On Fri, Nov 9, 2018 at 11:14 AM Paul Ramsey wrote: > > Neglected to include the footnotes... > > As I promised at PgConf.eu, here's a follow-up email about PostGIS > parallelism and function inlining (an odd combination, it is true). > > So, context: > > - We want

Re: Changing SQL Inlining Behaviour (or...?)

2018-11-09 Thread Paul Ramsey
> Neglected to include the footnotes... As I promised at PgConf.eu, here's a follow-up email about PostGIS parallelism and function inlining (an odd combination, it is true). So, context: - We want PostGIS to parallelize more. In order to achieve that we need to mark our functions with more real

Changing SQL Inlining Behaviour (or...?)

2018-11-09 Thread Paul Ramsey
As I promised at PgConf.eu, here's a follow-up email about PostGIS parallelism and function inlining (an odd combination, it is true). So, context: - We want PostGIS to parallelize more. In order to achieve that we need to mark our functions with more realistic COSTs. Much much higher COSTs. - Wh

Re: Compressed TOAST Slicing

2018-11-02 Thread Paul Ramsey
As threatened, I have also added a patch to left() to also use sliced access. compressed-datum-slicing-20190102a.patch Description: Binary data compressed-datum-slicing-left-20190102a.patch Description: Binary data

Re: Compressed TOAST Slicing

2018-11-02 Thread Paul Ramsey
On Thu, Nov 1, 2018 at 4:02 PM Tom Lane wrote: > Paul Ramsey writes: > > On Thu, Nov 1, 2018 at 2:29 PM Stephen Frost wrote: > >> and secondly, why we wouldn't consider > >> handling a non-zero offset. A non-zero offset would, of course, still > >>

Re: Compressed TOAST Slicing

2018-11-01 Thread Paul Ramsey
On Thu, Nov 1, 2018 at 2:29 PM Stephen Frost wrote: > Greetings, > > * Paul Ramsey (pram...@cleverelephant.ca) wrote: > > The attached patch adds in a code path to do a partial decompression of > the > > TOAST entry, when the requested slice is at the start of the obje

Compressed TOAST Slicing

2018-11-01 Thread Paul Ramsey
Currently, PG_DETOAST_DATUM_SLICE when run on a compressed TOAST entry will first decompress the whole object, then extract the relevant slice. When the desired slice is at or near the front of the object, this is obviously non-optimal. The attached patch adds in a code path to do a partial decom

Re: Parallel threads in query

2018-11-01 Thread Paul Ramsey
On Wed, Oct 31, 2018 at 2:11 PM Tom Lane wrote: > =?UTF-8?Q?Darafei_=22Kom=D1=8Fpa=22_Praliaskouski?= > writes: > > Question is, what's the best policy to allocate cores so we can play nice > > with rest of postgres? > > There is not, because we do not use or support multiple threads inside >

Re: MemoryContextCreate change in PG 11 how should contexts be created

2017-12-19 Thread Paul Ramsey
On Tue, Dec 19, 2017 at 7:00 AM, Tom Lane wrote: > Paul Ramsey writes: >> Our use of MemoryContextCreate is solely in order to get use >> MemoryContextDelete as a callback so that, at the end of a statement, >> we can clean up externally allocated memory that we're ho

Re: MemoryContextCreate change in PG 11 how should contexts be created

2017-12-19 Thread Paul Ramsey
On Tue, Dec 19, 2017 at 6:54 AM, Alvaro Herrera wrote: > Paul Ramsey wrote: > >> Our use of MemoryContextCreate is solely in order to get use >> MemoryContextDelete as a callback so that, at the end of a statement, >> we can clean up externally allocated memory that we&

Re: MemoryContextCreate change in PG 11 how should contexts be created

2017-12-19 Thread Paul Ramsey
On Tue, Dec 19, 2017 at 12:24 AM, Regina Obe wrote: > On December 13th this change to context creation was committed, which broke > PostGIS trunk compile against PostgreSQL 11 head. > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=9fa6f00b1308d > d10da4eca2f31ccbfc7b35bb461 > > Tic

Re: Inlining functions with "expensive" parameters

2017-11-17 Thread Paul Ramsey
On Thu, Nov 16, 2017 at 12:37 PM, Andres Freund wrote: > Hi, > > On 2017-11-16 15:27:59 -0500, Tom Lane wrote: > > Andres Freund writes: > > > On November 16, 2017 11:44:52 AM PST, Tom Lane > wrote: > > >> Yeah, there's no mechanism like that now, but there could be. > > > > > Right, but it doe

Re: Inlining functions with "expensive" parameters

2017-11-16 Thread Paul Ramsey
On Thu, Nov 9, 2017 at 12:11 PM, Paul Ramsey wrote: > All, > > As we try and make PostGIS more "parallel sensitive" we have been added > costs to our functions, so that their relative CPU cost is more accurately > reflected in parallel plans. > > This has resulted