Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-18 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Agreed. Yet I'm thinking that a more coherent approach to optimising the > tuple memory usage in the executor tree might be better than the special > cases we seem to have in various places. I don't know what that is, or > even if its possible though. Yeah

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-18 Thread Simon Riggs
On Wed, 2008-06-18 at 09:45 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > You've not covered the idea that we just alter the execution so we just > > detoast once. > > That's because I already considered and rejected that idea. There's > no very good place to do it. See t

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-18 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > You've not covered the idea that we just alter the execution so we just > detoast once. That's because I already considered and rejected that idea. There's no very good place to do it. See thread on postgis-devel: http://postgis.refractions.net/pipermai

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-18 Thread Simon Riggs
On Mon, 2008-06-16 at 15:35 -0400, Tom Lane wrote: > Recent discussions with the PostGIS hackers led me to think about ways > to reduce overhead when the same TOAST value is repeatedly detoasted. > In the example shown here > http://archives.postgresql.org/pgsql-hackers/2008-06/msg00384.php > 90%

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-17 Thread Greg Stark
> > I definitely think it's worth it, even if it doesn't handle an > > inline-compressed datum. > > Yeah. I'm not certain how much benefit we could get there anyway. > If the datum isn't out-of-line then there's a small upper limit on how > big it can be and hence a small upper limit on how long i

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-17 Thread Tom Lane
Teodor Sigaev <[EMAIL PROTECTED]> writes: >> But we can resolve that by ruling that the required lifetime is the same >> as the value would have had if it'd really been palloc'd --- IOW, until >> the memory context that was current at the time gets deleted or reset. > Many support functions of GiS

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-17 Thread Teodor Sigaev
But we can resolve that by ruling that the required lifetime is the same as the value would have had if it'd really been palloc'd --- IOW, until the memory context that was current at the time gets deleted or reset. Many support functions of GiST/GIN live in very short memory context - only fo

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-17 Thread Tom Lane
Jeff <[EMAIL PROTECTED]> writes: > On Jun 16, 2008, at 3:35 PM, Tom Lane wrote: >> the result of decompressing an inline-compressed datum, because those >> have no unique ID that could be used for a lookup key. This puts a >> bit of a > Wouldn't the tid fit this? or table oid + tid? No. The kil

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-17 Thread Jeff
On Jun 16, 2008, at 3:35 PM, Tom Lane wrote: to a cache entry rather than a freshly palloc'd value. The cache lookup key is the toast table OID plus value OID. Now pg_detoast_datum() has no ... the result of decompressing an inline-compressed datum, because those have no unique ID tha

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-16 Thread Tom Lane
Stephen Frost <[EMAIL PROTECTED]> writes: > * Tom Lane ([EMAIL PROTECTED]) wrote: >> Comments, better ideas? Anyone think this is too much trouble to take >> for the problem? > I definitely think it's worth it, even if it doesn't handle an > inline-compressed datum. Yeah. I'm not certain how mu

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-16 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > One unsolved problem is that this scheme doesn't provide any way to cache > the result of decompressing an inline-compressed datum, because those have > no unique ID that could be used for a lookup key. That's pretty unfortunate. > Ideas? Not at the moment

[HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-16 Thread Tom Lane
Recent discussions with the PostGIS hackers led me to think about ways to reduce overhead when the same TOAST value is repeatedly detoasted. In the example shown here http://archives.postgresql.org/pgsql-hackers/2008-06/msg00384.php 90% of the runtime is being consumed by repeated detoastings of a