Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-18 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Agreed. Yet I'm thinking that a more coherent approach to optimising the > tuple memory usage in the executor tree might be better than the special > cases we seem to have in various places. I don't know what that is, or > even if its possible though. Yeah

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-18 Thread Simon Riggs
On Wed, 2008-06-18 at 09:45 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > You've not covered the idea that we just alter the execution so we just > > detoast once. > > That's because I already considered and rejected that idea. There's > no very good place to do it. See t

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-18 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > You've not covered the idea that we just alter the execution so we just > detoast once. That's because I already considered and rejected that idea. There's no very good place to do it. See thread on postgis-devel: http://postgis.refractions.net/pipermai

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-18 Thread Simon Riggs
On Mon, 2008-06-16 at 15:35 -0400, Tom Lane wrote: > Recent discussions with the PostGIS hackers led me to think about ways > to reduce overhead when the same TOAST value is repeatedly detoasted. > In the example shown here > http://archives.postgresql.org/pgsql-hackers/2008-06/msg00384.php > 90%

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-17 Thread Greg Stark
> > I definitely think it's worth it, even if it doesn't handle an > > inline-compressed datum. > > Yeah. I'm not certain how much benefit we could get there anyway. > If the datum isn't out-of-line then there's a small upper limit on how > big it can be and hence a small upper limit on how long i

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-17 Thread Tom Lane
Teodor Sigaev <[EMAIL PROTECTED]> writes: >> But we can resolve that by ruling that the required lifetime is the same >> as the value would have had if it'd really been palloc'd --- IOW, until >> the memory context that was current at the time gets deleted or reset. > Many support functions of GiS

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-17 Thread Teodor Sigaev
But we can resolve that by ruling that the required lifetime is the same as the value would have had if it'd really been palloc'd --- IOW, until the memory context that was current at the time gets deleted or reset. Many support functions of GiST/GIN live in very short memory context - only fo

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-17 Thread Tom Lane
Jeff <[EMAIL PROTECTED]> writes: > On Jun 16, 2008, at 3:35 PM, Tom Lane wrote: >> the result of decompressing an inline-compressed datum, because those >> have no unique ID that could be used for a lookup key. This puts a >> bit of a > Wouldn't the tid fit this? or table oid + tid? No. The kil

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-17 Thread Jeff
On Jun 16, 2008, at 3:35 PM, Tom Lane wrote: to a cache entry rather than a freshly palloc'd value. The cache lookup key is the toast table OID plus value OID. Now pg_detoast_datum() has no ... the result of decompressing an inline-compressed datum, because those have no unique ID tha

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-16 Thread Tom Lane
Stephen Frost <[EMAIL PROTECTED]> writes: > * Tom Lane ([EMAIL PROTECTED]) wrote: >> Comments, better ideas? Anyone think this is too much trouble to take >> for the problem? > I definitely think it's worth it, even if it doesn't handle an > inline-compressed datum. Yeah. I'm not certain how mu

Re: [HACKERS] Reducing overhead for repeat de-TOASTing

2008-06-16 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > One unsolved problem is that this scheme doesn't provide any way to cache > the result of decompressing an inline-compressed datum, because those have > no unique ID that could be used for a lookup key. That's pretty unfortunate. > Ideas? Not at the moment