On Tue, May 19, 2009 at 12:32 PM, Simon Riggs <si...@2ndquadrant.com> wrote: > > If we have a query that uses multiple sorts, we may have a top-level > sort, with child nodes that contain sorts also. In some cases we may > find with sub-nodes that have both inner and outer sub-trees that > contain sorts also.
Well a top-level sort and a child sort wouldn't both be accumulating rows at the same time. The child could still be alive behaving like a tuplestore though. > If we allocate large chunks of memory we use malloc(). So complex > queries can have multiple mallocs, followed by multiple reallocs. That > in itself seems likely to end up with roughly double memory use, since > realloc won't work properly/quickly with multiple mallocs. (Double since > we allocate X bytes, then 2X bytes etc until we hit the limit.) I think it's even worse than that since the old and new allocation have to briefly coexist. So at least transiently we use 3x the size of the actual array. > When we later free() the memory, do we always free() it in the reverse > order in which it was allocated? If not, how does that effect reducing > the sbrk point, or other aspects of reusing allocated memory? > > Is it possible that Solaris's default malloc isn't appropriate for > repeated use in complex queries that use multiple sorts? Well anything's possible. Do you have any specific ideas? I would expect any decent malloc library to shrink sbrk based on statically analyzing where its allocations actually are, so I wouldn't expect the pattern of frees to matter on that front. It might still fragment memory if we allocate a bunch of large tuplestore/tuplesorts and then allocate one object in a longer lived memory context. What problems have you seen? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers