On Fri, 2005-09-23 at 11:31 -0400, Tom Lane wrote:
> Simon Riggs <[EMAIL PROTECTED]> writes:
> > Since we know the predicted size of the sort set prior to starting the
> > sort node, could we not use that information to allocate memory
> > appropriately? i.e. if sort size is predicted to be more th
PROTECTED]] On Behalf Of Meir Maor
Sent: Friday, September 23, 2005
10:24 PM
To: Simon Riggs
Cc: pgsql-hackers@postgresql.org;
pgsql-performance@postgresql.org
Subject: Re: [HACKERS] Releasing
memory during External sorting?
Calculating Optimal
memory for disk based sort is based only on
Calculating Optimal memory for disk based sort is based only on minimizing IO.
A previous post stated we can merge as many subfiles as we want in a single pass,
this is not accurate, as we want to eliminate disk seeks also in the merge phase,
also the merging should be done by reading blocks of dat
On Fri, Sep 23, 2005 at 06:39:35PM +0200, Pailloncy Jean-Gerard wrote:
> >On most platforms it's quite unlikely that any memory would actually
> >get released back to the OS before transaction end, because the
> >memory blocks belonging to the tuplesort context will be intermixed
> >with blocks bel
On most platforms it's quite unlikely that any memory would
actually get
released back to the OS before transaction end, because the memory
blocks belonging to the tuplesort context will be intermixed with
blocks
belonging to other contexts. So I think this is pretty pointless.
(If you can't
Simon Riggs <[EMAIL PROTECTED]> writes:
> Since we know the predicted size of the sort set prior to starting the
> sort node, could we not use that information to allocate memory
> appropriately? i.e. if sort size is predicted to be more than twice the
> size of work_mem, then just move straight to
On Fri, 2005-09-23 at 10:09 -0400, Tom Lane wrote:
> Simon Riggs <[EMAIL PROTECTED]> writes:
> > If not, I would propose that when we move from qsort to tapesort mode we
> > free the larger work_mem setting (if one exists) and allocate only a
> > lower, though still optimal setting for the tapesort
Simon Riggs <[EMAIL PROTECTED]> writes:
> If not, I would propose that when we move from qsort to tapesort mode we
> free the larger work_mem setting (if one exists) and allocate only a
> lower, though still optimal setting for the tapesort. That way the
> memory can be freed for use by other users
I have concerns about whether we are overallocating memory for use in
external sorts. (All code relating to this is in tuplesort.c)
When we begin a sort we allocate (work_mem | maintenance_work_mem) and
attempt to do the sort in memory. If the sort set is too big to fit in
memory we then write to
