On Fri, 2005-09-23 at 11:31 -0400, Tom Lane wrote:
Simon Riggs [EMAIL PROTECTED] writes:
Since we know the predicted size of the sort set prior to starting the
sort node, could we not use that information to allocate memory
appropriately? i.e. if sort size is predicted to be more than twice
PROTECTED]] On Behalf Of Meir Maor
Sent: Friday, September 23, 2005
10:24 PM
To: Simon Riggs
Cc: pgsql-hackers@postgresql.org;
pgsql-performance@postgresql.org
Subject: Re: [HACKERS] Releasing
memory during External sorting?
Calculating Optimal
memory for disk based sort is based only on minimizing
I have concerns about whether we are overallocating memory for use in
external sorts. (All code relating to this is in tuplesort.c)
When we begin a sort we allocate (work_mem | maintenance_work_mem) and
attempt to do the sort in memory. If the sort set is too big to fit in
memory we then write to
Simon Riggs [EMAIL PROTECTED] writes:
If not, I would propose that when we move from qsort to tapesort mode we
free the larger work_mem setting (if one exists) and allocate only a
lower, though still optimal setting for the tapesort. That way the
memory can be freed for use by other users or
On Fri, 2005-09-23 at 10:09 -0400, Tom Lane wrote:
Simon Riggs [EMAIL PROTECTED] writes:
If not, I would propose that when we move from qsort to tapesort mode we
free the larger work_mem setting (if one exists) and allocate only a
lower, though still optimal setting for the tapesort. That
Simon Riggs [EMAIL PROTECTED] writes:
Since we know the predicted size of the sort set prior to starting the
sort node, could we not use that information to allocate memory
appropriately? i.e. if sort size is predicted to be more than twice the
size of work_mem, then just move straight to the
On most platforms it's quite unlikely that any memory would
actually get
released back to the OS before transaction end, because the memory
blocks belonging to the tuplesort context will be intermixed with
blocks
belonging to other contexts. So I think this is pretty pointless.
(If you
On Fri, Sep 23, 2005 at 06:39:35PM +0200, Pailloncy Jean-Gerard wrote:
On most platforms it's quite unlikely that any memory would actually
get released back to the OS before transaction end, because the
memory blocks belonging to the tuplesort context will be intermixed
with blocks belonging
Calculating Optimal memory for disk based sort is based only on minimizing IO.
A previous post stated we can merge as many subfiles as we want in a single pass,
this is not accurate, as we want to eliminate disk seeks also in the merge phase,
also the merging should be done by reading blocks of
