Hi,
On 07/26/2010 06:33 PM, Robert Haas wrote:
It would be nice to think, for example, that this could be used as
infrastructure for parallel query to stream results back from worker
processes to the backend connected to the user. If you're using 16
processors to concurrently scan 16 partitions of an appendrel and
stream those results back to the master
Now, *that* sounds like music to my ears ;-)
Or put another way: yes, I think imessages and the bgworker
infrastructure stuff could enable or at least help that goal.
Dynamically allocating out of a 2MB
segment gives up most of that flexibility.
Absolutely, that's why I'd like to see other modules that use the
dynamic allocator. The more the better.
What I think will end up happening here is that you'll always have to
size the segment used by the dynamic allocator considerably larger
than the amount of memory you expect to actually be used, so that
performance doesn't go into the toilet when it fills up. As Markus
pointed out upthread, you'll always need some hard limit on the amount
of space that imessages can use, but you can make that limit much
larger if it's not reserved for a single purpose. If you use the
"temporarily allocated shared buffers" method, then you could set the
default limit to something like "64MB, but not more than 1/8th of
shared buffers".
I've been thinking about such rules as well. They quickly get more
complex if you begin to take OOM situations and their counter-measures
into account.
In a way, fixing every separate pool to its specific size just is the
very simples rule-set I can think of. The dynamic allocator buys you
more flexibility, but choosing good limits and rules between the
sub-systems is another issue.
Regards
Markus Wanner
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
