En Mon, 09 Nov 2009 16:05:31 -0300, Eyal Gordon <eyal.gor...@gmail.com>
escribió:
background:
we are using python 2.4.3 on CentOS 5.3 with many threads - and our
shell's
default stack size limit is set to 10240KB (i.e. ~10MB).
we noticed that python's Threading module appears to create threads with
this value as their stack size (we ran a sample program that creates 10
threads and measured its virtual memory size, then reduced the stack size
limit of the shell to 5120KB - and saw that the program's virtual memory
size was reduced by ~50MBs).
the problem:
our program uses numerous threads, and thus the virtual memory size gets
to
be very large. we would like to reduce the size of the stack to reduce
this
size. we were looking for information about recommendation for the stack
size to use, but found none.
You can set the thread stack size (for threads that are going to be
created, not existing threads) using threading.stack_size(SIZE)
http://docs.python.org/library/threading.html#threading.stack_size
questions:
1. is there some rule-of-thumb for the recommended stack size for python
programs of various sorts?
No idea. I've been always happy with the default settings.
2. is there a way for us, at runtime (from inside the code or outside the
process), to find how much of a thread's stack we are using (in KB or
some
other size units)?
see top(1)
3. when we define local objects - do the objects themselves get
allocated on
the stack - or are they allocated on the heap and only references to them
are kept on the stack?
They're allocated on the heap, and most references to objects are on the
heap too. The Python stack of execution frames is allocated on the heap
too. Basically, the OS stack is used for local variables in C code only.
4. would the size of the stacks (which are probably not really allocated
by
the linux virtual memory sub-system, unless used) have a noticeable
performance effect on a python program? same question regarding the use
of a
large number of threads?
I think it doesn't matter, unless you create so many threads as to exhaust
the available addressing space (in 32bits, 4GB address space and 10MB per
thread means 400 threads maximum).
--
Gabriel Genellina
--
http://mail.python.org/mailman/listinfo/python-list
