On 2016-09-21 16:15, Jonathan Bober wrote:
(I've swtiched from sage-support to to sage-devel.)
I can test and review. I think I know what to do and I would have just
tried to implement this myself, except that it would take me a while to
figure out how to make the change in such a way that it fits into the
Sage build process.
OK, I will do it then.
- PARI uses it's own internal stack for quick memory allocation
- At the "top level", if I'm using PARI as a C-library, or through the
GP interpreter, the stack is empty between function calls, so it can be
resized.
- While in use, though, the stack can't really be resized, because
references to memory allocated on the stack don't use the stack pointer,
and resizing might require moving the stack
- If the stack runs out of space, PARI throws some error, and once upon
a time Sage would notice this error, increase the stack size, and repeat
- Now instead Sage just sets the stack size to be really big, which
probably is generally fine because modern systems generally don't
allocation memory until it is actually touched
(Is that all correct?)
It is more complicated than that... PARI has two stack sizes: a real
stack size and a virtual stack size. In Sage, the real stack size is
relatively small (a few megabytes) while the virtual stack size can be
huge. The virtual stack size is what is mmap()ed, so it counts towards
virtual memory.
If possible, PARI uses only the real stack size. It has a garbage
collection mechanism to clean up the stack when it is getting full. Only
when this is not sufficient, PARI increases the real stack size within
the virtual stack. Important point: this resizing is in-place, unlike
realloc(). So all existing pointers remain valid.
The virtual stack size can only be changed when the stack is empty. In
GP, this means that it can happen only at the user prompt. In Sage, this
means any time between 2 calls through the PARI interface.
In GP, the real stack size is automatically reset to the original real
stack size whenever it gets back to the user prompt. In Sage, the real
stack size is never automatically decreased.
I'm probably going to try to modify a clean copy of PARI to do this, or
just write some completely separate test code to check that an mmap call
with PROT_NONE will work like we think it will work.
I tested this with a small stand-alone C program: the flags
PROT_NONE | MAP_PRIVATE | MAP_ANONYMOUS
allow to allocate huge amounts of virtual memory, even with overcommit=2.
Jeroen.
--
You received this message because you are subscribed to the Google Groups
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.
