marche147 <[email protected]> added the comment:
Thanks for the repro! It did help for pinpointing the issue.
So I took a little spare time and dived into xnu kernel code, here is my
assumption based on what I found (N.B. : My assumption comes from a simple
experiment and a brief skim of the source code within 15 minutes or less, it
could be seriously wrong since I'm not an expert of XNU kernel, and I currently
don't have the time to build and debug it.) :
In bsd/kern/kern_resource.c, there's a function `dosetrlimit` which handles the
`setrlimit` request, and here's part of it:
```
case RLIMIT_STACK:
// ...
if (limp->rlim_cur > alimp->rlim_cur) {
user_addr_t addr;
user_size_t size;
/* grow stack */
size = round_page_64(limp->rlim_cur);
size -= round_page_64(alimp->rlim_cur);
addr = p->user_stack - round_page_64(limp->rlim_cur);
kr = mach_vm_protect(current_map(),
addr, size,
FALSE, VM_PROT_DEFAULT);
if (kr != KERN_SUCCESS) {
error = EINVAL;
goto out;
}
} // ...
```
As we can see, the kernel will try to `mprotect` the memory preceding the stack
to `VM_PROT_DEFAULT` (presumably read & write). I then used `vmmap` to see the
difference between two binaries compiled with different commands. And the
results are :
1. Binary compiled without default stack size:
```
- Before calling setrlimit
...
STACK GUARD 00007ffee76d9000-00007ffeeaed9000 [ 56.0M 0K 0K
0K] ---/rwx SM=NUL stack guard for thread 0
...
Stack 00007ffeeaed9000-00007ffeeb6d9000 [ 8192K 20K 20K
0K] rw-/rwx SM=PRV thread 0
...
VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE
NONVOL EMPTY REGION
REGION TYPE SIZE SIZE SIZE SIZE SIZE
SIZE SIZE COUNT (non-coalesced)
=========== ======= ======== ===== ======= ========
====== ===== =======
Kernel Alloc Once 8K 4K 4K 0K 0K
0K 0K 2
MALLOC guard page 16K 0K 0K 0K 0K
0K 0K 5
MALLOC metadata 60K 60K 60K 0K 0K
0K 0K 6
MALLOC_SMALL 16.0M 16K 16K 0K 0K
0K 0K 3 see MALLOC ZONE table below
MALLOC_TINY 2048K 32K 32K 0K 0K
0K 0K 3 see MALLOC ZONE table below
STACK GUARD 56.0M 0K 0K 0K 0K
0K 0K 2
Stack 8192K 20K 20K 0K 0K
0K 0K 2
__DATA 2324K 1192K 208K 0K 0K
0K 0K 43
__LINKEDIT 192.7M 21.7M 0K 0K 0K
0K 0K 4
__TEXT 9448K 8224K 0K 0K 0K
0K 0K 48
shared memory 8K 8K 8K 0K 0K
0K 0K 3
=========== ======= ======== ===== ======= ========
====== ===== =======
TOTAL 286.3M 31.0M 348K 0K 0K
0K 0K 110
...
- After calling setrlimit
...
STACK GUARD 00007ffee76d9000-00007ffee76da000 [ 4K 0K 0K
0K] ---/rwx SM=NUL stack guard for thread 0
...
Stack 00007ffee76da000-00007ffeeaed9000 [ 56.0M 0K 0K
0K] rw-/rwx SM=NUL thread 0
Stack 00007ffeeaed9000-00007ffeeb6d9000 [ 8192K 20K 20K
0K] rw-/rwx SM=PRV thread 0
...
VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE
NONVOL EMPTY REGION
REGION TYPE SIZE SIZE SIZE SIZE SIZE
SIZE SIZE COUNT (non-coalesced)
=========== ======= ======== ===== ======= ========
====== ===== =======
Kernel Alloc Once 8K 4K 4K 0K 0K
0K 0K 2
MALLOC guard page 16K 0K 0K 0K 0K
0K 0K 5
MALLOC metadata 60K 60K 60K 0K 0K
0K 0K 6
MALLOC_SMALL 16.0M 20K 20K 0K 0K
0K 0K 3 see MALLOC ZONE table below
MALLOC_TINY 2048K 32K 32K 0K 0K
0K 0K 3 see MALLOC ZONE table below
STACK GUARD 4K 0K 0K 0K 0K
0K 0K 2
Stack 64.0M 20K 20K 0K 0K
0K 0K 3
__DATA 2324K 1192K 208K 0K 0K
0K 0K 43
__LINKEDIT 192.7M 21.7M 0K 0K 0K
0K 0K 4
__TEXT 9448K 8224K 0K 0K 0K
0K 0K 48
shared memory 8K 8K 8K 0K 0K
0K 0K 3
=========== ======= ======== ===== ======= ========
====== ===== =======
TOTAL 286.3M 31.0M 352K 0K 0K
0K 0K 111
...
```
2. Binary compiled with default stack size:
```
Before calling setrlimit :
...
STACK GUARD 00007ffee09c2000-00007ffee09c3000 [ 4K 0K 0K
0K] ---/rwx SM=NUL stack guard for thread 0
...
Stack 00007ffee09c3000-00007ffee19c3000 [ 16.0M 20K 20K
0K] rw-/rwx SM=PRV thread 0
...
VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE
NONVOL EMPTY REGION
REGION TYPE SIZE SIZE SIZE SIZE SIZE
SIZE SIZE COUNT (non-coalesced)
=========== ======= ======== ===== ======= ========
====== ===== =======
Kernel Alloc Once 8K 4K 4K 0K 0K
0K 0K 2
MALLOC guard page 16K 0K 0K 0K 0K
0K 0K 5
MALLOC metadata 60K 60K 60K 0K 0K
0K 0K 6
MALLOC_SMALL 8192K 12K 12K 0K 0K
0K 0K 2 see MALLOC ZONE table below
MALLOC_TINY 1024K 20K 20K 0K 0K
0K 0K 2 see MALLOC ZONE table below
STACK GUARD 4K 0K 0K 0K 0K
0K 0K 2
Stack 16.0M 20K 20K 0K 0K
0K 0K 2
__DATA 2324K 1192K 208K 0K 0K
0K 0K 43
__LINKEDIT 192.7M 22.3M 0K 0K 0K
0K 0K 4
__TEXT 9448K 8232K 0K 0K 0K
0K 0K 48
shared memory 8K 8K 8K 0K 0K
0K 0K 3
=========== ======= ======== ===== ======= ========
====== ===== =======
TOTAL 229.3M 31.7M 332K 0K 0K
0K 0K 108
```
As we can see, it seems that the kernel tried to `mprotect` (or we can say,
allocate) from the "STACK GUARD" region. So where does this "STACK GUARD"
region comes from? Let's see this:
bsd/kern/kern_exec.c, in `create_unix_stack` function (where the kernel creates
the stack for a new task, I assume) :
```
...
#define unix_stack_size(p) (p->p_rlimit[RLIMIT_STACK].rlim_cur)
...
if (load_result->user_stack_size == 0) {
load_result->user_stack_size = unix_stack_size(p);
prot_size = mach_vm_trunc_page(size - load_result->user_stack_size);
} else {
prot_size = PAGE_SIZE;
}
prot_addr = addr;
kr = mach_vm_protect(map,
prot_addr,
prot_size,
FALSE,
VM_PROT_NONE);
...
```
So that comes my conclusion: if the binary has a specified default stack size,
this `load_result->user_stack_size` would not be zero (this should be set
somewhere inside the mach-o parser/loader, I guess), so the kernel will only
map a small page for the "STACK GUARD" region, otherwise the kernel will use
the current stack size soft limit (inherited from the parent) as the
`user_stack_size` and calculates a `prot_size`, which should be (rlim_max -
rlim_cur). And of course, the python3 binary was built with default stack size,
so the kernel does not provide a huge enough "STACK GUARD" region for the
`setrlimit` syscall to allot more stack space than the default stack size.
----------
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue34602>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
