Charles-François Natali added the comment:
Alright, it bothered me so I wrote a small C testcase (attached),
which calls malloc in a loop, and can call memset upon the allocated
block right after allocation:
$ gcc -o /tmp/test /tmp/test.c; /tmp/test
malloc() returned NULL after 3050MB
$ gcc -DDO_MEMSET -o /tmp/test /tmp/test.c; /tmp/test
malloc() returned NULL after 2130MB
Without memset, the kernel happily allocates until we reach the 3GB
user address space limit.
With memset, it bails out way before.
I don't know what this'll give on 64-bit, but I assume one should get
comparable result.
I would guess that the reason why the Python list allocation fails is
because of the exponential allocation scheme: since memory is
allocated in large chunks before being used, the kernel happily
overallocates.
With a more progressive allocation+usage, it should return ENOMEM at some point.
Anyway, that's probably off-topic!
----------
Added file: http://bugs.python.org/file35059/test.c
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue21233>
_______________________________________
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define BLOCK_SIZE (10*1024*1024)
int main(int argc, char *argv[])
{
unsigned long size = 0;
char *p;
while ((p = malloc(BLOCK_SIZE)) != NULL) {
#ifdef DO_MEMSET
memset(p, 0, BLOCK_SIZE);
#endif
size += BLOCK_SIZE;
}
printf("malloc() returned NULL after %uMB\n", (size/(1024*1024)));
exit(EXIT_SUCCESS);
}
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
