Charles-Francois Natali <[email protected]> added the comment:
It's definitely a stack overflow.
Most of the backtraces show an important number of frames.
The last frame is this:
#0 PyMarshal_ReadLastObjectFromFile (fp=0x13e8200)
at ../Python/marshal.c:1026
filesize = <value optimized out>
and a disassembly show us that the segfault is generated on a callq:
0x4bd4d6 <PyMarshal_ReadLastObjectFromFile+54>: callq 0x4168e8 <fil...@plt>
And if you look at the code, it's obvious what's happening:
PyObject *
PyMarshal_ReadLastObjectFromFile(FILE *fp)
{
/* 75% of 2.1's .pyc files can exploit SMALL_FILE_LIMIT.
* REASONABLE_FILE_LIMIT is by defn something big enough for Tkinter.pyc.
*/
#define SMALL_FILE_LIMIT (1L << 14)
#define REASONABLE_FILE_LIMIT (1L << 18)
#ifdef HAVE_FSTAT
off_t filesize;
#endif
#ifdef HAVE_FSTAT
filesize = getfilesize(fp);
if (filesize > 0) {
char buf[SMALL_FILE_LIMIT];
char* pBuf = NULL;
if (filesize <= SMALL_FILE_LIMIT)
pBuf = buf;
else if (filesize <= REASONABLE_FILE_LIMIT)
pBuf = (char *)PyMem_MALLOC(filesize);
if (pBuf != NULL) {
[...]
}
SMALL_FILE_LIMIT is 1 << 14 which is roughly 16K (not that reasonable :-).
So when we enter PyMarshal_ReadLastObjectFromFile and allocate buf, we push
around 16K on the stack, which is a lot. That's why we segfault soon after when
we call a function (callq), there's no space left on the stack.
So there are several solutions:
- make buf static, but it would increase Python size by almost 16K
- reduce SMALL_FILE_LIMIT, or remove it altogether. I guess SMALL_FILE_LIMIT is
there to speedup loading of small files, but I'm not sure that malloc() would
incur an important overhead
- reading the whole file to memory sounds weird, we sould probably be using
mmap() here
Peers ?
----------
nosy: +neologix
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue7332>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
