[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Ned Deily]

Ned Deily added the comment:

Excellent!  Thanks for digging into that.  Increasing the iterations seems to 
work fine within the (understandable) limitations of the existing embedded 
tests, e.g. only work from a source tree build (not an installed build nor from 
a separate build directory) etc.  Case closed.

--
resolution:  -> fixed
stage: test needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Roundup Robot]

Roundup Robot added the comment:

New changeset 4f69626fd923 by Ned Deily in branch 'default':
Issue #27736: Improve the existing embedded interpreter init/fini test
https://hg.python.org/cpython/rev/4f69626fd923

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Xiang Zhang]

Xiang Zhang added the comment:

We don't have to change Lib/test/test_capi.py. It has already got a repeated 
init and fini test[1]. All we need to do is increase the loop number[2]. I have 
tried with 15, the test case fails reliably. And with the previous patch 
applied, it won't fail.

[1] https://hg.python.org/cpython/file/tip/Lib/test/test_capi.py#l386
[2] https://hg.python.org/cpython/file/tip/Programs/_testembed.c#l44

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Ned Deily]

Ned Deily added the comment:

I've applied the patch for 3.6.0a4 so the immediate issue is fixed.  I wonder 
if it would be worthwhile adapting Simon's test case to add to the embedding 
test cases in Lib/test/test_capi.py.  I'm going to leave the issue open for a 
while in case someone is interested in doing that.

--
assignee: ned.deily -> 
priority: release blocker -> 
stage: commit review -> test needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Roundup Robot]

Roundup Robot added the comment:

New changeset a95d98086621 by Ned Deily in branch 'default':
Issue #27736: Prevent segfault after interpreter re-initialization due
https://hg.python.org/cpython/rev/a95d98086621

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Brett Cannon]

Brett Cannon added the comment:

Patch LGTM.

--
assignee: brett.cannon -> ned.deily
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Ned Deily]

Ned Deily added the comment:

Thanks for the patch.  FWIW, that does seem to solve the crash. Brett, look 
good to you?  If so, I can push the fix.

--
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Xiang Zhang]

Xiang Zhang added the comment:

Try Simon's snippet I think I am right.

With the current code, the result is:

1..1
Segmentation fault (core dumped)

With Py_INCREF:

1..1
ok 1 - was able to loop 100 times

I upload the working patch.

--
keywords: +patch
Added file: http://bugs.python.org/file44093/issue27736.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Xiang Zhang]

Xiang Zhang added the comment:

It looks to me from the traceback Simon given that DirEntryType's dealloc 
function is called. From the code, shouldn't we Py_INCREF DirEntryType before 
PyModule_AddObject?

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Brett Cannon]

Brett Cannon added the comment:

The revision that Ned found only changed posixmodule.c by adding 
PyModule_AddObject(m, "DirEntry", (PyObject *)); at the end of the 
module initializer. Only thing I can think of is it needs to go into the `if 
(!initialized)` block: 
https://github.com/python/cpython/blob/11b48001fb8d908f6db164e9d2233e911f22d4f4/Modules/posixmodule.c#L13214
 (maybe as an else clause for the PyType_Ready() call at 
https://github.com/python/cpython/blob/11b48001fb8d908f6db164e9d2233e911f22d4f4/Modules/posixmodule.c#L13259).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Ned Deily]

Ned Deily added the comment:

Thanks for the good test, Simon.  Bisection points to b841972ed0bd for 
Issue27038 as the culprit here.  (It was pushed between 3.6.0a2 and a3.)  
Brett, Jelle, can you please take a look at this?  I'm going to keep it as a 
Release Blocker for now.

--
assignee:  -> brett.cannon
nosy: +Jelle Zijlstra, brett.cannon
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Gregory P. Smith]

Changes by Gregory P. Smith :


--
nosy: +ned.deily
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Simon McVittie]

Simon McVittie added the comment:

This might be a duplicate of https://bugs.python.org/issue24853 but there 
wasn't enough detail on that bug for me to be sure.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27736] repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults

[Simon McVittie]

New submission from Simon McVittie:

dbus-python has a regression test for 
https://bugs.freedesktop.org/show_bug.cgi?id=23831 which repeatedly initializes 
the interpreter, imports dbus and finalizes the interpreter. This test passes 
in Python up to 3.5, but is failing under Python 3.6 nightly builds on 
Travis-CI, and in Python 3.6.0a3 (package version 3.6.0~a3-1) on Debian.

I've been able to reproduce the crash without anything specific to dbus with 
this C code:

#include 

#include 

int main(void)
{
int i;

puts("1..1");

for (i = 0; i < 100; ++i) {
Py_Initialize();
if (PyRun_SimpleString("\n") != 0) {
puts("not ok 1 - there was an exception");
return 1;
}
Py_Finalize();
}

puts("ok 1 - was able to loop 100 times");

return 0;
}

It appears the crash is reliably in the 10th repeat:

Program received signal SIGSEGV, Segmentation fault.
0x7783a4bb in type_dealloc (type=0x77d8ad80 )
at ../Objects/typeobject.c:3032
3032../Objects/typeobject.c: No such file or directory.
(gdb) bt
#0  0x7783a4bb in type_dealloc (type=0x77d8ad80 )
at ../Objects/typeobject.c:3032
#1  0x77817a1b in insertdict (value=, hash=, 
key=, mp=) at ../Objects/dictobject.c:806
#2  PyDict_SetItem (
op=op@entry={'open': None, 'O_DIRECT': None, 'chdir': None, 'O_ACCMODE': 
None, '__package__': None, 'WCOREDUMP': None, 'setgroups': None, 'O_CREAT': 
None, 'O_CLOEXEC': None, 'chown': None, 'sched_getscheduler': None, 
'RTLD_NODELETE': None, 'terminal_size': None, 'EX_IOERR': None, 
'sched_setaffinity': None, 'XATTR_SIZE_MAX': None, 'fstat': None, 
'sched_rr_get_interval': None, 'O_LARGEFILE': None, 'times_result': None, 
'get_inheritable': None, 'WIFEXITED': None, 'ST_NODEV': None, 'forkpty': None, 
'ctermid': None, 'O_RSYNC': None, 'SCHED_FIFO': None, 'stat': None, 'replace': 
None, 'EX_NOINPUT': None, 'WUNTRACED': None, 'set_blocking': None, 
'_have_functions': None, 'unsetenv': None, 'setresgid': None, 'fchown': None, 
'getgrouplist': None, 'openpty': None, 'lockf': None, 'chroot': None, 'readv': 
None, 'EX_NOHOST': None, 'error': None, 'WEXITSTATUS': None, 'WIFSIGNALED': 
None, 'WNOHANG': None, 'POSIX_FADV_WILLNEED': None, 'SEEK_HOLE': None, 'dup': 
None, 'POSIX_FADV_NOREUSE': None, 'ki
 ll': None, 'statvfs_result': None, 'WIFCON...(truncated), 
key='DirEntry', value=value@entry=None) at ../Objects/dictobject.c:1228
#3  0x7782659c in _PyModule_ClearDict (
d={'open': None, 'O_DIRECT': None, 'chdir': None, 'O_ACCMODE': None, 
'__package__': None, 'WCOREDUMP': None, 'setgroups': None, 'O_CREAT': None, 
'O_CLOEXEC': None, 'chown': None, 'sched_getscheduler': None, 'RTLD_NODELETE': 
None, 'terminal_size': None, 'EX_IOERR': None, 'sched_setaffinity': None, 
'XATTR_SIZE_MAX': None, 'fstat': None, 'sched_rr_get_interval': None, 
'O_LARGEFILE': None, 'times_result': None, 'get_inheritable': None, 
'WIFEXITED': None, 'ST_NODEV': None, 'forkpty': None, 'ctermid': None, 
'O_RSYNC': None, 'SCHED_FIFO': None, 'stat': None, 'replace': None, 
'EX_NOINPUT': None, 'WUNTRACED': None, 'set_blocking': None, '_have_functions': 
None, 'unsetenv': None, 'setresgid': None, 'fchown': None, 'getgrouplist': 
None, 'openpty': None, 'lockf': None, 'chroot': None, 'readv': None, 
'EX_NOHOST': None, 'error': None, 'WEXITSTATUS': None, 'WIFSIGNALED': None, 
'WNOHANG': None, 'POSIX_FADV_WILLNEED': None, 'SEEK_HOLE': None, 'dup': None, 
'POSIX_FADV_NOREUSE': None, 'kill': None,
  'statvfs_result': None, 'WIFCON...(truncated))
at ../Objects/moduleobject.c:593
#4  0x7782672e in _PyModule_Clear (m=m@entry=)
at ../Objects/moduleobject.c:544
#5  0x778d5874 in PyImport_Cleanup () at ../Python/import.c:452
#6  0x778e3a38 in Py_FinalizeEx () at ../Python/pylifecycle.c:588
#7  0x00400795 in main () at 
/home/smcv/src/dbus-python/test/import-repeatedly.c:19
(gdb) frame 7
#7  0x00400795 in main () at 
/home/smcv/src/dbus-python/test/import-repeatedly.c:19
19  Py_Finalize();
(gdb) p i
$1 = 10

--
components: Interpreter Core
messages: 272423
nosy: smcv
priority: normal
severity: normal
status: open
title: repeated Py_Initialize/PyRun_SimpleString/Py_Finalize segfaults
type: crash
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com