[issue11321] 9th import of module _pickle always crashes

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11321] 9th import of module _pickle always crashes

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 1ae0b7b8de0b by Antoine Pitrou in branch '3.2':
Issue #11321: Fix a crash with multiple imports of the _pickle module when
http://hg.python.org/cpython/rev/1ae0b7b8de0b

New changeset 6674272754da by Antoine Pitrou in branch 'default':
Issue #11321: Fix a crash with multiple imports of the _pickle module when
http://hg.python.org/cpython/rev/6674272754da

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11321] 9th import of module _pickle always crashes

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Should be fixed now, thank you!

--
resolution:  -> fixed
stage:  -> committed/rejected
status: open -> closed
versions:  -Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11321] 9th import of module _pickle always crashes

[Palm Kevin]

New submission from Palm Kevin :

Please have a look to the following app:

#include "Python.h"

main(int argc, char **argv)
{
  int i;
  printf("START\n");
  for(i = 0; i < 20; i++){
Py_Initialize();

printf("Try import #%i ...", i);
PyRun_SimpleString("import _pickle\n");

printf("SUCCESS\n");
Py_Finalize();
  }
  printf("END\n");
}


If I run that app (Linux), then I get the following output:

START
Try import #0 ...SUCCESS
Try import #1 ...SUCCESS
Try import #2 ...SUCCESS
Try import #3 ...SUCCESS
Try import #4 ...SUCCESS
Try import #5 ...SUCCESS
Try import #6 ...SUCCESS
Try import #7 ...SUCCESS
Segmentation fault


It systematically crashes on the 9th call ...
Does anybody know the reason of this segmentation fault problem??

--
components: Interpreter Core
messages: 129384
nosy: palm.kevin
priority: normal
severity: normal
status: open
title: 9th import of module _pickle always crashes
type: crash
versions: Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11321] 9th import of module _pickle always crashes

[Palm Kevin]

Changes by Palm Kevin :


--
versions: +Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11321] 9th import of module _pickle always crashes

[Éric Araujo]

Changes by Éric Araujo :


--
nosy: +alexandre.vassalotti, brett.cannon, ncoghlan, pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11321] 9th import of module _pickle always crashes

[Nick Coghlan]

Nick Coghlan  added the comment:

I can definitely confirm that one:

~/devel/py3k$ gcc -I/usr/local/include/python3.3m -o issue11321 issue11321.c 
-L/usr/local/lib/python3.3 -lpython3.3m -lm -lpthread -ldl -lutil -rdynamic
~/devel/py3k$ ./issue11321
START
Try import #0 ...SUCCESS
Try import #1 ...SUCCESS
Try import #2 ...SUCCESS
Try import #3 ...SUCCESS
Try import #4 ...SUCCESS
Try import #5 ...SUCCESS
Try import #6 ...SUCCESS
Try import #7 ...SUCCESS
Segmentation fault

I can also confirm that it isn't a regression w.r.t 3.1:

~/devel/py3k$ gcc -I/usr/include/python3.1 -o issue11321 issue11321.c 
-L/usr/lib/python3.1 -lpython3.1 -lm -lpthread -ldl -lutil -rdynamic
~/devel/py3k$ ./issue11321
START
Try import #0 ...SUCCESS
Try import #1 ...SUCCESS
Try import #2 ...SUCCESS
Try import #3 ...SUCCESS
Try import #4 ...SUCCESS
Try import #5 ...SUCCESS
Try import #6 ...SUCCESS
Try import #7 ...SUCCESS
Segmentation fault

It *is*, however, a regression from the 2.x line (where, presumably, cPickle 
simply never got reinitialised)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11321] 9th import of module _pickle always crashes

[Nick Coghlan]

Nick Coghlan  added the comment:

I haven't worked out where the error is happening yet, but the 13 allocated 
static variables and the complete lack of a module cleanup function aren't 
going to be helping matters.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11321] 9th import of module _pickle always crashes

[Nick Coghlan]

Nick Coghlan  added the comment:

Here's a sample stack trace of it blowing up:

Program received signal SIGSEGV, Segmentation fault.
0x00479404 in type_dealloc (type=0x76a82f20) at 
Objects/typeobject.c:2523
2523_PyObject_GC_UNTRACK(type);

(gdb) bt
#0  0x00479404 in type_dealloc (type=0x76a82f20) at 
Objects/typeobject.c:2523
#1  0x00465a2b in dict_dealloc (mp=0x8f61b0) at 
Objects/dictobject.c:1020
#2  0x004e43c8 in _PyImport_FixupExtensionUnicode (mod=0x77f68ef0, 
name=0x7fffcd40 "_pickle", filename=0x77f12570) at 
Python/import.c:611
#3  0x004e7fcf in _PyImport_LoadDynamicModule (name=0x7fffcd40 
"_pickle", 
pathname=0x7fffbc70 
"/usr/local/lib/python3.3/lib-dynload/_pickle.cpython-33m.so", 
fp=) at ./Python/importdl.c:85
#4  0x004e5f79 in import_submodule (mod=0x796b20, 
subname=0x7fffcd40 "_pickle", 
fullname=0x7fffcd40 "_pickle") at Python/import.c:2894
#5  0x004e61ef in load_next (mod=0x796b20, altmod=, 
p_name=, buf=0x7fffcd40 "_pickle", 
p_buflen=0x7fffcd38)
at Python/import.c:2706
#6  0x004e6931 in import_module_level (name=0x0, globals=0x922060, 
locals=, fromlist=0x796b20, level=)
at Python/import.c:2422
#7  0x004e6e74 in PyImport_ImportModuleLevel (name=0x77e6c550 
"_pickle", 
globals=0x922060, locals=0x922060, fromlist=0x796b20, level=)
at Python/import.c:2474
#8  0x004c642b in builtin___import__ (self=, 
args=, kwds=) at 
Python/bltinmodule.c:167
#9  0x0042c897 in PyObject_Call (func=0x77edf8c0, arg=0x8e79e0, 
kw=0xa2e79)
at Objects/abstract.c:2149
#10 0x004c6c73 in PyEval_CallObjectWithKeywords (func=0x77edf8c0, 
arg=0x77e85ef0, kw=0xa2e79) at Python/ceval.c:3739
#11 0x004ca6de in PyEval_EvalFrameEx (f=0xae4490, throwflag=)
at Python/ceval.c:2316
#12 0x004cf343 in PyEval_EvalCodeEx (_co=0x77ea5690, globals=, 
locals=, args=0x0, argcount=, 
kws=, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, 
closure=0x0)
at Python/ceval.c:3295
#13 0x004cf5db in PyEval_EvalCode (co=0x76a82f20, globals=0x8e79e0, 
locals=0xa2e79)
at Python/ceval.c:761
#14 0x00419c7e in run_mod (command=, flags=)
at Python/pythonrun.c:1769
#15 PyRun_StringFlags (command=, flags=)
at Python/pythonrun.c:1703
#16 PyRun_SimpleStringFlags (command=, flags=)
at Python/pythonrun.c:1276
#17 0x00415c41 in main ()

So _pickle itself doesn't seem to be dying, but it is eventually leaving things 
in a bad state so that the *next* time around, the import machinery blows up. 
Note that switching to a different module (e.g. "_struct") eliminates the 
segfault.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11321] 9th import of module _pickle always crashes

[Andreas Stührk]

Andreas Stührk  added the comment:

It's because the _pickle module doesn't incref Pickler_Type and Unpickler_Type 
before calling `PyModule_AddObject()` (which steals a reference).

The attached patch fixes the issue.

--
keywords: +patch
nosy: +Trundle
Added file: http://bugs.python.org/file20931/issue11321.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11321] 9th import of module _pickle always crashes

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc  added the comment:

The patch looks good. It may also fix issue9197, which fails in a similar 
fashion

--
nosy: +amaury.forgeotdarc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11321] 9th import of module _pickle always crashes

[Nick Coghlan]

Nick Coghlan  added the comment:

While Andreas's patch does indeed prevent the crash, there is something more 
going on here.

I modified his patch to print out the reference counts immediately after the 
new INCREF commands.

With the INCREF commands commented out, it looks like this:

~/devel/py3k$ ./issue113213.3
START
Try import #0 ...Initialising _pickle
Pickler type references: 10
Unpickler type references: 8
Module initialised
SUCCESS
Try import #1 ...Initialising _pickle
Pickler type references: 10
Unpickler type references: 8
Module initialised
SUCCESS
Try import #2 ...Initialising _pickle
Pickler type references: 9
Unpickler type references: 7
Module initialised
SUCCESS
Try import #3 ...Initialising _pickle
Pickler type references: 8
Unpickler type references: 6
Module initialised
SUCCESS
Try import #4 ...Initialising _pickle
Pickler type references: 7
Unpickler type references: 5
Module initialised
SUCCESS
Try import #5 ...Initialising _pickle
Pickler type references: 6
Unpickler type references: 4
Module initialised
SUCCESS
Try import #6 ...Initialising _pickle
Pickler type references: 5
Unpickler type references: 3
Module initialised
SUCCESS
Try import #7 ...Initialising _pickle
Pickler type references: 4
Unpickler type references: 2
Module initialised
SUCCESS
Try import #8 ...Initialising _pickle
Pickler type references: 3
Unpickler type references: 1
Module initialised
Segmentation fault

Note that it does the right thing the first time Py_Finalize is called, but the 
reference counts start going down after that.

When the INCREF is included, the count goes up initially and then levels out.

So I believe the simple patch is just masking the problem instead of fixing it. 
The correct answer is to follow the documentation and implement the module 
finalisation protocol for _pickle (i.e. move all the static globals into the 
module struct and implement module traversal and finalisation functions).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11321] 9th import of module _pickle always crashes

[Andreas Stührk]

Andreas Stührk  added the comment:

That initial increase you can see happens because the module's dict is copied 
in `_PyImport_FixupExtensionUnicode()`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11321] 9th import of module _pickle always crashes

[Nick Coghlan]

Nick Coghlan  added the comment:

Looking at the example of _struct.c, I'll withdraw my objection to the patch. 
The segfault is due to the attempt to destroy a statically allocated type 
object, and the only viable solution to that is to ensure the reference count 
never drops to 0, even during Py_Finalize.

I also created issue11349 to cover the fact that _pickle really shouldn't be 
stashing things in static globals now that a better alternative is available.

--
nosy: +loewis
versions: +Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com