[issue19088] TypeError with pickle in embedded python3.3 when starting multiple Interpreters.
Roundup Robot added the comment: New changeset 96d1207d33d0 by Alexandre Vassalotti in branch '3.3': Issue #19088: Fix incorrect caching of the copyreg module. http://hg.python.org/cpython/rev/96d1207d33d0 New changeset 1ceb6f84b617 by Alexandre Vassalotti in branch 'default': Issue #19088: Merge with 3.3. http://hg.python.org/cpython/rev/1ceb6f84b617 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19088 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19088] TypeError with pickle in embedded python3.3 when starting multiple Interpreters.
Changes by Alexandre Vassalotti alexan...@peadrop.com: -- assignee: - alexandre.vassalotti resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19088 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19088] TypeError with pickle in embedded python3.3 when starting multiple Interpreters.
Antoine Pitrou added the comment: Thanks for the fix. Perhaps you could have added some tests for this? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19088 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19088] TypeError with pickle in embedded python3.3 when starting multiple Interpreters.
New submission from Larry Pete:
Hexchat (fork of XChat IRC Client) switched with version 2.9.6 to Python 3.3
for their Python plugins.
Hexchat loads plugins in a similar fashion like the attached C file (not
entirely sure I used the C-API right though). For every plugin a new
interpreter is started and Py_EndInterpreter is used to unload the plugin.
When using pickle in such a python plugin, it raises the Exception (in the
attached example):
Traceback (most recent call last):
File string, line 7, in module
TypeError: attribute of type 'NoneType' is not callable
when trying to pickle an object (using pickle.dump or pickle.dumps).
The Exception happens on the line where pickle.dumps is called, though
pickle.dumps is not None.
It happens:
- with python3.3.2 (I also tested it with python3.4.0a2 since I happened to
have it installed, same issue). No issue with python2.7. I did not test it with
3.x versions prior to 3.3.2
- when trying to pickle a user defined class. Python objects like dictionaries
and lists work fine.
- only with the second (and any additional) interpreter.
- when destroying the interpreter and starting a new one, in that order. When
two interpreters are started and execute the code before any of them is
destroyed, it works fine.
The full output of the attached file when executed is btw:
First output:
Pickle dumps:
b'\x80\x03c__main__\nSomeClass\nq\x00)\x81q\x01}q\x02X\x08\x00\x00\x00some_varq\x03K{sb.'
Second output:
Pickle dumps:
Traceback (most recent call last):
File string, line 7, in module
TypeError: attribute of type 'NoneType' is not callable
--
components: Extension Modules
files: fail.c
messages: 198386
nosy: Larry.Pete, alexandre.vassalotti, pitrou
priority: normal
severity: normal
status: open
title: TypeError with pickle in embedded python3.3 when starting multiple
Interpreters.
type: behavior
versions: Python 3.3
Added file: http://bugs.python.org/file31867/fail.c
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19088
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19088] TypeError with pickle in embedded python3.3 when starting multiple Interpreters.
Amaury Forgeot d'Arc added the comment: The NoneType that fails is in typeobject.c: _PyObject_CallMethodId(copyreg, PyId__slotnames, O, cls); The error here is that copyreg comes from a cached reference to the module, stored in a static variable (cached_copyreg_module). When the interpreter shuts down the first time, all the module members are set to None... but the reference is kept forever. This variable should be reset, and reloaded (with a fresh module from the new interpreter) the next time it's used. I added a call to _PyType_Fini() in Py_EndInterpreter, this fixes the given example. Two sets of questions though: - There are many of these _Fini functions, called in Py_Finalize. Which ones should we call in Py_EndInterpreter? and why? Maybe this PyType_Fini() is not a _Fini, but should be renamed PyType_EndInterpreter? - one copyreg for multiple interpreters... this looks wrong: each interpreter has its own list of modules, but copyreg.__globals__ belongs to only one... A good solution would be to cache the copyreg module in the PyInterpreterState (it already has codec_search_cache). Or use PyImport_GetModuleDict() -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19088 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19088] TypeError with pickle in embedded python3.3 when starting multiple Interpreters.
Antoine Pitrou added the comment: one copyreg for multiple interpreters... this looks wrong: each interpreter has its own list of modules, but copyreg.__globals__ belongs to only one... A good solution would be to cache the copyreg module in the PyInterpreterState (it already has codec_search_cache). Yes, this is a reasonable solution. -- stage: - needs patch versions: +Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19088 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
