New submission from Raoul Gough:
Description
===========
Running the attached example program with Python 2.7.12 produces the output
below. The demo deliberately raises a user-defined exception during the
unpickling process, but the problem is that this exception does not propagate
out of the unpickle call. Instead, it gets converted into a TypeError which is
confusing and does not help identify the original problem.
INFO:root:Creating BadReduce object
INFO:root:Pickling
INFO:root:Unpickling
INFO:root:Raising exception "BadReduce init failed"
Traceback (most recent call last):
File "cpickle_reduce_failure.py", line 48, in <module>
main()
File "cpickle_reduce_failure.py", line 41, in main
pickler.loads(s1)
File "cpickle_reduce_failure.py", line 27, in __init__
raise exception
TypeError: __init__() takes exactly 2 arguments (4 given)
If you change the demo to use the Python pickle module, i.e. "import pickle as
pickler", it produces the expected output below:
INFO:root:Creating BadReduce object
INFO:root:Pickling
INFO:root:Unpickling
INFO:root:Raising exception "BadReduce init failed"
INFO:root:Got MyException "BadReduce init failed"
INFO:root:Done
Analysis
========
The following code in Modules/cPickle.c in the function Instance_New (around
https://github.com/python/cpython/blob/2.7/Modules/cPickle.c#L3917) does a
PyErr_Restore with the exception type MyException, as raised from
BadReduce.__init__, but it replaces the exception value with a tuple
(original_exception, cls, args):
PyObject *tp, *v, *tb, *tmp_value;
PyErr_Fetch(&tp, &v, &tb);
tmp_value = v;
/* NULL occurs when there was a KeyboardInterrupt */
if (tmp_value == NULL)
tmp_value = Py_None;
if ((r = PyTuple_Pack(3, tmp_value, cls, args))) {
Py_XDECREF(v);
v=r;
}
PyErr_Restore(tp,v,tb);
Later, PyErr_NormalizeException attempts to convert the exception value (the
tuple) to the original exception type. This fails because
MyException.__init__() can't handle the multiple arguments. This is what
produces the TypeError "__init__() takes exactly 2 arguments (4 given)"
Proposed Fix
============
I think it would be better if Instance_New did the PyErr_NormalizeException
itself instead of allowing it to be done lazily. If the normalize works, i.e.
the exception type accepts the extra arguments, the behaviour would be almost
unchanged - the only difference being the PyErr_NormalizeException happens
earlier. However, if the normalize fails, Instance_New can restore the original
exception unchanged. That means that we lose the cls, args information in this
case, but not the original exception.
----------
components: Library (Lib)
files: cpickle_reduce_failure.py
messages: 282795
nosy: raoul_gough_baml
priority: normal
severity: normal
status: open
title: Confusing exception from cPickle on reduce failure
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file45822/cpickle_reduce_failure.py
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue28925>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
