[issue13520] Patch to make pickle aware of __qualname__
Changes by Alexandre Vassalotti alexan...@peadrop.com: -- assignee: - alexandre.vassalotti resolution: - duplicate stage: patch review - committed/rejected status: open - closed superseder: - Implement PEP 3154 (pickle protocol 4) versions: +Python 3.4 -Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13520 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13520] Patch to make pickle aware of __qualname__
Richard Oudkerk added the comment:
There is a cute way to use operator.attrgetter to produce backwards compatible
pickles using the qualname:
import pickle, copyreg, operator, sys, pickletools, types
class AttrGetter(object):
def __init__(self, name):
self.name = name
def __call__(self): # pretend to be callable
raise RuntimeError
def __reduce__(self):
return operator.attrgetter, (self.name,)
def reduce_by_qualname(obj):
mod = sys.modules[obj.__module__]
first, rest = obj.__qualname__.split('.', 1)
firstobj = getattr(mod, first)
assert operator.attrgetter(rest)(firstobj) is obj
return AttrGetter(rest), (firstobj,)
# FunctionType defaults to save_global but uses fallback if it fails
copyreg.pickle(types.FunctionType, reduce_by_qualname)
class A(object):
class B(object):
class C(object):
@staticmethod
def foo():
print(foo foo foo)
def bar():
print(bar bar bar)
for obj in [A.B.C.foo, bar]:
data = pickle.dumps(obj, 2)
pickletools.dis(data)
func = pickle.loads(data)
assert func is obj
func()
This produces
0: \x80 PROTO 2
2: cGLOBAL 'operator attrgetter'
23: qBINPUT 0
25: XBINUNICODE 'B.C.foo'
37: qBINPUT 1
39: \x85 TUPLE1
40: qBINPUT 2
42: RREDUCE
43: qBINPUT 3
45: cGLOBAL '__main__ A'
57: qBINPUT 4
59: \x85 TUPLE1
60: qBINPUT 5
62: RREDUCE
63: qBINPUT 6
65: .STOP
highest protocol among opcodes = 2
foo foo foo
0: \x80 PROTO 2
2: cGLOBAL '__main__ bar'
16: qBINPUT 0
18: .STOP
highest protocol among opcodes = 2
bar bar bar
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13520
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13520] Patch to make pickle aware of __qualname__
Changes by Hynek Schlawack h...@ox.cx: -- nosy: -hynek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13520 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13520] Patch to make pickle aware of __qualname__
Changes by Łukasz Langa luk...@langa.pl: -- nosy: +lukasz.langa ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13520 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13520] Patch to make pickle aware of __qualname__
Changes by Hynek Schlawack h...@ox.cx: -- nosy: +hynek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13520 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13520] Patch to make pickle aware of __qualname__
sbt shibt...@gmail.com added the comment: It looks like Issue 3657 is really about builtin methods (i.e. builtin_function_or_method objects where __self__ is not a module). It causes no problem for normal python instance methods. If we tried the getattr approach for builtin methods too then it should only be used as a fallback when saving as a global will not work. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13520 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13520] Patch to make pickle aware of __qualname__
Alexandre Vassalotti alexan...@peadrop.com added the comment: This might not be the case anymore, but __module__ can sometime be None. There is some discussion about this in Issue 3657. We should define the expected behavior when this happens. Also, I don't think backward-compatibility of the protocol is a big issue if we use the getattr approach. I feel bumping the protocol version is only necessary if we introduce new opcodes or change their interpretation. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13520 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13520] Patch to make pickle aware of __qualname__
New submission from sbt shibt...@gmail.com: The attached patch makes pickle use an object's __qualname__ attribute if __name__ does not work. This makes nested classes, unbound instance methods and static methods picklable (assuming that __module__ and __qualname__ give the correct address). BTW, It would not be hard to make instance methods picklable (and, as a by-product, class methods) by giving instance methods a __reduce__ method equivalent to def __reduce__(self): return (getattr, (self.__self__, self.__name__)) Is there any reason not to make such a change? -- files: pickle_qualname.patch keywords: patch messages: 148759 nosy: pitrou, sbt priority: normal severity: normal status: open title: Patch to make pickle aware of __qualname__ versions: Python 3.3 Added file: http://bugs.python.org/file23836/pickle_qualname.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13520 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13520] Patch to make pickle aware of __qualname__
Changes by Ezio Melotti ezio.melo...@gmail.com: -- components: +Extension Modules stage: - patch review type: - feature request ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13520 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13520] Patch to make pickle aware of __qualname__
Changes by STINNER Victor victor.stin...@haypocalc.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13520 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13520] Patch to make pickle aware of __qualname__
Antoine Pitrou pit...@free.fr added the comment: The attached patch makes pickle use an object's __qualname__ attribute if __name__ does not work. This makes nested classes, unbound instance methods and static methods picklable (assuming that __module__ and __qualname__ give the correct address). Thanks. I'm not sure that's a good approach for protocol 3: some pickles created by Python 3.3 would not be readable by Python 3.2. That's why I initially added that idea to PEP 3154, for a hypothetical protocol 4. However, perhaps it is possible to use the same reduce/getattr trick you are proposing for instance methods? BTW, It would not be hard to make instance methods picklable (and, as a by-product, class methods) by giving instance methods a __reduce__ method equivalent to def __reduce__(self): return (getattr, (self.__self__, self.__name__)) Is there any reason not to make such a change? I don't see any personally. Actually, multiprocessing has a Pickler subclass called ForkingPickler which does something similar (in Lib/multiprocessing/forking.py). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13520 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13520] Patch to make pickle aware of __qualname__
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +alexandre.vassalotti ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13520 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
