[issue18068] pickle + weakref.proxy(self)

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The weakref.proxy object delegates all attribute access to referred object, 
including special attributes used in pickling, like __class__ or __getstate__.

test.recursive is not test, but it looks as TestPickle instance for the 
pickler. It is pickled and unpickled as TestPickle instance.

>>> test.recursive.__class__

>>> test.recursive.__getstate__
>
>>> test.recursive.__reduce_ex__(2)
__getstate__ 3071478700
(, (,), 
{'recursive': }, None, 
None)
>>> pickle.loads(pickle.dumps(test.recursive, 2))
__getstate__ 3071478700
__setstate__ 3071525356
<__main__.TestPickle object at 0xb713c1ec>

Since test.recursive.recursive is test.recursive, this recursion is detected by 
pickler.

This issue is similar to issue6395, but since weakref.proxy is not 
subclassable, we can just implement correct __reduce__.

--
assignee:  -> serhiy.storchaka
nosy: +serhiy.storchaka
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18068] pickle + weakref.proxy(self)

[Antoine Pitrou]

Antoine Pitrou added the comment:

I don't think weakref objects are meant to be picklable at all. At least 
there's no support code for that, so pickling weak proxies only works by chance.

--
nosy: +alexandre.vassalotti
type: behavior -> enhancement
versions: +Python 3.4 -Python 2.6, Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18068] pickle + weakref.proxy(self)

[Georg Brandl]

Changes by Georg Brandl :


--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18068] pickle + weakref.proxy(self)

[Oleg Broytman]

New submission from Oleg Broytman:

Hello! I've found a problematic behaviour of pickle when it pickles
(direct or indirect) weak proxies to self. I suspect pickle cannot
detect reference loops with weak proxies. I managed to narrow the case
to the following example:

import pickle, weakref

class TestPickle(object):
def __init__(self):
self.recursive = weakref.proxy(self)

def __getstate__(self):
print "__getstate__", id(self)
return self.__dict__.copy()

def __setstate__(self, d):
print "__setstate__", id(self)
self.__dict__.update(d)

print "- 1 -"
test = TestPickle()
print "- 2 -"
data = pickle.dumps(test, pickle.HIGHEST_PROTOCOL)
print "- 3 -"
test2 = pickle.loads(data)
print "- 4 -"
print "Result:", id(test2)
print "- 5 -"

   It prints:

- 1 -
- 2 -
__getstate__ 3075348620
__getstate__ 3075348620
- 3 -
__setstate__ 3075348844
__setstate__ 3075349004
- 4 -
Result: 3075349004
- 5 -

   That is, __getstate__ is called twice for the same object. And what
is worse, __setstate__ is called twice for different objects. The
resulting unpickled object is the last one, but in the library that I
have been debugging creation of two different objects during unpickling
is a bug.

   I can fix it by avoiding pickling the proxy and recreating the proxy
on unpickling:

import pickle, weakref

class TestPickle(object):
def __init__(self):
self.recursive = weakref.proxy(self)

def __getstate__(self):
print "__getstate__", id(self)
d = self.__dict__.copy()
del d['recursive']
return d

def __setstate__(self, d):
print "__setstate__", id(self)
self.__dict__.update(d)
self.recursive = weakref.proxy(self)

print "- 1 -"
test = TestPickle()
print "- 2 -"
data = pickle.dumps(test, pickle.HIGHEST_PROTOCOL)
print "- 3 -"
test2 = pickle.loads(data)
print "- 4 -"
print "Result:", id(test2)
print "- 5 -"

- 1 -
- 2 -
__getstate__ 3075070092
- 3 -
__setstate__ 3075070188
- 4 -
Result: 3075070188
- 5 -

   But I wonder if it's a bug that should be fixed? If it's an expected
behaviour it perhaps should be documented as a warning in docs for
pickle or weakref or both.

--
components: Library (Lib)
messages: 190105
nosy: phd
priority: normal
severity: normal
status: open
title: pickle + weakref.proxy(self)
type: behavior
versions: Python 2.6, Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com