#25426: pickling SimpleLazyObject fails just after accessing related object of
wrapped model instance.
-------------------------------------+-------------------------------------
Reporter: iru | Owner: nobody
Type: Bug | Status: new
Component: Core | Version: 1.8
(Serialization) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by iru):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Old description:
> Reproduction step :
> {{{
> import pickle
> import django
> django.setup()
>
> from django.contrib.auth.models import User
> from django.utils.functional import SimpleLazyObject
>
> u = User.objects.select_related('profile').get(id=153)
> o = SimpleLazyObject(lambda :u)
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/usr/local/lib/python2.7/pickle.py", line 1374, in dumps
> Pickler(file, protocol).dump(obj)
> File "/usr/local/lib/python2.7/pickle.py", line 224, in dump
> self.save(obj)
> File "/usr/local/lib/python2.7/pickle.py", line 331, in save
> self.save_reduce(obj=obj, *rv)
> File "/usr/local/lib/python2.7/pickle.py", line 419, in save_reduce
> save(state)
> File "/usr/local/lib/python2.7/pickle.py", line 286, in save
> f(self, obj) # Call unbound method with explicit self
> File "/usr/local/lib/python2.7/pickle.py", line 649, in save_dict
> self._batch_setitems(obj.iteritems())
> File "/usr/local/lib/python2.7/pickle.py", line 661, in _batch_setitems
> for k, v in items:
> RuntimeError: dictionary changed size during iteration
> pickle.dumps(o) # SUCCEEDS
> }}}
>
> This also fails :
> {{{
> u = User.objects.get(id=153)
> o = SimpleLazyObject(lambda :u)
> o.profile
> pickle.dumps(o)
> }}}
> This looks to be related with change in Model.__reduce__ in django 1.8
> adding DJANGO_VERSION_PICKLE_KEY in __dict__
>
> {{{
> def __reduce__(self):
> """
> Provides pickling support. Normally, this just dispatches to
> Python's
> standard handling. However, for models with deferred field
> loading, we
> need to do things manually, as they're dynamically created
> classes and
> only module-level classes can be pickled by the default path.
> """
> data = self.__dict__
> data[DJANGO_VERSION_PICKLE_KEY] = get_version()
> if not self._deferred:
> class_id = self._meta.app_label, self._meta.object_name
> return model_unpickle, (class_id, [], simple_class_factory),
> data
> defers = []
> for field in self._meta.fields:
> if isinstance(self.__class__.__dict__.get(field.attname),
> DeferredAttribute):
> defers.append(field.attname)
> model = self._meta.proxy_for_model
> class_id = model._meta.app_label, model._meta.object_name
> return (model_unpickle, (class_id, defers,
> deferred_class_factory), data)
> }}}
>
> This is quite hard to patch if we just pass
New description:
Reproduction step :
{{{
import pickle
import django
django.setup()
from django.contrib.auth.models import User
from django.utils.functional import SimpleLazyObject
u = User.objects.select_related('profile').get(id=153)
o = SimpleLazyObject(lambda :u)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/pickle.py", line 1374, in dumps
Pickler(file, protocol).dump(obj)
File "/usr/local/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/usr/local/lib/python2.7/pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "/usr/local/lib/python2.7/pickle.py", line 419, in save_reduce
save(state)
File "/usr/local/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/local/lib/python2.7/pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "/usr/local/lib/python2.7/pickle.py", line 661, in _batch_setitems
for k, v in items:
RuntimeError: dictionary changed size during iteration
pickle.dumps(o) # SUCCEEDS
}}}
This also fails :
{{{
u = User.objects.get(id=153)
o = SimpleLazyObject(lambda :u)
o.profile
pickle.dumps(o)
}}}
This looks to be related with change in Model.__reduce__ in django 1.8
adding DJANGO_VERSION_PICKLE_KEY in __dict__
{{{
def __reduce__(self):
"""
Provides pickling support. Normally, this just dispatches to
Python's
standard handling. However, for models with deferred field
loading, we
need to do things manually, as they're dynamically created classes
and
only module-level classes can be pickled by the default path.
"""
data = self.__dict__
data[DJANGO_VERSION_PICKLE_KEY] = get_version()
if not self._deferred:
class_id = self._meta.app_label, self._meta.object_name
return model_unpickle, (class_id, [], simple_class_factory),
data
defers = []
for field in self._meta.fields:
if isinstance(self.__class__.__dict__.get(field.attname),
DeferredAttribute):
defers.append(field.attname)
model = self._meta.proxy_for_model
class_id = model._meta.app_label, model._meta.object_name
return (model_unpickle, (class_id, defers,
deferred_class_factory), data)
}}}
--
--
Ticket URL: <https://code.djangoproject.com/ticket/25426#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/061.86dd89933eb612cdc148327e98fc1db1%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
