[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Change by Pierre Glaser :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

New submission from Pierre Glaser :

Hello all,
This 16-year old commit (*) allows an object's state to be updated
using its slots instead of its __dict__ at unpickling time. To use this
functionality, the state keyword-argument of Pickler.save_reduce (which maps to
the third item of the tuple returned by __reduce__) should be a length-2 tuple.
As far as I can tell, this is not mentioned in the documentation (**). I 
suggest having the docs updated. What do you think?

(*) 
https://github.com/python/cpython/commit/ac5b5d2e8b849c499d323b0263ace22e56b4f0d9
(**) https://docs.python.org/3.8/library/pickle.html#object.__reduce__

--
assignee: docs@python
components: Documentation
messages: 335031
nosy: alexandre.vassalotti, docs@python, pierreglaser, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: python doc does not say that the state kwarg in Pickler.save_reduce can 
be a tuple (and not only a dict)
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Does it still work?  With both the C and Python pickler?
Can you post an example?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Pierre Glaser  added the comment:

It turns out that both pickle and _pickle implement this feature, but the 
behavior is inconsistent.

- As a reminder, instances of slotted classes do not have a dict attribute (1)
- On the other side, when pickling slotted class instances, __getstate__ can 
return a tuple of 2 dicts. The first dict represents the __dict__ attribute. 
Because of (1), this first dict should simply be a sentinel value. In pickle, 
the condition is that it evaluates to False, but in _pickle, it should be 
explicitly None.

(- Finally, The second dict in state contains the slotted attribute. )

Here are the lines in the two files causing the inconsistent behavior:
https://github.com/python/cpython/blob/master/Modules/_pickle.c#L6236
https://github.com/python/cpython/blob/master/Lib/pickle.py#L1549

I included an example that illustrates it.

--
Added file: https://bugs.python.org/file48111/test_slots.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Change by Pierre Glaser :


Added file: https://bugs.python.org/file48112/test_slots.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Change by Pierre Glaser :


Removed file: https://bugs.python.org/file48111/test_slots.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Change by Pierre Glaser :


Removed file: https://bugs.python.org/file48112/test_slots.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Change by Pierre Glaser :


Added file: https://bugs.python.org/file48113/test_slots.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Antoine Pitrou]

Antoine Pitrou  added the comment:

You can have both a dict and slots by subclassing:

>>> class A: 
...: __slots__ = ('x',) 
...:

   
>>> class B(A): pass
>>> 
>>>
>>> 
>>> 
>>>
>>> b = B() 
>>> 
>>>
>>> b.x = 5 
>>> 
>>>
>>> b.y = 6 
>>> 
>>>
>>> b.__dict__  
>>> 
>>>
{'y': 6}
>>> A.x 
>>> 
>>>

>>> A.x.__get__(b)  
>>> 
>>>
5

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Interestingly, you can also put an instance dict in slots:

>>> class A:
__slots__ = ['x', '__dict__']

>>> a = A()
>>> a.x = 5
>>> a.y = 6
>>> a.__dict__
{'y': 6}
>>> a.x
5

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Pierre Glaser  added the comment:

Thanks Antoine and Raymond for the feedback.

Indeed, a subclass of a slotted class can have a dict: I enriched the script, 
pickling_depickling instances of such subclasses, with the length-2 tuple 
__getstate__ method, and made sure their attributes were properly retrieved.

Apart from the different checks on state carried out in the c load_build and 
the python load_build, AFAICT, it seems like this feature works :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Change by Pierre Glaser :


Removed file: https://bugs.python.org/file48113/test_slots.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Change by Pierre Glaser :


Added file: https://bugs.python.org/file48114/test_slots.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Change by Pierre Glaser :


--
keywords: +patch
pull_requests: +11981
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

A slotted class will have a dict also when it inherits it from a non-slotted 
class. This is why the base class of slotted class should have slots if you do 
not want an instance dict.

__getstate__ and __setstate__ for slotted classes are described in PEP 307. 
Unfortunately this was not copied to the module documentation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Pierre Glaser  added the comment:

I added a PR with a small patch to document this behavior and reconcile 
_pickle.c and pickle.py

Some explanations on why I am pushing this forward:
 
Pickling instances of classes/subclasses with slots is done natively for pickle 
protocol >= 2. Mentioning this behavior in the docs should *not* make the user 
worry about implementing custom __getstate__ methods just to preserve slots.

Here is the reason why I think this functionality (allowing state and 
slotstate) is worth documenting: pickle gives us a lot of flexibility for 
reducing thanks to the dispatch_table. But unpickling is rather rigid: if no 
__setstate__ exists, we have to deal with the default state updating procedure 
present in load_build. Might as well document all of it ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

See also issue26579 which propose to add a function or method for standard 
implementation of __getstate__ and use it consistently in custom __getstate__ 
implementations. I am not sure about API yet.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com