[issue44772] Regression in memory use of instances due to dictionary ordering

[Mark Shannon]

Mark Shannon  added the comment:

Duplicate of https://bugs.python.org/issue40116

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44772] Regression in memory use of instances due to dictionary ordering

[Brandt Bucher]

Change by Brandt Bucher :


--
nosy: +brandtbucher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44772] Regression in memory use of instances due to dictionary ordering

[Mark Shannon]

Mark Shannon  added the comment:

Raymond,
When you say "this was mostly a net win" do you mean the more compact layout or 
ordering?

The compact layout is obviously a win, and doesn't conflict with sharing. The 
problem is that ordering conflicts with sharing.

As long as instances all have attributes that are a subset of the shared keys 
stored on the class, they can share attributes, or least they could in 3.5.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44772] Regression in memory use of instances due to dictionary ordering

[Raymond Hettinger]

Raymond Hettinger  added the comment:

ISTM this was mostly a net win.  We get compactness and sharing most of the 
time, and lose sharing only in cases like this where different instances of the 
same class happen to have different attributes.  Personally, I would not have 
expected sharing to occur in those cases.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44772] Regression in memory use of instances due to dictionary ordering

[Mark Shannon]

New submission from Mark Shannon :

class C:
def __init__(self, cond):
if cond:
self.a = 1
self.b = 2

c1 = C(True)
c2 = C(False)

   
In Python 3.5, the dictionary keys are shared
-

>>> sys.getsizeof(c2)
56
>>> sys.getsizeof(c1.__dict__)
96
>>> sys.getsizeof(c2.__dict__)
96

In Python 3.9, the keys are not shared
--

>>> sys.getsizeof(c2)
48
>>> sys.getsizeof(c1.__dict__)
272
>>> sys.getsizeof(c2.__dict__)
232

This represents an increase of memory use for c1 of 110%. (48+272)/(56+96) == 
2.1

With compact object layout (https://github.com/faster-cpython/ideas/issues/69), 
any failure to share keys will result in a tripling of memory use per object.

It is not an uncommon pattern for some attributes to initialized lazily,
which causes also prevents key-sharing.

Not only does it increase memory use, it prevents optimizations that attempt to 
specialize attribute access for instances of a class, as different instances 
will have different layouts.

The purpose of compact dicts was to save memory, but in this case we are using 
a lot more memory by preventing PEP 412 from working.

--
keywords: 3.9regression
messages: 398475
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Regression in memory use of instances due to dictionary ordering

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com