[issue23601] use small object allocator for dict key storage

Raymond Hettinger added the comment: Thanks for the patch and for your patience. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue23601] use small object allocator for dict key storage

STINNER Victor added the comment: Nice change. I opened the issue #26249 to continue the investigation. -- nosy: +haypo ___ Python tracker ___

[issue23601] use small object allocator for dict key storage

Roundup Robot added the comment: New changeset 425fec4a79c7 by Raymond Hettinger in branch 'default': Issue #23601: Use small object allocator for dict key objects https://hg.python.org/cpython/rev/425fec4a79c7 -- nosy: +python-dev ___ Python

[issue23601] use small object allocator for dict key storage

Tim Peters added the comment: +1 from me. Julian, you have the patience of a saint ;-) -- ___ Python tracker ___

[issue23601] use small object allocator for dict key storage

Raymond Hettinger added the comment: If there are no objects, I'll apply the patch tomorrow. -- assignee: -> rhettinger versions: +Python 3.6 -Python 3.5 ___ Python tracker

[issue23601] use small object allocator for dict key storage

Julian Taylor added the comment: ping, this has been sitting for 4 years and two python releases. Its about time this stupidly simple thing gets merged. -- ___ Python tracker

[issue23601] use small object allocator for dict key storage

Stefan Behnel added the comment: Benchmark results look good to me (although a header line is missing) and seem to match my expectations. Sounds like we should allow this change. -- ___ Python tracker rep...@bugs.python.org

[issue23601] use small object allocator for dict key storage

Mark Shannon added the comment: +1 from me. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23601 ___ ___ Python-bugs-list mailing list

[issue23601] use small object allocator for dict key storage

Mark Shannon added the comment: Yes, but that shouldn't block this issue. I've opened issue 24648 instead. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23601 ___

[issue23601] use small object allocator for dict key storage

Serhiy Storchaka added the comment: If use small object allocator for dict key storage, why not use it for dict value storage? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23601 ___

[issue23601] use small object allocator for dict key storage

Julian Taylor added the comment: ok I ran it again, but note the machine was under use the full time so the results are likely have no meaning. python perf.py -r -b default /tmp/normal/bin/python3 /tmp/opt/bin/python3 Min: 0.399279 - 0.376527: 1.06x faster Avg: 0.410819 - 0.383315: 1.07x

[issue23601] use small object allocator for dict key storage

Julian Taylor added the comment: Your benchmarks are not affected by this change see the other issue. They are also not representative of every workload out there. I can at least see the argument why you didn't want to put the other variant of this change in as it made the code a tiny bit

[issue23601] use small object allocator for dict key storage

Stefan Behnel added the comment: Your benchmarks are not affected by this change see the other issue. Then you ran them, I assume? Could you show the output here that you got? They are also not representative of every workload out there. Certainly not, but they do provide a certain variety

[issue23601] use small object allocator for dict key storage

Stefan Behnel added the comment: It's generally worth running the benchmark suite for this kind of optimisation. Being mostly Python code, it should benefit quite clearly from dictionary improvements, but it should also give an idea of how much of an improvement actual Python code (and not

[issue23601] use small object allocator for dict key storage

Julian Taylor added the comment: Large objects are just if size 512: return malloc(size) there is no reason it should be slower. Also for large objects allocation speed does not matter as much. -- ___ Python tracker rep...@bugs.python.org

[issue23601] use small object allocator for dict key storage

Mark Shannon added the comment: I still think that this is a good idea, but I would like to see a small speed test for large objects. Just to be sure that it is no slower. -- ___ Python tracker rep...@bugs.python.org

[issue23601] use small object allocator for dict key storage

Mark Shannon added the comment: Indeed there is no *obvious* reason why they should be slower. But when it comes to optimisation, *never* trust your (or anyone else's) intuition. Running a simple check is always worth the effort. -- ___ Python

[issue23601] use small object allocator for dict key storage

Serhiy Storchaka added the comment: See also issue16465. -- nosy: +pitrou, rhettinger, serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23601 ___

[issue23601] use small object allocator for dict key storage

Antoine Pitrou added the comment: Interesting. I can reproduce the speedup on 64-bit Linux (Ubuntu 14.10). -- stage: - patch review type: - performance ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23601

[issue23601] use small object allocator for dict key storage

Serhiy Storchaka added the comment: $ ./python -m timeit dict(a=5, b=2) Unpatched: 10 loops, best of 3: 2.5 usec per loop issue16465 patch: 100 loops, best of 3: 1.87 usec per loop issue23601 patch: 100 loops, best of 3: 1.98 usec per loop --

[issue23601] use small object allocator for dict key storage

Mark Shannon added the comment: I don't remember why PyMem_Malloc rather than PyObject_MALLOC was used, it may have been inherited from the allocation of dict tables in the earlier implementation. My only concern is that the benchmark only tests performance for very small dictionaries. The

[issue23601] use small object allocator for dict key storage

Marc-Andre Lemburg added the comment: There seem to be quite a few other places where this simple optimization could make sense as well, perhaps even going as far as converting all uses of PyMem_MALLOC to PyObject_MALLOC. There was a time when the small memory allocator did not return free

[issue23601] use small object allocator for dict key storage

New submission from Julian Taylor: dictionary creation spends a not insignificant amount of time in malloc allocating keys objects. Python has a nice small object allocator that avoids a lot of this overhead and falls back to malloc for larger allocations. Is there a reason the dictionary does

[issue23601] use small object allocator for dict key storage

Changes by Mark Shannon m...@hotpy.org: -- nosy: +Mark.Shannon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23601 ___ ___ Python-bugs-list

[issue23601] use small object allocator for dict key storage

Changes by SilentGhost ghost@gmail.com: -- nosy: +lemburg, tim.peters ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23601 ___ ___

[issue23601] use small object allocator for dict key storage

Julian Taylor added the comment: PyObject_Malloc just calls malloc above the threshold so there is no problem for larger dicts. For larger dicts the performance of malloc is also irrelevant as the time will be spent elsewhere. -- ___ Python tracker