[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Raymond Hettinger added the comment: Based on all the negative comments, I'm closing this one. It adds too much complication in return for dubious value. -- resolution: -> rejected status: open -> closed ___ Python tracker

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Changes by Trent Nelson : -- nosy: +trent ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.or

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Changes by Andrew Svetlov : -- nosy: +asvetlov ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyth

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Changes by Giampaolo Rodola' : -- nosy: +giampaolo.rodola ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Changes by STINNER Victor : -- nosy: +haypo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Antoine Pitrou added the comment: > By the way CPython's list type does more than log(N) resize ops: Obviously it's an asymptotic complexity, not a hard number. > I understand the proposal as a power user tool. Most people don't > need a pneumatic hammer, an ordinary hammer suffices. But in som

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Christian Heimes added the comment: Gregory, thanks. :) By the way CPython's list type does more than log(N) resize ops: $ ./listresize.py 10 100 1000 1 10 10 list.append() do 4 resize ops. 100 list.append() do 11 resize ops. 1000 list.append() do 28 resize ops. 1 list.append() do

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Stefan Behnel added the comment: I agree with basically all counter-arguments that were brought up so far. While I would eventually like to use the available length hints in the special case of Cython's list comprehensions, I'm far from seeing a general applicability for this. The use cases ar

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Gregory P. Smith added the comment: The gain will be more noticeable the faster the Python implementation it is running under is. It is going to avoid logN relloc's in just about all implementations. That CPython is relatively slow is not a justification to avoid adding the feature. I like C

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Charles-François Natali added the comment: > Here is an experimental patch. The speedup is ... measurable. It's likely to be more useful for dict and set, to avoid/limit rehashs. Also, the allocation overhead depends on the implementation, I suspect the gain would be more important with PyPy.

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Christian Heimes added the comment: Here is an experimental patch. The speedup is ... measurable. $ ./python -m timeit -n1000 "l = []" "l.__preallocate__(1)" "app = l.append" "for i in range(1): app(i)" "l.__shrink__()" 1000 loops, best of 3: 3.68 msec per loop $ ./python -m timeit -n1

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Christian Heimes added the comment: My python-ideas posting about the same topic http://mail.python.org/pipermail/python-ideas/2013-March/019807.html -- ___ Python tracker ___ _

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Changes by Christian Heimes : -- nosy: +christian.heimes ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:/

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Serhiy Storchaka added the comment: This unlikely will be useful. C code uses __length_hint__ because C code is fast enough and fill time can be comparable to resize time. But Python code will be one or two order slower and resize time is insignificant fraction of total time for any real code.

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Changes by Brett Cannon : -- nosy: +brett.cannon ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Alex Gaynor added the comment: That strategy only works if you know the exact count, it fails if you only have an estimate (as __length_hint__ gives you). -- ___ Python tracker

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Antoine Pitrou added the comment: > There's an obvious need for this, CPython uses optimizations like this > internally, as does PyPy. I don't know if it's a need or if it's just "nice to have". By the way, in the list case it also makes C code simpler, since once the list is preallocated you ju

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Benjamin Peterson added the comment: I think the emphasis is on "internally" :). -- ___ Python tracker ___ ___ Python-bugs-list mailin

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Alex Gaynor added the comment: python-dev/ideas may be a better place to have this discussion, but basically if you're going to insist that stuff like this doesn't go into Python "because it isn't C++", people are going to have to write C++, and that makes me incredibly sad. There's an obviou

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Antoine Pitrou added the comment: I think it is going a bit too far in the direction of exposing low-level optimizations. Python is not C++. I also agree with Raymond that it's rather rare to construct read-only containers (in the sense that they are allocated once and for all). -- no

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Raymond Hettinger added the comment: Guido has previously rejected any kind of user specified presizing parameter for dictionaries. He felt that it was breaking the abstraction in a way that caused people to focus on implementation specific details. Also, set() and dict() use presizing (and h

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

Benjamin Peterson added the comment: Definitely needs to be discussed on python-ideas/python-dev. -- nosy: +benjamin.peterson ___ Python tracker ___ _

[issue17338] Add length_hint parameter to list, dict, set constructors to allow efficient presizing

New submission from Alex Gaynor: Following the length_hint PEP, we should expose this facility to end-python programmers. The semantics would be basically: the list has behavior identical to if you hadn't provided length_hint, except the VM is free to preallocate efficiently. CPython (and PyP