On Aug 1, 2012, at 1:46 AM, Mark Shannon wrote:
'''
Being able to pre-allocate lists based on the expected size, as estimated by
__length_hint__,
can be a significant optimization.
PyPy has been observed to run some code slower than CPython, purely because
this optimization is absent.
While the idea behind PEP 424 is sound, the text of the PEP is rather vague and
missing a lot of details.
There was extended discussion on the details, but none of that has appeared in
the PEP yet.
So Alex, how about adding those details?
Also the rationale is rather poor.
Given that CPython
On Wed, Aug 1, 2012 at 10:46 AM, Mark Shannon m...@hotpy.org wrote:
While the idea behind PEP 424 is sound, the text of the PEP is rather vague
and missing a lot of details.
There was extended discussion on the details, but none of that has appeared
in the PEP yet.
So Alex, how about adding
Maciej Fijalkowski wrote:
On Wed, Aug 1, 2012 at 10:46 AM, Mark Shannon m...@hotpy.org wrote:
While the idea behind PEP 424 is sound, the text of the PEP is rather vague
and missing a lot of details.
There was extended discussion on the details, but none of that has appeared
in the PEP yet.
So
On Wed, Aug 1, 2012 at 12:06 PM, Mark Shannon m...@hotpy.org wrote:
Maciej Fijalkowski wrote:
On Wed, Aug 1, 2012 at 10:46 AM, Mark Shannon m...@hotpy.org wrote:
While the idea behind PEP 424 is sound, the text of the PEP is rather
vague
and missing a lot of details.
There was extended
On Sun, Jul 15, 2012 at 1:36 PM, Antoine Pitrou solip...@pitrou.net wrote:
On Sun, 15 Jul 2012 18:47:38 +1000
Nick Coghlan ncogh...@gmail.com wrote:
I'm not seeing the value in returning None over 0 for the don't know
case - it just makes the API harder to use.
The point is that 0 is a
On Tue, 17 Jul 2012 13:19:55 +1000
Nick Coghlan ncogh...@gmail.com wrote:
There are no provisions for infinite iterators, that is not within the
scope of
this proposal.
I'll repeat my observation that remaining silent on this point is
effectively identical to blessing the practice of
On Sun, Jul 15, 2012 at 1:28 AM, Alexandre Zani
alexandre.z...@gmail.com wrote:
I'm +1 on not having a public API for this. Ultimately the contract
for a length hint will depend heavily upon what you need it for. Some
applications would require a length hint to be an at least others an
at most
Mark Shannon, 15.07.2012 16:14:
Alex Gaynor wrote:
CPython currently defines an ``__length_hint__`` method on several types,
such
as various iterators. This method is then used by various other functions
(such as ``map``) to presize lists based on the estimated returned by
Don't use map as
Antoine Pitrou, 15.07.2012 17:06:
On Sun, 15 Jul 2012 16:33:23 +0200
Christian Heimes wrote:
Am 15.07.2012 16:22, schrieb Antoine Pitrou:
On Mon, 16 Jul 2012 00:08:41 +1000
Nick Coghlan wrote:
Right, I agree on the value in being able to return something to say this
cannot be converted to a
Alex Gaynor, 15.07.2012 00:11:
CPython currently defines an ``__length_hint__`` method on several types, such
as various iterators. This method is then used by various other functions
(such
as ``map``) to presize lists based on the estimated returned by
``__length_hint__``. Types can then
Proposing anything substantially more complicated than what is currently
implemented in CPython will just get the idea rejected. The substantial
immediate gain for PyPy is in skipping the memory resizing when building
containers from itertools iterators, not anywhere else.
--
Sent from my phone,
Nick Coghlan, 16.07.2012 10:26:
Proposing anything substantially more complicated than what is currently
implemented in CPython will just get the idea rejected. The substantial
immediate gain for PyPy is in skipping the memory resizing when building
containers from itertools iterators, not
On Jul 15, 2012, at 7:22 AM, Antoine Pitrou wrote:
On Mon, 16 Jul 2012 00:08:41 +1000
Nick Coghlan ncogh...@gmail.com wrote:
Right, I agree on the value in being able to return something to say this
cannot be converted to a concrete container.
Who would be able to return that, apart from
On 16.07.12 10:36, Stefan Behnel wrote:
Return values could be -1 for don't know and -2 for infinite at the C
level, and NotImplemented for don't know at the Python level.
PY_SSIZE_T_MAX is better value for infinite. In any case no difference
for consumer between PY_SSIZE_T_MAX and a real
On Jul 16, 2012, at 12:36 AM, Stefan Behnel wrote:
I'd like to more visibly repeat my suggestion to make this a slot method
tp_length_hint() in extension types that returns a Py_ssize_t.
That is merely an implementation detail, but it would be a nice improvement.
Raymond
On Mon, Jul 16, 2012 at 3:36 AM, Stefan Behnel stefan...@behnel.de wrote:
Alex Gaynor, 15.07.2012 00:11:
CPython currently defines an ``__length_hint__`` method on several
types, such
as various iterators. This method is then used by various other
functions (such
as ``map``) to presize
On 7/16/2012 9:54 AM, Stefan Behnel wrote:
Mark Shannon, 15.07.2012 16:14:
Alex Gaynor wrote:
CPython currently defines an ``__length_hint__`` method on several types,
such
as various iterators. This method is then used by various other functions
(such as ``map``) to presize lists based on the
M Stefan wrote:
Also, what would iter([1,2,3]).__length_hint__() return? 3 or unknown?
If 3, then the semantics of l=[1,2,3]; l += iter(l) will change
(infinite loop without __length_hint__ vs. list of 6 elements with __length_hint__).
What __length_hint__ returns is irrelevant -- it's only a
I've updated the PEP to reflect the discussion. There are two major changes:
1) NotImplemented may be used by __length_hint__ to indicate that there is no
finite length hint available.
2) callers of operator.length_hint() must provide their own default value, this
is also required by the
On Tue, Jul 17, 2012 at 1:03 PM, Alex Gaynor alex.gay...@gmail.com wrote:
I've updated the PEP to reflect the discussion. There are two major changes:
1) NotImplemented may be used by __length_hint__ to indicate that there is no
finite length hint available.
I've been thinking about this a
Nick Coghlan wrote:
On Sun, Jul 15, 2012 at 9:18 AM, Benjamin Peterson benja...@python.org wrote:
Open questions
==
There are two open questions for this PEP:
* Should ``list`` expose a kwarg in it's constructor for supplying a length
hint.
* Should a function be added either to
On Sun, Jul 15, 2012 at 6:21 PM, Steven D'Aprano st...@pearwood.info wrote:
I suggest:
* object (and hence all other types that don't explicitly override it)
should have a __length_hint__ that raises TypeError;
We can keep it simpler than that just by changing the order of the checks.
*
Alex Gaynor, 15.07.2012 07:20:
there's no way for the __lenght_hint__ to specify that
that particular instance can't have a length hint computed. e.g. imagine
some sort of lazy stream that cached itself, and only wanted to offer a
length hint if it had already been evaluated. Without an
On Sun, 15 Jul 2012 18:47:38 +1000
Nick Coghlan ncogh...@gmail.com wrote:
* __length_hint__ should be allowed to return None to indicate don't know
or -1 to indicate infinite.
Presumably anything that wishes to create a list or other sequence from an
object with a hint of -1 could
Antoine Pitrou wrote:
The point is that 0 is a legitimate value for a length hint. Simple
implementations of __length_hint__ will start returning 0 as a
legitimate value and you will wrongly interpret that as don't know,
which kinds of defeat the purpose of __length-hint__ ;)
That said, I
Right, I agree on the value in being able to return something to say this
cannot be converted to a concrete container.
I still haven't seen a use case where the appropriate response to I don't
know differs from the appropriate response to a hint of zero - that is,
you don't preallocate, you just
On Mon, 16 Jul 2012 00:08:41 +1000
Nick Coghlan ncogh...@gmail.com wrote:
Right, I agree on the value in being able to return something to say this
cannot be converted to a concrete container.
Who would be able to return that, apart from trivial cases like
itertools.cycle()?
Regards
Antoine.
Am 15.07.2012 16:22, schrieb Antoine Pitrou:
On Mon, 16 Jul 2012 00:08:41 +1000
Nick Coghlan ncogh...@gmail.com wrote:
Right, I agree on the value in being able to return something to say this
cannot be converted to a concrete container.
Who would be able to return that, apart from trivial
Nick Coghlan wrote:
Right, I agree on the value in being able to return something to say
this cannot be converted to a concrete container.
I still haven't seen a use case where the appropriate response to I
don't know differs from the appropriate response to a hint of zero -
that is, you
Alex Gaynor wrote:
Hi all,
I've just submitted a PEP proposing making __length_hint__ a public API for
users to define and other VMs to implement:
These seems back-to-front.
__length_hint__ is *used* by the VM, not provided by it.
It should be part of the object model, rather than the API.
On Sun, Jul 15, 2012 at 10:39 AM, Mark Shannon m...@hotpy.org wrote:
Nick Coghlan wrote:
Right, I agree on the value in being able to return something to say
this cannot be converted to a concrete container.
I still haven't seen a use case where the appropriate response to I
don't know
Am 15.07.2012 16:39, schrieb Mark Shannon:
1. Don't implement it at all.
2. Implement __length_hint__() but don't want to return any value.
Either raise an exception (TypeError) -- As suggested in the PEP.
or return NotImplemented -- my preferred option.
How is this different from
Brett Cannon wrote:
On Sun, Jul 15, 2012 at 10:39 AM, Mark Shannon m...@hotpy.org
mailto:m...@hotpy.org wrote:
Nick Coghlan wrote:
Right, I agree on the value in being able to return something to
say this cannot be converted to a concrete container.
I still
On Sun, 15 Jul 2012 16:33:23 +0200
Christian Heimes li...@cheimes.de wrote:
Am 15.07.2012 16:22, schrieb Antoine Pitrou:
On Mon, 16 Jul 2012 00:08:41 +1000
Nick Coghlan ncogh...@gmail.com wrote:
Right, I agree on the value in being able to return something to say this
cannot be converted
On Sun, 15 Jul 2012 16:08:00 +0100
Mark Shannon m...@hotpy.org wrote:
What should happen if I am silly enough to do this:
list(itertools.count())
This will fail; it should fail quickly.
Why should it? AFAIK it's not a common complaint. You said it yourself:
it's a silly thing to do.
On Sun, Jul 15, 2012 at 8:08 AM, Mark Shannon m...@hotpy.org wrote:
Brett Cannon wrote:
On Sun, Jul 15, 2012 at 10:39 AM, Mark Shannon m...@hotpy.org
mailto:m...@hotpy.org wrote:
Nick Coghlan wrote:
Right, I agree on the value in being able to return something to
say
Mark Shannon wrote:
So how does an iterator express infinite length?
The suggestion was it should return -1.
What should happen if I am silly enough to do this:
list(itertools.count())
This will fail; it should fail quickly.
That depends on your OS. I've just tested it now on Linux
On Mon, Jul 16, 2012 at 1:55 AM, Steven D'Aprano st...@pearwood.info wrote:
(I expect the difference in behaviour is due to the default ulimit under
Debian/Mint and RedHat/Fedora systems.)
Possibly also virtual memory settings. Allocating gobs of memory with
a huge page file slows everything
Antoine Pitrou wrote:
First, you can't implement __length_hint__ for a generator, which is the
preferred (the most practical) way of writing iterators in pure Python.
Limitations of generators are no reason for not improving iterators which are
not generators. __length_hint__ already exists;
On Mon, 16 Jul 2012 02:00:58 +1000
Chris Angelico ros...@gmail.com wrote:
On Mon, Jul 16, 2012 at 1:55 AM, Steven D'Aprano st...@pearwood.info wrote:
(I expect the difference in behaviour is due to the default ulimit under
Debian/Mint and RedHat/Fedora systems.)
Possibly also virtual
On Mon, 16 Jul 2012 02:21:20 +1000
Steven D'Aprano st...@pearwood.info wrote:
My conclusion is that an infinite iterator is a documentation issue.
Just tell the user that it doesn't stop, and let them shoot themselves
in the foot in they want to.
Buffer overflows are a documentation
Steven D'Aprano wrote:
With a length hint, we could strengthen that promise:
if __length_hint__ returns a negative number, list, tuple and set will
fail immediately with MemoryError
which I think is a good safety feature for some things which cannot
possibly succeed, but risk DOSing your
Nick Coghlan writes:
Right, I agree on the value in being able to return something to say this
cannot be converted to a concrete container.
I still haven't seen a use case where the appropriate response to I don't
know differs from the appropriate response to a hint of zero - that is,
On Jul 16, 2012 1:52 PM, Stephen J. Turnbull step...@xemacs.org wrote:
The point is that I don't really see the value in returning a precise
estimate that cannot be relied on to be accurate. OK, Python is a
consenting adults language, but returning an integer here seems like
invitation to
Stephen J. Turnbull wrote:
The point is that I don't really see the value in returning a precise
estimate that cannot be relied on to be accurate. OK, Python is a
consenting adults language, but returning an integer here seems like
invitation to abuse.
Since __length_hint__ already exists
Hi all,
I've just submitted a PEP proposing making __length_hint__ a public API for
users to define and other VMs to implement:
PEP: 424
Title: A method for exposing a length hint
Version: $Revision$
Last-Modified: $Date
Author: Alex Gaynor alex.gay...@gmail.com
Status: Draft
Type: Standards
2012/7/14 Alex Gaynor alex.gay...@gmail.com:
Proposal
This PEP proposes formally documenting ``__length_hint__`` for other
interpreter and non-standard library Python to implement.
``__length_hint__`` must return an integer, and is not required to be
accurate.
It may return a
On Sat, Jul 14, 2012 at 4:18 PM, Benjamin Peterson benja...@python.orgwrote:
2012/7/14 Alex Gaynor alex.gay...@gmail.com:
Proposal
This PEP proposes formally documenting ``__length_hint__`` for other
interpreter and non-standard library Python to implement.
On Sat, Jul 14, 2012 at 4:21 PM, Alex Gaynor alex.gay...@gmail.com wrote:
On Sat, Jul 14, 2012 at 4:18 PM, Benjamin Peterson benja...@python.org
wrote:
2012/7/14 Alex Gaynor alex.gay...@gmail.com:
Proposal
This PEP proposes formally documenting ``__length_hint__`` for
2012/7/14 Alex Gaynor alex.gay...@gmail.com:
On Sat, Jul 14, 2012 at 4:18 PM, Benjamin Peterson benja...@python.org
wrote:
2012/7/14 Alex Gaynor alex.gay...@gmail.com:
Proposal
This PEP proposes formally documenting ``__length_hint__`` for other
interpreter and
On 7/14/2012 6:11 PM, Alex Gaynor wrote:
...
Various thoughts:
This method is then used by various other functions (such +as ``map``)
to presize lists
-- map no longer produces lists. This only makes sense in 3.x if you
mean that map can pass along the value of its inputs.
Types can then
On Sun, Jul 15, 2012 at 9:18 AM, Benjamin Peterson benja...@python.org wrote:
Open questions
==
There are two open questions for this PEP:
* Should ``list`` expose a kwarg in it's constructor for supplying a length
hint.
* Should a function be added either to ``builtins`` or
On Sat, Jul 14, 2012 at 10:16 PM, Nick Coghlan ncogh...@gmail.com wrote:
On Sun, Jul 15, 2012 at 9:18 AM, Benjamin Peterson benja...@python.org
wrote:
Open questions
==
There are two open questions for this PEP:
* Should ``list`` expose a kwarg in it's constructor for
54 matches
Mail list logo
