On Tue, Jul 17, 2012 at 2:57 PM, John O'Connor wrote:
>>
>> The second approach is consistently 10-20% faster than the first one
>> (depending on input) for trunk Python 3.3
>>
>
> I think the difference is that StringIO spends extra time reallocating
> memory during the write loop as it grows, wh
>
> The second approach is consistently 10-20% faster than the first one
> (depending on input) for trunk Python 3.3
>
I think the difference is that StringIO spends extra time reallocating
memory during the write loop as it grows, whereas bytes.join computes
the allocation size first since it alr
While working on #1767933, Serhiy came up with an observation that
"monkey-patching" one of the base classes of io is faster than using
BytesIO when in need of a file-like object for writing into.
I've distilled it into this standalone test:
import io
data = [b'a'*10, b'bb'*5, b'ccc'*5] * 1
On Tue, Jul 17, 2012 at 1:03 PM, Alex Gaynor wrote:
> I've updated the PEP to reflect the discussion. There are two major changes:
>
> 1) NotImplemented may be used by __length_hint__ to indicate that there is no
>finite length hint available.
I've been thinking about this a bit more, and I t
I've updated the PEP to reflect the discussion. There are two major changes:
1) NotImplemented may be used by __length_hint__ to indicate that there is no
finite length hint available.
2) callers of operator.length_hint() must provide their own default value, this
is also required by the cu
M Stefan wrote:
Also, what would iter([1,2,3]).__length_hint__() return? 3 or unknown?
If 3, then the semantics of l=[1,2,3]; l += iter(l) will change
(infinite loop without __length_hint__ vs. list of 6 elements with __length_hint__).
What __length_hint__ returns is irrelevant -- it's only a
On 7/16/2012 9:54 AM, Stefan Behnel wrote:
Mark Shannon, 15.07.2012 16:14:
Alex Gaynor wrote:
CPython currently defines an ``__length_hint__`` method on several types,
such
as various iterators. This method is then used by various other functions
(such as ``map``) to presize lists based on the
On Tue, 17 Jul 2012 00:18:55 +1000
Nick Coghlan wrote:
>
> Given that all standard library containers default to assuming a size
> of zero (absent information indicating otherwise), and a special value
> would need to be special cased by *every* client of the API (and
> almost always defaulted to
On Tue, Jul 17, 2012 at 12:01 AM, Antoine Pitrou wrote:
> On Mon, 16 Jul 2012 23:23:18 +1000
> Nick Coghlan wrote:
>>
>> - distinguishing between different reasons for saying "don't
>> preallocate any space" (i.e. returning zero). I still haven't heard a
>> convincing rationale for this one
>
> T
On Mon, 16 Jul 2012 23:23:18 +1000
Nick Coghlan wrote:
>
> - distinguishing between different reasons for saying "don't
> preallocate any space" (i.e. returning zero). I still haven't heard a
> convincing rationale for this one
The point is that zero is a valid value for a length hint. By making
On Mon, Jul 16, 2012 at 7:21 PM, Tim Golden wrote:
>> Speaking of which - I find this bikeshed disgusting.
>
> Disgusting? Irritating, perhaps, but why should a PEP -- even one whose
> purpose is to codify existing practice -- not result in discussions
> about its subject matter?
>
> The final P s
On Mon, Jul 16, 2012 at 3:36 AM, Stefan Behnel wrote:
> Alex Gaynor, 15.07.2012 00:11:
> > CPython currently defines an ``__length_hint__`` method on several
> types, such
> > as various iterators. This method is then used by various other
> functions (such
> > as ``map``) to presize lists based
> *If* resizing list is so slow, then why not make it faster?
A simple solution to speed up such problem is to change the
overallocation factor, but it may waste memory. Python tries to be
fast and to not waste too much memory.
> Why is it a significant optimisation?
> How much slower is it?
> Wh
Looking better at the code, the fileobject change isn't necessary. A simpler
fix is to just ignore and clear errors from _Py_DisplaySourceLine.
I´ll prepare a defect/patch
K
Frá: python-dev-bounces+kristjan=ccpgames@python.org
[python-dev-bounces+kristja
On Jul 16, 2012, at 12:36 AM, Stefan Behnel wrote:
> I'd like to more visibly repeat my suggestion to make this a slot method
> "tp_length_hint()" in extension types that returns a Py_ssize_t.
That is merely an implementation detail, but it would be a nice improvement.
Raymond
___
Hi there.
I've been busy taking the current beta candidate and merging it into the
stackless repo.
As expected, things don't just go smoothly and there are the usual startup
errors, this being a rather intrusive patch and all that.
However, I found that early startup errors were not being re
On 16.07.12 10:36, Stefan Behnel wrote:
Return values could be -1 for "don't know" and -2 for "infinite" at the C
level, and NotImplemented for "don't know" at the Python level.
PY_SSIZE_T_MAX is better value for "infinite". In any case no difference
for consumer between PY_SSIZE_T_MAX and a r
On Jul 15, 2012, at 7:22 AM, Antoine Pitrou wrote:
> On Mon, 16 Jul 2012 00:08:41 +1000
> Nick Coghlan wrote:
>> Right, I agree on the value in being able to return something to say "this
>> cannot be converted to a concrete container".
>
> Who would be able to return that, apart from trivial c
Maciej Fijalkowski, 16.07.2012 11:14:
> On Mon, Jul 16, 2012 at 11:02 AM, Stefan Behnel wrote:
>> Mark Shannon, 16.07.2012 10:37:
>>> If resizing of lists is too slow, then we should reconsider the 9/8
>>> factor and/or look to tweak the resizing code.
>>
>> The thing is that the performance is pla
> Speaking of which - I find this bikeshed disgusting.
Disgusting? Irritating, perhaps, but why should a PEP -- even one whose
purpose is to codify existing practice -- not result in discussions
about its subject matter?
The final P stands for Proposal, not for Pronouncement.
TJG
__
On Mon, Jul 16, 2012 at 11:02 AM, Stefan Behnel wrote:
> Mark Shannon, 16.07.2012 10:37:
> > If resizing of lists is too slow, then we should reconsider the 9/8
> factor
> > and/or look to tweak the resizing code.
>
> The thing is that the performance is platform specific. On systems with a
> fas
On Jul 16, 2012, at 1:37 AM, Mark Shannon wrote:
> To quote from PEP 424:
>
>> Rationale
>> =
>> Being able to pre-allocate lists based on the expected size, as estimated by
>> ``__length_hint__``, can be a significant optimization. CPython has been
>> observed to run some code faster t
Mark Shannon, 16.07.2012 10:37:
> If resizing of lists is too slow, then we should reconsider the 9/8 factor
> and/or look to tweak the resizing code.
The thing is that the performance is platform specific. On systems with a
fast memory allocator, especially on Linux, the difference is negligible.
Nick Coghlan, 16.07.2012 10:26:
> Proposing anything substantially more complicated than what is currently
> implemented in CPython will just get the idea rejected. The substantial
> immediate gain for PyPy is in skipping the memory resizing when building
> containers from itertools iterators, not
To quote from PEP 424:
Rationale
=
Being able to pre-allocate lists based on the expected size, as estimated by
``__length_hint__``, can be a significant optimization. CPython has been
observed to run some code faster than PyPy, purely because of this optimization
being present.
Wh
Proposing anything substantially more complicated than what is currently
implemented in CPython will just get the idea rejected. The substantial
immediate gain for PyPy is in skipping the memory resizing when building
containers from itertools iterators, not anywhere else.
--
Sent from my phone, t
Alex Gaynor, 15.07.2012 00:11:
> CPython currently defines an ``__length_hint__`` method on several types, such
> as various iterators. This method is then used by various other functions
> (such
> as ``map``) to presize lists based on the estimated returned by
> ``__length_hint__``. Types can th
Antoine Pitrou, 15.07.2012 17:06:
> On Sun, 15 Jul 2012 16:33:23 +0200
> Christian Heimes wrote:
>> Am 15.07.2012 16:22, schrieb Antoine Pitrou:
>>> On Mon, 16 Jul 2012 00:08:41 +1000
>>> Nick Coghlan wrote:
Right, I agree on the value in being able to return something to say "this
cannot
28 matches
Mail list logo
