Redirecting to python-ideas, so trimming less than I might.
Chris Barker writes:
On Mon, Aug 11, 2014 at 11:07 PM, Stephen J. Turnbull step...@xemacs.org
wrote:
I'm referring to removing the unnecessary information that there's a
better way to do it, and simply raising an error (as
On 12 Aug 2014, at 10:02, Armin Rigo ar...@tunes.org wrote:
Hi all,
The core of the matter is that if we repeatedly __add__ strings from a
long list, we get O(n**2) behavior. For one point of view, the
reason is that the additions proceed in left-to-right order. Indeed,
sum() could
On Tue, Aug 12, 2014 at 11:21 PM, Stephen J. Turnbull step...@xemacs.org
wrote:
Redirecting to python-ideas, so trimming less than I might.
reasonable enough -- you are introducing some more significant ideas for
changes.
I've said all I have to say about this -- I don't seem to see anything
Ethan Furman writes:
On 08/11/2014 08:50 PM, Stephen J. Turnbull wrote:
Chris Barker - NOAA Federal writes:
It seems pretty pedantic to say: we could make this work well,
but we'd rather chide you for not knowing the proper way to do
it.
Nobody disagrees. But backward
On 12 Aug 2014 11:21, Chris Barker - NOAA Federal chris.bar...@noaa.gov
wrote:
Sorry for the bike shedding here, but:
The quadratic behaviour of repeated str summation is a subtle, silent
error.
OK, fair enough. I suppose it would be hard and ugly to catch those
instances and raise an
Hi all,
The core of the matter is that if we repeatedly __add__ strings from a
long list, we get O(n**2) behavior. For one point of view, the
reason is that the additions proceed in left-to-right order. Indeed,
sum() could proceed in a more balanced tree-like order: from [x0, x1,
x2, x3, ...],
On Mon, Aug 11, 2014 at 11:07 PM, Stephen J. Turnbull step...@xemacs.org
wrote:
I'm referring to removing the unnecessary information that there's a
better way to do it, and simply raising an error (as in Python 3.2,
say) which is all a RealProgrammer[tm] should ever need!
I can't imagine
chris.bar...@noaa.gov
An:Kein Empfnger
Cc:Python Dev python-dev@python.org
Betreff:Re: [Python-Dev] sum(...) limitation
On Mon, Aug 11, 2014 at 11:07 PM, Stephen J. Turnbull step...@xemacs.org wrote:
Im referring to removing the unnecessary information that theres a
better way to do it, and simply
Chris Barker chris.bar...@noaa.gov writes:
What I fail to see is why it's better to raise an exception and point users
to a better way, than to simply provide an optimization so that it's a mute
issue.
The only justification offered here is that will teach people that summing
strings (and
It seems to me this is something of a pointless discussion -- I highly
doubt the current situation is going to change, and it works very well.
Even if not perfect, sum() is for numbers, sep.join() for strings. However,
I will add one comment:
I'm overall -1 on trying to change the current
I'm very sympathetic to Steven's explanation that we
wouldn't be having this discussion if we used a different operator for
string concatenation.
Sure -- but just imagine the conversations we could be having instead
: what does bit wise and of a string mean? A bytes object? I cod see
it as a
On 04.08.2014 22:22, Jim J. Jewett wrote:
Sat Aug 2 12:11:54 CEST 2014, Julian Taylor wrote (in
https://mail.python.org/pipermail/python-dev/2014-August/135623.html ) wrote:
Andrea Griffini agriff at tin.it wrote:
However sum([[1,2,3],[4],[],[5,6]], []) concatenates the lists.
On 8/11/2014 8:26 AM, Ben Hoyt wrote:
It seems to me this is something of a pointless discussion -- I highly
doubt the current situation is going to change, and it works very well.
Even if not perfect, sum() is for numbers, sep.join() for strings.
However, I will add one comment:
I'm
On 12 Aug 2014 03:03, Chris Barker - NOAA Federal chris.bar...@noaa.gov
wrote:
My confusion is still this:
Repeated summation of strings has been optimized in cpython even
though it's not the recommended way to solve that problem.
The quadratic behaviour of repeated str summation is a
On Mon, Aug 11, 2014 at 8:19 PM, Nick Coghlan ncogh...@gmail.com wrote:
Teaching users the difference between linear time operations and quadratic
ones isn't about purity, it's about passing along a fundamental principle
of algorithm scalability.
I would understand if this was done in
Sorry for the bike shedding here, but:
The quadratic behaviour of repeated str summation is a subtle, silent error.
OK, fair enough. I suppose it would be hard and ugly to catch those
instances and raise an exception pointing users to .join.
*is* controversial that CPython silently optimises
Chris Barker - NOAA Federal writes:
Is there anything in the language spec that says string concatenation is
O(n^2)? Or for that matter any of the performs characteristics of build in
types? Those striker as implementation details that SHOULD be particular to
the implementation.
On 08/11/2014 08:50 PM, Stephen J. Turnbull wrote:
Chris Barker - NOAA Federal writes:
It seems pretty pedantic to say: we could make this work well,
but we'd rather chide you for not knowing the proper way to do
it.
Nobody disagrees. But backward compatibility gets in the way.
Something
Alexander Belopolsky writes:
On Sat, Aug 9, 2014 at 3:08 AM, Stephen J. Turnbull step...@xemacs.org
wrote:
All the suggestions
I've seen so far are (IMHO, YMMV) just as ugly as the present
situation.
What is ugly about allowing strings? CPython certainly has a way to to
On Aug 10, 2014, at 05:24 PM, Stephen J. Turnbull wrote:
Actually ... if I were a fan of the .join() idiom, I'd seriously
propose 0.sum(numeric_iterable) as the RightThang{tm]. Then we could
deprecate .join(string_iterable) in favor of .sum(string_iterable)
(with the same efficient semantics).
On 8/10/2014 1:24 AM, Stephen J. Turnbull wrote:
Actually ... if I were a fan of the .join() idiom, I'd seriously
propose 0.sum(numeric_iterable) as the RightThang{tm]. Then we could
deprecate .join(string_iterable) in favor of .sum(string_iterable)
(with the same efficient semantics).
On Sun, 10 Aug 2014 13:12:26 -0700, Glenn Linderman v+pyt...@g.nevcal.com
wrote:
On 8/10/2014 1:24 AM, Stephen J. Turnbull wrote:
Actually ... if I were a fan of the .join() idiom, I'd seriously
propose 0.sum(numeric_iterable) as the RightThang{tm]. Then we could
deprecate
On Sun, 10 Aug 2014 13:12:26 -0700, Glenn Linderman v+pyt...@g.nevcal.com
wrote:
On 8/10/2014 1:24 AM, Stephen J. Turnbull wrote:
Actually ... if I were a fan of the .join() idiom, I'd seriously
propose 0.sum(numeric_iterable) as the RightThang{tm]. Then we could
deprecate
Glenn Linderman writes:
On 8/10/2014 1:24 AM, Stephen J. Turnbull wrote:
Actually ... if I were a fan of the .join() idiom, I'd seriously
propose 0.sum(numeric_iterable) as the RightThang{tm]. Then we could
deprecate .join(string_iterable) in favor of .sum(string_iterable)
(with
Alexander Belopolsky writes:
Why have builtin sum at all if its use comes with so many caveats?
Because we already have it. If the caveats had been known when it was
introduced, maybe it wouldn't have been. The question is whether you
can convince python-dev that it's worth changing the
On 9 August 2014 06:08, Steven D'Aprano st...@pearwood.info wrote:
py with Stopwatch():
... sum(carray) # carray is a numpy array of 7500 floats.
...
11250.0
time taken: 52.659770 seconds
py with Stopwatch():
... numpy.sum(carray)
...
11250.0
time taken: 0.161263
On Sat, Aug 9, 2014 at 3:08 AM, Stephen J. Turnbull step...@xemacs.org
wrote:
All the suggestions
I've seen so far are (IMHO, YMMV) just as ugly as the present
situation.
What is ugly about allowing strings? CPython certainly has a way to to
make sum(x, '') at least as efficient as y='';for
On Sat, Aug 9, 2014 at 2:02 PM, Alexander Belopolsky
alexander.belopol...@gmail.com wrote:
y='';for in in x; y+= x
Should have been
y=''
for i in x; y += i
___
Python-Dev mailing list
Python-Dev@python.org
On Sat, Aug 9, 2014 at 1:08 AM, Steven D'Aprano st...@pearwood.info wrote:
We wouldn't be having
these interminable arguments about using sum() to concatenate strings
(and lists, and tuples) if the operator was used for concatenation and
+ was only used for numeric addition.
But we would
On Sat, Aug 9, 2014 at 12:20 PM, Alexander Belopolsky
alexander.belopol...@gmail.com wrote:
On Sat, Aug 9, 2014 at 1:08 AM, Steven D'Aprano st...@pearwood.info wrote:
We wouldn't be having
these interminable arguments about using sum() to concatenate strings
(and lists, and tuples) if the
On Thu, Aug 7, 2014 at 4:01 PM, Ethan Furman et...@stoneleaf.us wrote:
I don't remember where, but I believe that cPython has an optimization
built in for repeated string concatenation, which is probably why you
aren't seeing big differences between the + and the sum().
Indeed -- clearly so.
On 08/08/2014 08:23 AM, Chris Barker wrote:
So my final question is this:
repeated string concatenation is not the recommended way to do this -- but
nevertheless, cPython has an optimization
that makes it fast and efficient, to the point that there is no practical
performance reason to
On Aug 8, 2014, at 11:09 AM, Ethan Furman et...@stoneleaf.us wrote:
So why not apply a similar optimization to sum() for strings?
That I cannot answer -- I find the current situation with sum highly
irritating.
It is only irritating if you are misusing sum().
The str.__add__
On 08/08/2014 05:34 PM, Raymond Hettinger wrote:
On Aug 8, 2014, at 11:09 AM, Ethan Furman et...@stoneleaf.us
mailto:et...@stoneleaf.us wrote:
So why not apply a similar optimization to sum() for strings?
That I cannot answer -- I find the current situation with sum highly irritating.
On Fri, Aug 8, 2014 at 8:56 PM, Ethan Furman et...@stoneleaf.us wrote:
I don't use sum at all, or at least very rarely, and it still irritates me.
You are not alone. When I see sum([a, b, c]), I think it is a + b + c, but
in Python it is 0 + a + b + c. If we had a join operator for strings
On Fri, Aug 08, 2014 at 10:20:37PM -0400, Alexander Belopolsky wrote:
On Fri, Aug 8, 2014 at 8:56 PM, Ethan Furman et...@stoneleaf.us wrote:
I don't use sum at all, or at least very rarely, and it still irritates me.
You are not alone. When I see sum([a, b, c]), I think it is a + b + c,
Steven D'Aprano wrote:
I've long believed that + is the wrong operator for concatenating
strings, and that makes a much better operator.
Do you have a reason for preferring '' in particular, or
do you just want something different from '+'?
Personally I can't see why bitwise and on strings
Le 09/08/2014 01:08, Steven D'Aprano a écrit :
On Fri, Aug 08, 2014 at 10:20:37PM -0400, Alexander Belopolsky wrote:
On Fri, Aug 8, 2014 at 8:56 PM, Ethan Furman et...@stoneleaf.us wrote:
I don't use sum at all, or at least very rarely, and it still irritates me.
You are not alone. When I
On Mon, Aug 4, 2014 at 11:10 AM, Steven D'Aprano st...@pearwood.info
wrote:
On Mon, Aug 04, 2014 at 09:25:12AM -0700, Chris Barker wrote:
Good point -- I was trying to make the point about .join() vs + for
strings
in an intro python class last year, and made the mistake of having the
On 08/07/2014 03:06 PM, Chris Barker wrote:
[snip timings, etc.]
I don't remember where, but I believe that cPython has an optimization built in for repeated string concatenation, which
is probably why you aren't seeing big differences between the + and the sum().
A little testing shows how
On 08/07/2014 04:01 PM, Ethan Furman wrote:
On 08/07/2014 03:06 PM, Chris Barker wrote:
the + and the sum().
Yeah, that 'sum' should be 'join' :/
--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
On 08/07/2014 04:01 PM, Ethan Furman wrote:
On 08/07/2014 03:06 PM, Chris Barker wrote:
-- timeit.Timer(for string in ['booya'] * 10: blah = blah + string, blah =
'').repeat(3, 1)
[0.021117210388183594, 0.013692855834960938, 0.00768280029296875]
-- timeit.Timer(for string in ['booya'] *
On Sat, Aug 2, 2014 at 1:35 PM, David Wilson dw+python-...@hmmz.org wrote:
Repeated list and str concatenation both have quadratic O(N**2)
performance, but people frequently build up strings with +
join() isn't preferable in cases where it damages readability while
simultaneously
On Mon, Aug 04, 2014 at 09:25:12AM -0700, Chris Barker wrote:
Good point -- I was trying to make the point about .join() vs + for strings
in an intro python class last year, and made the mistake of having the
students test the performance.
You need to concatenate a LOT of strings to see any
Steven D'Aprano schrieb am 04.08.2014 um 20:10:
On Mon, Aug 04, 2014 at 09:25:12AM -0700, Chris Barker wrote:
Good point -- I was trying to make the point about .join() vs + for strings
in an intro python class last year, and made the mistake of having the
students test the performance.
Sat Aug 2 12:11:54 CEST 2014, Julian Taylor wrote (in
https://mail.python.org/pipermail/python-dev/2014-August/135623.html ) wrote:
Andrea Griffini agriff at tin.it wrote:
However sum([[1,2,3],[4],[],[5,6]], []) concatenates the lists.
hm could this be a pure python case that
On 8/2/2014 1:57 AM, Allen Li wrote:
On Fri, Aug 01, 2014 at 02:51:54PM -0700, Guido van Rossum wrote:
No. We just can't put all possible use cases in the docstring. :-)
On Fri, Aug 1, 2014 at 2:48 PM, Andrea Griffini agr...@tin.it wrote:
help(sum) tells clearly that it should be used
On 02.08.2014 08:35, Terry Reedy wrote:
On 8/2/2014 1:57 AM, Allen Li wrote:
On Fri, Aug 01, 2014 at 02:51:54PM -0700, Guido van Rossum wrote:
No. We just can't put all possible use cases in the docstring. :-)
On Fri, Aug 1, 2014 at 2:48 PM, Andrea Griffini agr...@tin.it wrote:
Julian Taylor schrieb am 02.08.2014 um 12:11:
On 02.08.2014 08:35, Terry Reedy wrote:
On 8/2/2014 1:57 AM, Allen Li wrote:
On Fri, Aug 01, 2014 at 02:51:54PM -0700, Guido van Rossum wrote:
No. We just can't put all possible use cases in the docstring. :-)
On Fri, Aug 1, 2014 at 2:48 PM,
On Sat, Aug 2, 2014 at 3:39 AM, Steven D'Aprano st...@pearwood.info wrote:
String concatenation with + is an attractive
nuisance for many people, including some who actually know better but
nevertheless do it. Also, for reasons I don't understand, many people
dislike or cannot remember to use
Alexander Belopolsky schrieb am 02.08.2014 um 16:52:
On Sat, Aug 2, 2014 at 3:39 AM, Steven D'Aprano wrote:
String concatenation with + is an attractive
nuisance for many people, including some who actually know better but
nevertheless do it. Also, for reasons I don't understand, many people
On Sat, Aug 02, 2014 at 10:52:07AM -0400, Alexander Belopolsky wrote:
On Sat, Aug 2, 2014 at 3:39 AM, Steven D'Aprano st...@pearwood.info wrote:
String concatenation with + is an attractive
nuisance for many people, including some who actually know better but
nevertheless do it. Also, for
On 2014-08-02 16:27, Steven D'Aprano wrote:
On Sat, Aug 02, 2014 at 10:52:07AM -0400, Alexander Belopolsky wrote:
On Sat, Aug 2, 2014 at 3:39 AM, Steven D'Aprano st...@pearwood.info wrote:
String concatenation with + is an attractive
nuisance for many people, including some who actually know
On Sat, Aug 2, 2014 at 11:06 AM, Stefan Behnel stefan...@behnel.de wrote:
I don't think sum(strings) is beautiful enough
sum(strings) is more beautiful than ''.join(strings) in my view, but
unfortunately it does not work even for lists because the initial value
defaults to 0.
sum(strings, '')
On Sat, Aug 02, 2014 at 05:39:12PM +1000, Steven D'Aprano wrote:
Repeated list and str concatenation both have quadratic O(N**2)
performance, but people frequently build up strings with + and rarely
do the same for lists. String concatenation with + is an attractive
nuisance for many people,
help(sum) tells clearly that it should be used to sum numbers and not
strings, and with strings actually fails.
However sum([[1,2,3],[4],[],[5,6]], []) concatenates the lists.
Is this to be considered a bug?
Andrea
___
Python-Dev mailing list
No. We just can't put all possible use cases in the docstring. :-)
On Fri, Aug 1, 2014 at 2:48 PM, Andrea Griffini agr...@tin.it wrote:
help(sum) tells clearly that it should be used to sum numbers and not
strings, and with strings actually fails.
However sum([[1,2,3],[4],[],[5,6]], [])
On Fri, Aug 01, 2014 at 02:51:54PM -0700, Guido van Rossum wrote:
No. We just can't put all possible use cases in the docstring. :-)
On Fri, Aug 1, 2014 at 2:48 PM, Andrea Griffini agr...@tin.it wrote:
help(sum) tells clearly that it should be used to sum numbers and not
strings,
In the comments before the implementation of sum()
in mathmodule.c, note 4 says:
Note 4: A similar implementation is in Modules/cmathmodule.c.
Be sure to update both when making changes.
I can't find a sum implementation or similar code in cmathmodule.c.
Did someone intend to port the sum
We're still working on the implementation details for math.sum().
When it's finished, the cmath equilvalent will be added.
Raymond
- Original Message -
From: A.M. Kuchling [EMAIL PROTECTED]
To: python-dev@python.org
Sent: Thursday, June 19, 2008 7:16 PM
Subject: [Python-Dev] sum
[Tim]
For contrast, here's one that doesn't use frexp(), and is probably
faster because of that; internally, len(sums) probably won't exceed 5
in real life (but could get as large as 2K for pathological inputs,
spanning fp's full dynamic range):
def summer4(iterable):
sums = [0.0]
[Tim Peters]
Summer.add() _can_ lose info -- it needs additional
trickery to make it loss-free, and because x+y can lose (at most one
bit) when x and y have the same binary exponent.
Computing an error term can get the bit back and putting that term back
in the input queue restores the
[Raymond Hettinger]
Computing an error term can get the bit back and putting that term back
in the input queue restores the overall sum.
Right!
Once the inputs are exhausted, the components of exp2sum should be exact.
Ditto. I'll cover some subtleties below:
from math import frexp
from
the queue can be expected to grow to about half the length
of the original iterable by the time the original iterable is
exhausted.
x = z
mant, exp = frexp(x)
exp2sum[exp] = x
return sum(sorted(exp2sum.itervalues(), key=abs), 0.0)
The implementation
You guys have way too much time on your hands and neurons devoted to
this stuff. I'm glad that means that I can spend the afternoon playing
w/ my kids and searching python-dev when I need to add numbers =).
___
Python-Dev mailing list
[Alex]
If you're considering revisions to sum's design, my suggestion would
be
to find a way to let the user tell sum to use a more accurate approach
when summing floats.
FWIW, when accuracy is an issue, I use:
sum(sorted(data, key=abs))
Raymond
On Mar 11, 2005, at 19:39, Raymond Hettinger wrote:
[Alex]
If you're considering revisions to sum's design, my suggestion would
be
to find a way to let the user tell sum to use a more accurate approach
when summing floats.
FWIW, when accuracy is an issue, I use:
sum(sorted(data, key=abs))
[Alex]
FWIW, when accuracy is an issue, I use:
sum(sorted(data, key=abs))
...and you may still lose a LOT of accuracy that way:-(.
Raymond, you technically reviewed the 2nd ed Cookbook -- don't you
recall the sidebar about why partials are the RIGHT way to do
summations? I was SO
FYI, there are a lot of ways to do accurate fp summation, but in
general people worry about it too much (except for those who don't
worry about it at all -- they're _really_ in trouble 0.5 wink).
One clever way is to build on that whenever |x| and |y| are within a
factor of 2 of each other, x+y
69 matches
Mail list logo