Re: Speed-up for loops

2010-09-09 Thread BartC
Stefan Behnel stefan...@behnel.de wrote in message news:mailman.563.1283921317.29448.python-l...@python.org... BartC, 08.09.2010 03:45: Getting back to the OP's code again (trivial and pointless as it might seem), I got these results: C (gcc 3.4.5 -O3) 0.8 secs C (DMC-o) 2.3 secs C (lccwin32

Re: Speed-up for loops

2010-09-08 Thread Ulrich Eckhardt
BartC wrote: So 'range' is just a class like any other. And that a class is something you can blithely copy from one variable to another. And whenever you see 'range' anywhere, you can't always be certain that someone hasn't done: range = 42 at some point. True. I read an explanation

Re: Speed-up for loops

2010-09-08 Thread Steven D'Aprano
On Tue, 07 Sep 2010 11:00:03 +0100, BartC wrote: for i in xrange(1): a = a + f(i) then unrolling the loop is even less useful. The overhead of the loop itself is likely to be trivial compared to the cost of calling f() 100 million times -- the added complexity to shave 3 seconds

Re: Speed-up for loops

2010-09-08 Thread Alain Ketterlin
Steven D'Aprano st...@remove-this-cybersource.com.au writes: With Python 3 and def f(x): return x+1, unrolling this loop 4x improved speed by 15%; 4.00 minutes reduces to 3.30 minutes. I'm afraid that I can't replicate those figures. In my test, unrolling the loop causes a massive SLOWDOWN

Re: Speed-up for loops

2010-09-08 Thread BartC
Steven D'Aprano st...@remove-this-cybersource.com.au wrote in message news:4c878be5$0$3$c3e8...@news.astraweb.com... On Tue, 07 Sep 2010 11:00:03 +0100, BartC wrote: for i in xrange(1): a = a + f(i) With Python 3 and def f(x): return x+1, unrolling this loop 4x improved speed

Re: Speed-up for loops

2010-09-07 Thread BartC
Steven D'Aprano steve-remove-t...@cybersource.com.au wrote in message news:4c85adfe$0$5$c3e8...@news.astraweb.com... On Mon, 06 Sep 2010 11:38:22 +0100, BartC wrote: Manually unrolling such a loop four times (ie. 4 copies of the body, and counting only to 25 million) increased the speed

Re: Speed-up for loops

2010-09-07 Thread Roy Smith
In article ny3ho.32445$la2.18...@newsfe23.ams2, BartC ba...@freeuk.com wrote: (BTW why doesn't Python 3 just accept 'xrange' as a synonym for 'range'?) If you've ever tried to maintain a legacy code base, you'll understand why there's only one way to do it is A Good Thing. Imagine that

Re: Speed-up for loops

2010-09-07 Thread Aahz
In article o4oho.85508$za5.26...@newsfe16.ams2, BartC ba...@freeuk.com wrote: Steven D'Aprano steve-remove-t...@cybersource.com.au wrote in message news:4c85adfe$0$5$c3e8...@news.astraweb.com... xrange = range There, that wasn't hard, was it? I think I just learned more about Python than

Re: Speed-up for loops

2010-09-07 Thread Aahz
In article roy-0ec6f2.08091107092...@news.panix.com, Roy Smith r...@panix.com wrote: Imagine that you're looking at some code which was written years ago, by people who are no longer around to answer questions. In one place, you see: for i in range(n): blah and in another, you see: for

Re: Speed-up for loops

2010-09-07 Thread David Cournapeau
On Sun, Sep 5, 2010 at 8:28 PM, BartC ba...@freeuk.com wrote: One order of magnitude (say 10-20x slower) wouldn't be so bad. That's what you might expect for a dynamically typed, interpreted language. 10/20x slower than C is only reached by extremely well optimized dynamic languages. It would

Re: Speed-up for loops

2010-09-07 Thread Terry Reedy
On 9/7/2010 6:00 AM, BartC wrote: Why should it? But if you want it, you can do it: xrange = range There, that wasn't hard, was it? I think I just learned more about Python than from months of reading this group. So 'range' is just a class like any other. And that a class is something you

Re: Speed-up for loops

2010-09-07 Thread BartC
David Cournapeau courn...@gmail.com wrote in message news:mailman.546.1283897932.29448.python-l...@python.org... On Sun, Sep 5, 2010 at 8:28 PM, BartC ba...@freeuk.com wrote: One order of magnitude (say 10-20x slower) wouldn't be so bad. That's what you might expect for a dynamically typed,

Re: Speed-up for loops

2010-09-07 Thread MRAB
On 08/09/2010 02:45, BartC wrote: David Cournapeau courn...@gmail.com wrote in message news:mailman.546.1283897932.29448.python-l...@python.org... On Sun, Sep 5, 2010 at 8:28 PM, BartC ba...@freeuk.com wrote: One order of magnitude (say 10-20x slower) wouldn't be so bad. That's what you

Re: Speed-up for loops

2010-09-07 Thread Stefan Behnel
BartC, 08.09.2010 03:45: Getting back to the OP's code again (trivial and pointless as it might seem), I got these results: C (gcc 3.4.5 -O3) 0.8 secs C (DMC-o) 2.3 secs C (lccwin32 -O) 2.9 secs [...] I've seen LuaJIT in action. It's timing for this test is 1.5 secs: forget being only 10x

Re: Speed-up for loops

2010-09-06 Thread BartC
Stefan Behnel stefan...@behnel.de wrote in message news:mailman.470.1283712666.29448.python-l...@python.org... BartC, 05.09.2010 19:09: All those compilers that offer loop unrolling are therefore wasting their time... Sometimes they do, yes. Modifying the OP's code a little: a = 0 for i

Re: Speed-up for loops

2010-09-06 Thread Stefan Behnel
BartC, 06.09.2010 12:38: why doesn't Python 3 just accept 'xrange' as a synonym for 'range'? Because Python 3 deliberately breaks backwards compatibility in order to clean up the language. These are just some simple tests on my particular machine and implementations, but they bring up

Re: Speed-up for loops

2010-09-06 Thread Kushal Kumaran
On Mon, Sep 6, 2010 at 4:08 PM, BartC ba...@freeuk.com wrote: Stefan Behnel stefan...@behnel.de wrote in message news:mailman.470.1283712666.29448.python-l...@python.org... BartC, 05.09.2010 19:09: All those compilers that offer loop unrolling are therefore wasting their time... Sometimes

Re: Speed-up for loops

2010-09-06 Thread Antoine Pitrou
On Mon, 06 Sep 2010 13:20:01 +0200 Stefan Behnel stefan...@behnel.de wrote: (2) Integer arithmetic seems to go straight from 32-bits to long integers; why not use 64-bits before needing long integers? You are making assumptions based on Python 2, I guess. Try Python 3.1 or later

Re: Speed-up for loops

2010-09-06 Thread BartC
Stefan Behnel stefan...@behnel.de wrote in message news:mailman.485.1283772019.29448.python-l...@python.org... BartC, 06.09.2010 12:38: (2) Integer arithmetic seems to go straight from 32-bits to long integers; why not use 64-bits before needing long integers? You are making assumptions

Re: Speed-up for loops

2010-09-06 Thread Terry Reedy
On 9/6/2010 7:20 AM, Stefan Behnel wrote: BartC, 06.09.2010 12:38: (3) Since the loop variable is never used, why not have a special loop statement that repeats code so many times? There sort-of is, just slightly more general. Because special cases are not special enough to break the

Re: Speed-up for loops

2010-09-06 Thread Steven D'Aprano
On Mon, 06 Sep 2010 11:38:22 +0100, BartC wrote: Modifying the OP's code a little: a = 0 for i in xrange(1): # 100 million a = a + 10 # add 10 or 100 print a Manually unrolling such a loop four times (ie. 4 copies of the body, and counting only to 25

Re: Speed-up for loops

2010-09-05 Thread BartC
David Cournapeau courn...@gmail.com wrote in message news:mailman.455.1283665528.29448.python-l...@python.org... On Thu, Sep 2, 2010 at 7:02 PM, Michael Kreim mich...@perfect-kreim.de wrote: imax = 10 a = 0 for i in xrange(imax): a = a + 10 print a Unfortunately my Python Code

Re: Speed-up for loops

2010-09-05 Thread Steven D'Aprano
On Fri, 03 Sep 2010 21:17:44 +0100, BartC wrote: I'm not sure the Python developers were interested in getting fast loops. For-loops which iterate between two numbers are amongst the easiest things to make fast in a language. Yet originally you had to use: for i in range(N): I don't

Re: Speed-up for loops

2010-09-05 Thread Steven D'Aprano
On Sun, 05 Sep 2010 12:28:47 +0100, BartC wrote: Getting the above kind of code fast requires the interpreter to be clever enough so that it will use native machine operations on a int type instead of converting back and forth between internal representations. Writing for i in

Re: Speed-up for loops

2010-09-05 Thread Stefan Behnel
Steven D'Aprano, 05.09.2010 17:00: Of course, a real optimizing compiler would realise that the Pascal code did nothing at all, and compile it all away to an empty a.out file... Which is just one of the reasons why this kind if benchmark provides no insight into anything that should have an

Re: Speed-up for loops

2010-09-05 Thread BartC
Steven D'Aprano st...@remove-this-cybersource.com.au wrote in message news:4c83b425$0$28657$c3e8...@news.astraweb.com... On Sun, 05 Sep 2010 12:28:47 +0100, BartC wrote: It would be nice if you could directly code low-level algorithms in it without relying on accelerators, and not have to

Re: Speed-up for loops

2010-09-05 Thread Stefan Behnel
BartC, 05.09.2010 19:09: I've thought about it (writing an independent interpreter). But I don't know enough of the language, and a lot of it I don't understand (eg. OOP). Besides, I think the language itself deliberately makes it difficult to get it up to speed. Some of the reasons might be the

Re: Speed-up for loops

2010-09-04 Thread Gabriele Lanaro
Maybe for the simple sum you can just use the sum builtin: python -m timeit -s 'sum((10,)*1)' 1000 loops, best of 3: 0.0985 usec per loop About the loop in general it's a good practice to use list comprehension and generator expressions 2010/9/2 Michael Kreim mich...@perfect-kreim.de

Re: Speed-up for loops

2010-09-04 Thread David Cournapeau
On Thu, Sep 2, 2010 at 7:02 PM, Michael Kreim mich...@perfect-kreim.de wrote: Hi, I was comparing the speed of a simple loop program between Matlab and Python. My Codes: $ cat addition.py imax = 10 a = 0 for i in xrange(imax):    a = a + 10 print a $ cat addition.m imax =

Re: Speed-up for loops

2010-09-03 Thread Ulrich Eckhardt
Tim Wintle wrote: [..] under the hood, cpython does something like this (in psudo-code) itterator = xrange(imax) while 1: next_attribute = itterator.next try: i = next_attribute() except: break a = a + 10 There is one thing that strikes me here: The code claims that

Re: Speed-up for loops

2010-09-03 Thread Stefan Behnel
Ulrich Eckhardt, 03.09.2010 08:52: Tim Wintle wrote: [..] under the hood, cpython does something like this (in psudo-code) itterator = xrange(imax) while 1: next_attribute = itterator.next try: i = next_attribute() except: break a = a + 10 There is one thing that

Re: Speed-up for loops

2010-09-03 Thread Michael Kreim
Hi, thanks a lot for your answers. I learn a lot and I like to sum up your suggestions and show you the results of the time command on my machine: Original code by me: imax = 10 a = 0 for i in xrange(imax): a = a + 10 print a = runs (wall clock time): 1:55.14 Peter Otten

Re: Speed-up for loops

2010-09-03 Thread Stefan Behnel
Michael Kreim, 03.09.2010 11:21: So finally I followed the recommendation of Tim Wintle to use cython. I did not know this before, but I figured out the following: additionWintle2.pyx: def addition(): cdef long imax = 10 cdef long a = 0 cdef long i for i in

Re: Speed-up for loops

2010-09-03 Thread python
Michael, Thanks for summarizing and sharing your results. Very interesting. Regards, Malcolm -- http://mail.python.org/mailman/listinfo/python-list

Re: Speed-up for loops

2010-09-03 Thread Hrvoje Niksic
Ulrich Eckhardt eckha...@satorlaser.com writes: Tim Wintle wrote: [..] under the hood, cpython does something like this (in psudo-code) itterator = xrange(imax) while 1: next_attribute = itterator.next try: i = next_attribute() except: break a = a + 10 There is one

Re: Speed-up for loops

2010-09-03 Thread Tim Wintle
On Fri, 2010-09-03 at 08:52 +0200, Ulrich Eckhardt wrote: Tim Wintle wrote: [..] under the hood, cpython does something like this (in psudo-code) itterator = xrange(imax) while 1: next_attribute = itterator.next try: i = next_attribute() except: break a = a +

Re: Speed-up for loops

2010-09-03 Thread Nobody
On Fri, 03 Sep 2010 11:21:36 +0200, Michael Kreim wrote: An anonymous Nobody suggested to use Numpy. I did not do this, because I am very very new to Numpy and I did not figure out a Numpy specific way to do this. Maybe a Numpy expert has something for me? The problem with giving examples

Re: Speed-up for loops

2010-09-03 Thread BartC
Michael Kreim mich...@perfect-kreim.de wrote in message news:mailman.362.1283422325.29448.python-l...@python.org... I was comparing the speed of a simple loop program between Matlab and Python. My Codes: $ cat addition.py imax = 10 a = 0 for i in xrange(imax): a = a + 10 print a

Re: Speed-up for loops

2010-09-03 Thread Stefan Behnel
BartC, 03.09.2010 22:17: for i in range(N): which (if I understood correctly) actually created a list of N objects, populated it with the values 0, 1, 2...N-1 (presumably using a more sensible loop), then iterated between the values of the list! I guess what applies here is special cases

Speed-up for loops

2010-09-02 Thread Michael Kreim
Hi, I was comparing the speed of a simple loop program between Matlab and Python. My Codes: $ cat addition.py imax = 10 a = 0 for i in xrange(imax): a = a + 10 print a $ cat addition.m imax = 1e9; a = 0; for i=0:imax-1 a = a + 10; end disp(a); exit; The results look like

Re: Speed-up for loops

2010-09-02 Thread Peter Otten
Michael Kreim wrote: I was comparing the speed of a simple loop program between Matlab and Python. My Codes: $ cat addition.py imax = 10 a = 0 for i in xrange(imax): a = a + 10 print a Are there any ways to speed up the for/xrange loop? Move it into a function; this

Re: Speed-up for loops

2010-09-02 Thread Michael Kreim
Peter Otten wrote: Move it into a function; this turns a and i into local variables. def f(): imax = 10 a = 0 for i in xrange(imax): a = a + 10 print a f() Wow. It is still slower than Matlab, but your suggestion speeds up the code by ca 50%. But I do not

Re: Speed-up for loops

2010-09-02 Thread Peter Otten
Michael Kreim wrote: Peter Otten wrote: Move it into a function; this turns a and i into local variables. def f(): imax = 10 a = 0 for i in xrange(imax): a = a + 10 print a f() Wow. It is still slower than Matlab, but your suggestion speeds up the

Re: Speed-up for loops

2010-09-02 Thread Tim Wintle
On Thu, 2010-09-02 at 12:02 +0200, Michael Kreim wrote: Hi, I was comparing the speed of a simple loop program between Matlab and Python. Unfortunately my Python Code was much slower and I do not understand why. The main reason is that, under the hood, cpython does something like this (in

Re: Speed-up for loops

2010-09-02 Thread Stefan Behnel
Tim Wintle, 02.09.2010 14:55: If you really need to optimise it then you can convert that module to cython by adding a cdef, and then compile it: cdef int i for i in xrange(imax): a = a + 10 print a or you can write it in C it'll run a lot faster. Just to get the context right here: a

Re: Speed-up for loops

2010-09-02 Thread Hrvoje Niksic
Michael Kreim mich...@perfect-kreim.de writes: Are there any ways to speed up the for/xrange loop? Or do I have to live with the fact that Matlab beats Python in this example? To a point, yes. However, there are things you can do to make your Python code go faster. One has been pointed out

Re: Speed-up for loops

2010-09-02 Thread Roland Koebler
Hi, Are there any ways to speed up the for/xrange loop? You can use psyco. The following example should be about 4-times as fast as your example: import psyco psyco.full() def f(): imax = 10 a = 0 for i in xrange(imax): a += 10 print a f() regards,

Re: Speed-up for loops

2010-09-02 Thread Nobody
On Thu, 02 Sep 2010 12:02:40 +0200, Michael Kreim wrote: I was comparing the speed of a simple loop program between Matlab and Python. imax = 10 a = 0 for i in xrange(imax): a = a + 10 print a Are there any ways to speed up the for/xrange loop? Sure; the above can be

Re: Speed-up for loops

2010-09-02 Thread Tim Wintle
On Thu, 2010-09-02 at 16:13 +0200, Roland Koebler wrote: Hi, Are there any ways to speed up the for/xrange loop? You can use psyco. Assuming you've got a 32-bit machine. -- http://mail.python.org/mailman/listinfo/python-list

RE: Speed-up for loops

2010-09-02 Thread Philip Bloom
] On Behalf Of Nobody Sent: Thursday, September 02, 2010 9:05 AM To: python-list@python.org Subject: Re: Speed-up for loops On Thu, 02 Sep 2010 12:02:40 +0200, Michael Kreim wrote: I was comparing the speed of a simple loop program between Matlab and Python. imax = 10 a = 0 for i

RE: Speed-up for loops

2010-09-02 Thread Peter Otten
Philip Bloom wrote: Uh. Try: Imax=10 a=0 i=0 While(iimax): a= a+10 i=i+1 print a I suspect you will find it is way faster than using range or xrange for large numbers and map far more closely in the final result to what you are doing on matlab's side. At

Re: Speed-up for loops

2010-09-02 Thread Carl Banks
On Sep 2, 5:55 am, Tim Wintle tim.win...@teamrubber.com wrote: On Thu, 2010-09-02 at 12:02 +0200, Michael Kreim wrote: Hi, I was comparing the speed of a simple loop program between Matlab and Python. Unfortunately my Python Code was much slower and I do not understand why. The main

Re: Speed-up for loops

2010-09-02 Thread Terry Reedy
On 9/2/2010 8:55 AM, Tim Wintle wrote: On Thu, 2010-09-02 at 12:02 +0200, Michael Kreim wrote: Hi, I was comparing the speed of a simple loop program between Matlab and Python. Unfortunately my Python Code was much slower and I do not understand why. The main reason is that, under the