Re: [Numpy-discussion] How to remove fortran-like loops with numpy?
I look forward to an instructive reply: the "Pythonic" way to do it would be to
take advantage of the facts that Numpy is "pre-vectorized" and uses
broadcasting, but so far I haven't been able to figure out (though I haven't
yet really buckled down and tried real hard) how to broadcast a
conditionally-terminated iteration where the number of iterations will vary
among the array elements. Hopefully someone else already has. :-)
DG
--- On Mon, 6/8/09, Juanjo Gomez Navarro wrote:
> From: Juanjo Gomez Navarro
> Subject: [Numpy-discussion] How to remove fortran-like loops with numpy?
> To: numpy-discussion@scipy.org
> Date: Monday, June 8, 2009, 2:52 PM
>
> Hi all,
>
> I'm new in numpy. Actually, I'm new in Python. In
> order to learn a bit, I want to create a program to plot the
> Mandelbrot set. This program is quite simple, and I have
> already programmed it. The problem is that I come from
> fortran, so I use to think in "for" loops. I know
> that it is not the best way to use Python and in fact the
> performance of the program is more than poor.
>
>
>
> Here is the program:
>
> #!/usr/bin/python
>
>
>
> import numpy as np
>
> import matplotlib.pyplot as plt
>
>
>
> # Some parameters
> Xmin=-1.5
>
> Xmax=0.5
>
> Ymin=-1
>
> Ymax=1
>
>
> Ds = 0.01
>
>
> # Initialization of varibles
> X = np.arange(Xmin,Xmax,Ds)
>
>
> Y = np.arange(Ymax,Ymin,-Ds)
>
>
>
> N = np.zeros((X.shape[0],Y.shape[0]),'f')
>
>
> ## Here are inefficient the calculations
>
> for i in range(X.shape[0]):
> for j in range(Y.shape[0]):
> z= complex(0.0, 0.0)
> c = complex(X[i], Y[j])
>
> while N[i, j] < 30 and abs(z) < 2:
>
> N[i, j] += 1
> z = z**2 + c
> if N[i, j] == 29:
> N[i, j]=0
>
>
> # And now, just for ploting...
> N = N.transpose()
>
> fig = plt.figure()
>
> plt.imshow(N,cmap=plt.cm.Blues)
>
>
> plt.title('Mandelbrot set')
>
> plt.xticks([]); plt.yticks([])
>
>
> plt.show()
>
> fig.savefig('test.png')
>
>
>
> As you can see, it is very simple, but it takes several
> seconds running just to create a 200x200 plot. Fortran takes
> same time to create a 2000x2000 plot, around 100 times
> faster... So the question is, do you know how to programme
> this in a Python-like fashion in order to improve seriously
> the performance?
>
>
>
> Thanks in advance
> --
> Juan José Gómez Navarro
>
> Edificio CIOyN, Campus de Espinardo, 30100
> Departamento de Física
> Universidad de Murcia
> Tfno. (+34) 968 398552
>
> Email: juanjo.gomeznava...@gmail.com
>
> Web: http://ciclon.inf.um.es/Inicio.html
>
>
> -Inline Attachment Follows-
>
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How to remove fortran-like loops with numpy?
On Mon, Jun 8, 2009 at 17:04, David Goldsmith wrote: > > I look forward to an instructive reply: the "Pythonic" way to do it would be > to take advantage of the facts that Numpy is "pre-vectorized" and uses > broadcasting, but so far I haven't been able to figure out (though I haven't > yet really buckled down and tried real hard) how to broadcast a > conditionally-terminated iteration where the number of iterations will vary > among the array elements. Hopefully someone else already has. :-) You can't, really. What you can do is just keep iterating with the whole data set and ignore the parts that have already converged. Here is an example: z = np.zeros((201,201), dtype=complex) Y, X = np.mgrid[1:-1:-201j, -1.5:0.5:201j] c = np.empty_like(z) c.real = X c.imag = Y N = np.zeros(z.shape, dtype=int) while ((N<30) | (abs(z)<2)).all(): N += abs(z) < 2 z = z ** 2 + c N[N>=30] = 0 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How to remove fortran-like loops with numpy?
2009/6/8 Robert Kern : > On Mon, Jun 8, 2009 at 17:04, David Goldsmith wrote: >> >> I look forward to an instructive reply: the "Pythonic" way to do it would be >> to take advantage of the facts that Numpy is "pre-vectorized" and uses >> broadcasting, but so far I haven't been able to figure out (though I haven't >> yet really buckled down and tried real hard) how to broadcast a >> conditionally-terminated iteration where the number of iterations will vary >> among the array elements. Hopefully someone else already has. :-) > > You can't, really. What you can do is just keep iterating with the > whole data set and ignore the parts that have already converged. Here > is an example: Well, yes and no. This is only worth doing if the number of problem points that require many iterations is small - not the case here without some sort of periodicity detection - but you can keep an array of not-yet-converged points, which you iterate. When some converge, you store them in a results array (with fancy indexing) and remove them from your still-converging array. It's also worth remembering that the overhead of for loops is large but not enormous, so you can often remove only the inner for loop, in this case perhaps iterating over the image a line at a time. Anne > > z = np.zeros((201,201), dtype=complex) > Y, X = np.mgrid[1:-1:-201j, -1.5:0.5:201j] > c = np.empty_like(z) > c.real = X > c.imag = Y > N = np.zeros(z.shape, dtype=int) > > while ((N<30) | (abs(z)<2)).all(): > N += abs(z) < 2 > z = z ** 2 + c > > N[N>=30] = 0 > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma that is made terrible by our own mad attempt to interpret it as > though it had an underlying truth." > -- Umberto Eco > ___ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How to remove fortran-like loops with numpy?
Thanks, Robert! DG --- On Mon, 6/8/09, Robert Kern wrote: > I haven't been able to figure out (though I haven't yet > really buckled down and tried real hard) how to broadcast a > conditionally-terminated iteration where the number of > iterations will vary among the array elements. Hopefully > someone else already has. :-) > > You can't, really. What you can do is just keep iterating > with the > whole data set and ignore the parts that have already > converged. Here > is an example: > > z = np.zeros((201,201), dtype=complex) > Y, X = np.mgrid[1:-1:-201j, -1.5:0.5:201j] > c = np.empty_like(z) > c.real = X > c.imag = Y > N = np.zeros(z.shape, dtype=int) > > while ((N<30) | (abs(z)<2)).all(): > N += abs(z) < 2 > z = z ** 2 + c > > N[N>=30] = 0 > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, > a harmless > enigma that is made terrible by our own mad attempt to > interpret it as > though it had an underlying truth." > -- Umberto Eco > ___ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How to remove fortran-like loops with numpy?
--- On Mon, 6/8/09, Anne Archibald wrote: > > You can't, really. What you can do is just keep > iterating with the > > whole data set and ignore the parts that have already > converged. Here > > is an example: > > Well, yes and no. This is only worth doing if the number of > problem > points that require many iterations is small - not the case > here > without some sort of periodicity detection - but you can > keep an array > of not-yet-converged points, which you iterate. When some > converge, > you store them in a results array (with fancy indexing) and > remove > them from your still-converging array. Thanks, Anne. This is the way I had anticipated implementing it myself eventually, but the "fancy-indexing" requirement has caused me to keep postponing it, waiting for some time when I'll have a hefty block of time to figure it out and then, inevitably, debug it. :( Also, the transfer of points from un-converged to converged - when that's a large number, might that not be a large time-suck compared to Rob's method? (Too bad this wasn't posted a couple weeks ago: I'd've had time then to implement your method and "race" it against Rob's, but alas, now I have this doc editing job...but that's a good thing, as my fractals are not yet making me any real money.) :-) > It's also worth remembering that the overhead of for loops > is large > but not enormous, so you can often remove only the inner > for loop, in > this case perhaps iterating over the image a line at a > time. Yes, definitely well worth remembering - thanks for reminding us! Thanks again, DG > > Anne > ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How to remove fortran-like loops with numpy?
On Mon, Jun 8, 2009 at 18:01, David Goldsmith wrote: > > --- On Mon, 6/8/09, Anne Archibald wrote: > >> > You can't, really. What you can do is just keep >> iterating with the >> > whole data set and ignore the parts that have already >> converged. Here >> > is an example: >> >> Well, yes and no. This is only worth doing if the number of >> problem >> points that require many iterations is small - not the case >> here >> without some sort of periodicity detection - but you can >> keep an array >> of not-yet-converged points, which you iterate. When some >> converge, >> you store them in a results array (with fancy indexing) and >> remove >> them from your still-converging array. > > Thanks, Anne. This is the way I had anticipated implementing it myself > eventually, but the "fancy-indexing" requirement has caused me to keep > postponing it, waiting for some time when I'll have a hefty block of time to > figure it out and then, inevitably, debug it. :( Also, the transfer of > points from un-converged to converged - when that's a large number, might > that not be a large time-suck compared to Rob's method? (Too bad this wasn't > posted a couple weeks ago: I'd've had time then to implement your method and > "race" it against Rob's, but alas, now I have this doc editing job...but > that's a good thing, as my fractals are not yet making me any real money.) :-) The advantage of my implementation is that I didn't have to think too hard about it. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How to remove fortran-like loops with numpy?
Hi Juan
2009/6/8 Juanjo Gomez Navarro :
> I'm new in numpy. Actually, I'm new in Python. In order to learn a bit, I
> want to create a program to plot the Mandelbrot set. This program is quite
> simple, and I have already programmed it. The problem is that I come from
> fortran, so I use to think in "for" loops. I know that it is not the best
> way to use Python and in fact the performance of the program is more than
> poor.
>
> Here is the program:
>
>> #!/usr/bin/python
>>
>> import numpy as np
>> import matplotlib.pyplot as plt
>>
>> # Some parameters
>> Xmin=-1.5
>> Xmax=0.5
>> Ymin=-1
>> Ymax=1
>>
>> Ds = 0.01
>>
>> # Initialization of varibles
>> X = np.arange(Xmin,Xmax,Ds)
>> Y = np.arange(Ymax,Ymin,-Ds)
>>
>> N = np.zeros((X.shape[0],Y.shape[0]),'f')
>>
>> ## Here are inefficient the calculations
>> for i in range(X.shape[0]):
>> for j in range(Y.shape[0]):
>> z= complex(0.0, 0.0)
>> c = complex(X[i], Y[j])
>> while N[i, j] < 30 and abs(z) < 2:
>> N[i, j] += 1
>> z = z**2 + c
>> if N[i, j] == 29:
>> N[i, j]=0
>>
>>
>> # And now, just for ploting...
>> N = N.transpose()
>> fig = plt.figure()
>> plt.imshow(N,cmap=plt.cm.Blues)
>> plt.title('Mandelbrot set')
>> plt.xticks([]); plt.yticks([])
>> plt.show()
>> fig.savefig('test.png')
>
>
> As you can see, it is very simple, but it takes several seconds running just
> to create a 200x200 plot. Fortran takes same time to create a 2000x2000
> plot, around 100 times faster... So the question is, do you know how to
> programme this in a Python-like fashion in order to improve seriously the
> performance?
Here is another version, similar to Robert's, that I wrote up for the
documentation project last year:
http://mentat.za.net/numpy/intro/intro.html
We never used it, but I still like the pretty pictures :-)
Cheers
Stéfan
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How to remove fortran-like loops with numpy?
--- On Mon, 6/8/09, Robert Kern wrote: > Goldsmith > wrote: > > > The advantage of my implementation is that I didn't have to > think too > hard about it. > -- > Robert Kern > Agreed. :-) DG ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How to remove fortran-like loops with numpy?
2009/6/9 Stéfan van der Walt : > Hi Juan > <---cut ---> >> As you can see, it is very simple, but it takes several seconds running just >> to create a 200x200 plot. Fortran takes same time to create a 2000x2000 >> plot, around 100 times faster... So the question is, do you know how to >> programme this in a Python-like fashion in order to improve seriously the >> performance? > > Here is another version, similar to Robert's, that I wrote up for the > documentation project last year: > > http://mentat.za.net/numpy/intro/intro.html > > We never used it, but I still like the pretty pictures :-) +1 Hi Stéfan, this does look really nice !! Could it be put somewhere more prominent !? How about onto the SciPy site with a "newcomers start here"-link right on the first page My 2 cents, Sebastian Haase ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
