Re: [Tutor] job killed: too high numbers?
Hi, The number was high but it was only a sort of test.now I learned how it works, thanks Gb On Sep 20, 2016 12:48 PM, "Matt Ruffalo" wrote: > Hello- > > On 2016-09-20 11:48, Gabriele Brambilla wrote: > > does it mean that my number of points is too high? > > In short, yes. From your usage of the 'print' statement, you are running > the code under Python 2.x. In this version of Python, the 'range' > function creates a full list of numbers, and so you are asking 'range' > to create a list of 33.8 billion integers. Python lists are essentially > implemented in C as dense arrays of pointers to PyObject structs, so in > addition to the actual numeric values, you will need eight bytes per > value in the list (assuming a 64-bit OS and Python build). This is > already 270GB of memory just for these pointers, in addition to the > actual numeric values, which might take up to an additional 135GB (if > each numeric value is stored as a 32-bit integer). Are you running this > on a machine with ≥405GB memory? > > To solve your immediate problem, you could replace 'range' with 'xrange' > in your code, but this will probably only allow you to encounter another > problem: this loop will take a *very* long time to run, even without > doing any numerical work inside it. Unfortunately, there's no way to > suggest any algorithm/numerical analysis improvements without more > information about what you're trying to accomplish. > > MMR... > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] job killed: too high numbers?
Hi, I have this script import numpy as np import random as rnd from math import * import matplotlib.pyplot as plt folder = '../NOBACKUP/heavi3/' pre = '10' dat = '.dat' snshot = '4189' d = 0.02 d2 = d/2.0 tot = 156 half = 78 points = 3375000 BxN = np.zeros((tot,tot,tot)) ByN = np.zeros((tot,tot,tot)) BzN = np.zeros((tot,tot,tot)) ExN = np.zeros((tot,tot,tot)) EyN = np.zeros((tot,tot,tot)) EzN = np.zeros((tot,tot,tot)) Egnotte = [] ExS = [1,0,0] EyS = [0,1,0] EzS = [0,0,1] BxS = [0,1,1] ByS = [1,0,1] BzS = [1,1,0] numpyArrays =[ExN]#, EyN, EzN, BxN, ByN, BzN] FC = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0] fields = ['EX_']#, 'EY_', 'EZ_', 'BX_', 'BY_', 'BZ_'] stagger = [ExS]#, EyS, EzS, BxS, ByS, BzS] print 'im here' for field, arry, stag in zip(fields, numpyArrays, stagger): g=0 fname = folder+ field + pre + snshot + dat print fname f = open(fname, 'rb') print "pastopens" for iw in range(points): print iw But when I run it it is killed in the for cycle, it doesn't print any number: [gabriele:~/Desktop/GITcode] gbrambil% python EknotFromKostas.py im here ../NOBACKUP/heavi3/EX_104189.dat pastopens Killed does it mean that my number of points is too high? Thanks GB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading Fortran unformatted files with Python
I solved the issue. If I need more help I'll send another email. thanks GB On Mon, Jul 13, 2015 at 3:32 PM, Gabriele Brambilla < gb.gabrielebrambi...@gmail.com> wrote: > Hi, > > sorry for the confusion I understood that the Real(8) I'm using correspond > to dtype float64 in Python. > With the second method it seems to work but I get a wrong number of > elements. > > They should be grouped by 21 float64 but I don't find a number of data = > 21*m where m is an integer number. > > I think the problem is that Fortran write 4bytes at the beginning and at > the end of each file. > I found a way to remove the first 4 bytes but not the last. > > Do you have any suggestion? > > Thanks > > > On Mon, Jul 13, 2015 at 2:08 PM, Gabriele Brambilla < > gb.gabrielebrambi...@gmail.com> wrote: > >> The second method seem to work. >> But how can I know which dtype in Python corresponds to REAL in fortran? >> >> Thanks >> >> >> >> On Mon, Jul 13, 2015 at 11:29 AM, Laura Creighton wrote: >> >>> In a message of Mon, 13 Jul 2015 11:14:36 -0400, Gabriele Brambilla >>> writes: >>> >Hi, >>> > >>> >I have problems reading unformatted fortran output (binary) with python. >>> > >>> >I have a code in fortran where I write data on a file inside a cycle: >>> > >>> >write(11) x,y,z,BA01(i,j,k,1),BA01(i,j,k,2),1 >>> >BB01(i,j,k,1),BB01(i,j,k,2),2 BE01(i,j,k,1),3 >>> >EC01(i,j,k,1),EC01(i,j,k,2),4 ED01(i,j,k,1),ED01(i,j,k,2),5 >>> > EF01(i,j,k,1),6 rGH01(i,j,k,1),rGH01(i,j,k,2),7 >>> > rGI01(i,j,k,1),rGI01(i,j,k,2),8 rGJ01(i,j,k,1),1 >>> >rGL(i,j,k,2),rGM(i,j,k,2),rGN(i,j,k,2) >>> > >>> >How can I read this 21 real line by line in Python? How can I decode >>> this >>> >unit specifier 11? All the numbers are defined as REAL. >>> > >>> >thanks >>> > >>> >Gabriele >>> >___ >>> >Tutor maillist - Tutor@python.org >>> >To unsubscribe or change subscription options: >>> >https://mail.python.org/mailman/listinfo/tutor >>> >>> You want to use this package: >>> https://pypi.python.org/pypi/fortranfile >>> >>> You need to also load numpy for this to work. >>> >>> Some people cannot get this to work, and like this way of doing >>> things better: >>> >>> https://physics.ucf.edu/~dle/blog.php?id=1 >>> >>> Again, needs numpy. Write back if you need a numpy-free solution (i.e. >>> you are on Jython). >>> >>> Laura >>> >> >> > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading Fortran unformatted files with Python
Hi, sorry for the confusion I understood that the Real(8) I'm using correspond to dtype float64 in Python. With the second method it seems to work but I get a wrong number of elements. They should be grouped by 21 float64 but I don't find a number of data = 21*m where m is an integer number. I think the problem is that Fortran write 4bytes at the beginning and at the end of each file. I found a way to remove the first 4 bytes but not the last. Do you have any suggestion? Thanks On Mon, Jul 13, 2015 at 2:08 PM, Gabriele Brambilla < gb.gabrielebrambi...@gmail.com> wrote: > The second method seem to work. > But how can I know which dtype in Python corresponds to REAL in fortran? > > Thanks > > > > On Mon, Jul 13, 2015 at 11:29 AM, Laura Creighton wrote: > >> In a message of Mon, 13 Jul 2015 11:14:36 -0400, Gabriele Brambilla >> writes: >> >Hi, >> > >> >I have problems reading unformatted fortran output (binary) with python. >> > >> >I have a code in fortran where I write data on a file inside a cycle: >> > >> >write(11) x,y,z,BA01(i,j,k,1),BA01(i,j,k,2),1 >> >BB01(i,j,k,1),BB01(i,j,k,2),2 BE01(i,j,k,1),3 >> >EC01(i,j,k,1),EC01(i,j,k,2),4 ED01(i,j,k,1),ED01(i,j,k,2),5 >> > EF01(i,j,k,1),6 rGH01(i,j,k,1),rGH01(i,j,k,2),7 >> > rGI01(i,j,k,1),rGI01(i,j,k,2),8 rGJ01(i,j,k,1),1 >> >rGL(i,j,k,2),rGM(i,j,k,2),rGN(i,j,k,2) >> > >> >How can I read this 21 real line by line in Python? How can I decode this >> >unit specifier 11? All the numbers are defined as REAL. >> > >> >thanks >> > >> >Gabriele >> >___ >> >Tutor maillist - Tutor@python.org >> >To unsubscribe or change subscription options: >> >https://mail.python.org/mailman/listinfo/tutor >> >> You want to use this package: >> https://pypi.python.org/pypi/fortranfile >> >> You need to also load numpy for this to work. >> >> Some people cannot get this to work, and like this way of doing >> things better: >> >> https://physics.ucf.edu/~dle/blog.php?id=1 >> >> Again, needs numpy. Write back if you need a numpy-free solution (i.e. >> you are on Jython). >> >> Laura >> > > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading Fortran unformatted files with Python
The second method seem to work. But how can I know which dtype in Python corresponds to REAL in fortran? Thanks On Mon, Jul 13, 2015 at 11:29 AM, Laura Creighton wrote: > In a message of Mon, 13 Jul 2015 11:14:36 -0400, Gabriele Brambilla writes: > >Hi, > > > >I have problems reading unformatted fortran output (binary) with python. > > > >I have a code in fortran where I write data on a file inside a cycle: > > > >write(11) x,y,z,BA01(i,j,k,1),BA01(i,j,k,2),1 > >BB01(i,j,k,1),BB01(i,j,k,2),2 BE01(i,j,k,1),3 > >EC01(i,j,k,1),EC01(i,j,k,2),4 ED01(i,j,k,1),ED01(i,j,k,2),5 > > EF01(i,j,k,1),6 rGH01(i,j,k,1),rGH01(i,j,k,2),7 > > rGI01(i,j,k,1),rGI01(i,j,k,2),8 rGJ01(i,j,k,1),1 > >rGL(i,j,k,2),rGM(i,j,k,2),rGN(i,j,k,2) > > > >How can I read this 21 real line by line in Python? How can I decode this > >unit specifier 11? All the numbers are defined as REAL. > > > >thanks > > > >Gabriele > >___ > >Tutor maillist - Tutor@python.org > >To unsubscribe or change subscription options: > >https://mail.python.org/mailman/listinfo/tutor > > You want to use this package: > https://pypi.python.org/pypi/fortranfile > > You need to also load numpy for this to work. > > Some people cannot get this to work, and like this way of doing > things better: > > https://physics.ucf.edu/~dle/blog.php?id=1 > > Again, needs numpy. Write back if you need a numpy-free solution (i.e. > you are on Jython). > > Laura > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] reading Fortran unformatted files with Python
Hi, I have problems reading unformatted fortran output (binary) with python. I have a code in fortran where I write data on a file inside a cycle: write(11) x,y,z,BA01(i,j,k,1),BA01(i,j,k,2),1 BB01(i,j,k,1),BB01(i,j,k,2),2 BE01(i,j,k,1),3 EC01(i,j,k,1),EC01(i,j,k,2),4 ED01(i,j,k,1),ED01(i,j,k,2),5 EF01(i,j,k,1),6 rGH01(i,j,k,1),rGH01(i,j,k,2),7 rGI01(i,j,k,1),rGI01(i,j,k,2),8 rGJ01(i,j,k,1),1 rGL(i,j,k,2),rGM(i,j,k,2),rGN(i,j,k,2) How can I read this 21 real line by line in Python? How can I decode this unit specifier 11? All the numbers are defined as REAL. thanks Gabriele ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: glob and file names
oh yes! I had a problem inside the file name! (I had a lot of file to open
and I changed the one I was looking at!the one I was trying to open has a
wrong filename...)
Sorry for the misunderstanding.
Gabriele
2014-06-06 3:05 GMT-04:00 Peter Otten <__pete...@web.de>:
> Gabriele Brambilla wrote:
>
> > 2014-06-05 22:10 GMT-04:00 Peter Romfeld :
> >
> > On Friday, June 06, 2014 10:04 AM, Gabriele Brambilla wrote:
> >>>
> >>> fiLUMOname = 'Lsum_' + period + '_' + parts[2] + '_' + parts[3] + '_'
> >>> + parts[4] + '_*.dat'
> >>>
> >>> aaa = glob.glob(fiLUMOname)
> >>> print(aaa)
> >>> fiLUMO = open(aaa[0], 'r')
> >>
> >> i would do:
> >>
> >> aaa = glob.glob('Lsum_%s_%s_%s_%s_*.dat' % (period, parts[2], parts[3],
> >> parts[4]))
> >
> > thanks, it works.
>
> While Peter's way may be easier to read both approaches should give you the
> same result, assuming 'period' and the items in the 'parts' list are all
> strings. There must be an accidental change elsewhere -- maybe you changed
> the current working directory before invoking the script?
>
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] glob and file names
No, I wanted to say that Peter method works! Thanks anyway and sorry for the html. Gabriele sent from Samsung Mobile Il giorno 05/giu/2014 23:14, "Dave Angel" ha scritto: > Gabriele Brambilla Wrote in message: > > > > > (missing because you posted in html. Please tell your email program to use > text) > > Simplify your program to where it's small and self contained. You > currently have a string built up from several parts. Since you > think the problem is in the glob call, you should be able to show > it in 3 lines or so. > >filespec="f gytrr y yyrddty*.txt" > hh= glob.glo... >print (repr (hh)) > > Then specify the python version and OS. > > Since I have to guess, I'd say you had an unprintable in one of > those variables. > > > > > -- > DaveA > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Fwd: glob and file names
-- Forwarded message --
From: Gabriele Brambilla
Date: 2014-06-05 22:15 GMT-04:00
Subject: Re: [Tutor] glob and file names
To: Peter Romfeld
thanks,
it works.
Gabriele
2014-06-05 22:10 GMT-04:00 Peter Romfeld :
On Friday, June 06, 2014 10:04 AM, Gabriele Brambilla wrote:
> >
> > fiLUMOname = 'Lsum_' + period + '_' + parts[2] + '_' + parts[3] + '_'
> > + parts[4] + '_*.dat'
> >
> > aaa = glob.glob(fiLUMOname)
> > print(aaa)
> > fiLUMO = open(aaa[0], 'r')
>
> i would do:
>
> aaa = glob.glob('Lsum_%s_%s_%s_%s_*.dat' % (period, parts[2], parts[3],
> parts[4]))
>
> cheers
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] glob and file names
Hi, I'm trying to use glob to read a file of which I don't know the complete name (but only some parts). fiLUMOname = 'Lsum_' + period + '_' + parts[2] + '_' + parts[3] + '_' + parts[4] + '_*.dat' aaa = glob.glob(fiLUMOname) print(aaa) fiLUMO = open(aaa[0], 'r') where period, and the elements of parts are strings. but aaa results empty. (and so it cannot open the file) The file exist and I'm able to recover it if I write by hand inside glob.glob() the name between " " with the * where I don't know the name. How can I pass my string to glob.glob inside " "? Is there a simpler method to obtain the same result? thanks Gabriele ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] converting strings to float: strange case
solved, sorry for the disturb
http://stackoverflow.com/questions/23344345/strings-not-converting-to-float-as-expected/23344830#23344830
bye
Gabriele
2014-04-28 11:13 GMT-04:00 Gabriele Brambilla <
gb.gabrielebrambi...@gmail.com>:
> Hi,
>
> I'm trying to convert a string to a float. It seems a basic thing but I
> don't know why I'm getting this erroris
>
> Traceback (most recent call last):
> File "phresREADER.py", line 27, in
> tra = float(stri)
> ValueError: could not convert string to float:
>
> My file has this line
>
> 5.5e+000 5.5e+001 5.5e+002 5.5e+003
>
> my code is:
>
> my_line = f.readline()
> avg_energySTR = [str(i) for i in my_line.split(' ')]
> for stri in avg_energySTR:
> tra = float(stri)
>
> do you have any idea?
>
> thanks
>
> Gabriele
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] converting strings to float: strange case
Hi,
I'm trying to convert a string to a float. It seems a basic thing but I
don't know why I'm getting this erroris
Traceback (most recent call last):
File "phresREADER.py", line 27, in
tra = float(stri)
ValueError: could not convert string to float:
My file has this line
5.5e+000 5.5e+001 5.5e+002 5.5e+003
my code is:
my_line = f.readline()
avg_energySTR = [str(i) for i in my_line.split(' ')]
for stri in avg_energySTR:
tra = float(stri)
do you have any idea?
thanks
Gabriele
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] improving speed using and recalling C functions
ok Peter Otten code works (very fast),
and this is the profile
Sat Apr 12 11:15:39 2014restats
92834776 function calls in 6218.782 seconds
Ordered by: internal time
List reduced from 41 to 20 due to restriction <20>
ncalls tottime percall cumtime percall filename:lineno(function)
1 5301.641 5301.641 6218.763 6218.763 skymapsIO.py:50(mymain)
3489985 380.4690.000 452.4780.000
interpolate.py:394(_call_linear)
3489985 98.1860.000 227.2290.000
interpolate.py:454(_check_bounds)
6979970 96.5670.000 96.5670.000 {method 'reduce' of
'numpy.ufunc'objects}
3489985 44.8530.000 738.1350.000 interpolate.py:443(_evaluate)
7677978 41.0100.000 41.0100.000 {numpy.core.multiarray.array}
5 40.4308.086 40.6218.124 npyio.py:882(savetxt)
3489985 26.9520.000 26.9520.000 {method 'clip' of
'numpy.ndarray'objects}
3489985 24.7490.000 24.7490.000 {method 'searchsorted' of
'numpy.ndarray' objects}
3489985 22.4570.000 828.2380.000 polyint.py:37(__call__)
6979970 19.7200.000 116.2870.000 _methods.py:31(_any)
6979980 15.3300.000 35.0920.000 numeric.py:392(asarray)
3489985 14.8470.000 45.0390.000 polyint.py:57(_prepare_x)
3489990 12.9040.000 12.9040.000 {method 'reshape' of
'numpy.ndarray' objects}
6979970 12.7570.000 129.0440.000 {method 'any' of
'numpy.ndarray' objects}
3489985 11.6240.000 11.6240.000 {method 'astype' of
'numpy.ndarray' objects}
3489985 10.0770.000 10.0770.000 {numpy.core.multiarray.empty}
34899859.9450.000 22.6070.000 polyint.py:63(_finish_y)
34899857.0510.0007.0510.000 {method 'ravel' of
'numpy.ndarray' objects}
6979986.7460.0006.7460.000 {zip}
So I think that in this way it's ok.
Thank you all very much,
Gabriele
p.s: I didn't know this way to write: is there a tutorial for this kind of
operations?
kap = np.empty(x.shape)
sel = x >= 10.0
zsel = x[sel]
kap[sel] = 1.2533 * np.sqrt(zsel)*np.exp(-zsel)
sel = x < 0.001
zsel = x[sel]
kap[sel] = (2.1495 * np.exp(0.3 * np.log(zsel))
- 1.8138 * zsel)
sel = ~ ((x >= 10.0) | (x < 0.001))
zsel = x[sel]
result = my_inter(np.log10(zsel))
kap[sel] = 10**result
2014-04-12 9:34 GMT-04:00 Gabriele Brambilla :
> Ok, i just run Peter's code and it seems really faster...I hope to don't
> mistake this time!
>
> Thanks
>
> Gabriele
>
> sent from Samsung Mobile
> Il giorno 12/apr/2014 08:22, "Gabriele Brambilla" <
> gb.gabrielebrambi...@gmail.com> ha scritto:
>
> Ok guys,
>> I'm not expert about profile but help me to look at it.
>> this one is for 715853 elements (to multiply by 5, and for each of this
>> N*5 there is a loop of 200 times)
>>
>> Sat Apr 12 04:58:50 2014restats
>>
>> 9636507991 function calls in 66809.764 seconds
>>
>>Ordered by: internal time
>>List reduced from 47 to 20 due to restriction <20>
>>
>>ncallstottimepercallcumtime percall
>> filename:lineno(function)
>> 1 13548.507 13548.507 66809.692 66809.692
>> skymapsI.py:44(mymain)
>> 125800544 13539.337 0.00015998.9250.000
>> interpolate.py:394(_call_linear)
>> 880603808 5353.3820.000 5353.3820.000
>> {numpy.core.multiarray.array}
>> 715853000 4998.7400.000 52861.634 0.000
>> instruments.py:10(kappa)
>> 251601088 4550.9400.000 4550.9400.000{method 'reduce'
>> of 'numpy.ufunc' objects}
>> 125800544 4312.0780.00010163.6140.000
>> interpolate.py:454(_check_bounds)
>> 125800544 2944.1260.00014182.9170.000
>> interpolate.py:330(__init__)
>> 125800544 2846.5770.000 29484.2480.000
>> interpolate.py:443(_evaluate)
>> 125800544 1665.8520.0006000.6030.000
>> polyint.py:82(_set_yi)
>> 125800544 1039.4550.000 1039.4550.000 {method 'clip' of
>> 'numpy.ndarray' objects}
>> 251601088 944.8480.000 944.8480.000 {method 'reshape'
>> of 'numpy.ndarray' objects}
>> 251601088 922.9280.000 1651.218
>> 0.000numerictypes.py:735(issubdtype)
>> 503202176 897.0440.000 3434.7680.000
>> n
Re: [Tutor] improving speed using and recalling C functions
Ok, i just run Peter's code and it seems really faster...I hope to don't
mistake this time!
Thanks
Gabriele
sent from Samsung Mobile
Il giorno 12/apr/2014 08:22, "Gabriele Brambilla" <
gb.gabrielebrambi...@gmail.com> ha scritto:
> Ok guys,
> I'm not expert about profile but help me to look at it.
> this one is for 715853 elements (to multiply by 5, and for each of this
> N*5 there is a loop of 200 times)
>
> Sat Apr 12 04:58:50 2014restats
>
> 9636507991 function calls in 66809.764 seconds
>
>Ordered by: internal time
>List reduced from 47 to 20 due to restriction <20>
>
>ncallstottimepercallcumtime percall
> filename:lineno(function)
> 1 13548.507 13548.507 66809.692 66809.692
> skymapsI.py:44(mymain)
> 125800544 13539.337 0.00015998.9250.000
> interpolate.py:394(_call_linear)
> 880603808 5353.3820.000 5353.3820.000
> {numpy.core.multiarray.array}
> 715853000 4998.7400.000 52861.634 0.000
> instruments.py:10(kappa)
> 251601088 4550.9400.000 4550.9400.000{method 'reduce'
> of 'numpy.ufunc' objects}
> 125800544 4312.0780.00010163.6140.000
> interpolate.py:454(_check_bounds)
> 125800544 2944.1260.00014182.9170.000
> interpolate.py:330(__init__)
> 125800544 2846.5770.000 29484.2480.000
> interpolate.py:443(_evaluate)
> 125800544 1665.8520.0006000.6030.000 polyint.py:82(_set_yi)
> 125800544 1039.4550.000 1039.4550.000 {method 'clip' of
> 'numpy.ndarray' objects}
> 251601088 944.8480.000 944.8480.000 {method 'reshape'
> of 'numpy.ndarray' objects}
> 251601088 922.9280.000 1651.218
> 0.000numerictypes.py:735(issubdtype)
> 503202176 897.0440.000 3434.7680.000
> numeric.py:392(asarray)
> 125800544 816.4010.000 32242.4810.000
> polyint.py:37(__call__)
> 251601088 787.5930.000 5338.5330.000 _methods.py:31(_any)
> 125800544 689.7790.000 1989.1010.000
> polyint.py:74(_reshape_yi)
> 125800544 638.9460.000 638.9460.000 {method
> 'searchsorted' of 'numpy.ndarray' objects}
> 125800544 606.7780.000 2257.9960.000
> polyint.py:102(_set_dtype)
> 125800544 598.0000.0006598.6020.000
> polyint.py:30(__init__)
> 629002720 549.3580.000 549.3580.000 {issubclass}
>
>
> looking at tottime it seems that skymaps mymain() and interpolate take the
> same big amount of time...right?
>
> So it's true that I have to slow down mymain() but interpolate is a
> problem too!
>
> do you agree with me?
>
> Now I will read Peter Otten's code and run the new simulation with it
>
> thanks
>
> Gabriele
>
>
> 2014-04-12 6:21 GMT-04:00 Peter Otten <__pete...@web.de>:
>
>> Gabriele Brambilla wrote:
>>
>> > Ok guys, when I wrote that email I was excited for the apparent speed
>> > increasing (it was jumping the bottleneck for loop for the reason peter
>> > otten outlined).
>> > Now, instead the changes, the speed is not improved (the code still
>> > running from this morning and it's at one forth of the dataset).
>> >
>> > What can I do to speed it up?
>>
>> Not as easy as I had hoped and certainly not as pretty, here's my
>> modification of the code you sent me. What makes it messy is that
>> I had to inline your kappa() function; my first attempt with
>> numpy.vectorize() didn't help much. There is still stuff in the
>> 'for gammar...' loop that doesn't belong there, but I decided it
>> was time for me to stop ;)
>>
>> Note that it may still be worthwhile to consult a numpy expert
>> (which I'm not!).
>>
>> from scipy import stats
>> import matplotlib.pyplot as plt
>> from scipy import optimize
>> from matplotlib import colors, ticker, cm
>> import numpy as np
>>
>> phamin = 0
>> phamax = 2*pi
>> obamin = 0
>> obamax = pi
>> npha = 100
>> nobs = 181
>> stepPHA = (phamax-phamin)/npha
>> stepOB = (obamax-obamin)/nobs
>> freq = 10
>> c = 2.9979*(10**(10))
>> e = 4.8032*(10**(-10))
>> hcut = 1.0546*(10**(-27))
>> eVtoErg = 1.6022*(10**(-12))
>>
>> from math import *
>> import numpy as np
>> from scipy.interpolate import interp1d
>>
>> kaparg = [
>> -3.0, -2.0, -1.52287875, -1.22184875, -1.0, -0.69897
Re: [Tutor] improving speed using and recalling C functions
Ok guys,
I'm not expert about profile but help me to look at it.
this one is for 715853 elements (to multiply by 5, and for each of this N*5
there is a loop of 200 times)
Sat Apr 12 04:58:50 2014restats
9636507991 function calls in 66809.764 seconds
Ordered by: internal time
List reduced from 47 to 20 due to restriction <20>
ncallstottimepercallcumtime percall
filename:lineno(function)
1 13548.507 13548.507 66809.692 66809.692
skymapsI.py:44(mymain)
125800544 13539.337 0.00015998.9250.000
interpolate.py:394(_call_linear)
880603808 5353.3820.000 5353.3820.000
{numpy.core.multiarray.array}
715853000 4998.7400.000 52861.634 0.000
instruments.py:10(kappa)
251601088 4550.9400.000 4550.9400.000{method 'reduce'
of 'numpy.ufunc' objects}
125800544 4312.0780.00010163.6140.000
interpolate.py:454(_check_bounds)
125800544 2944.1260.00014182.9170.000
interpolate.py:330(__init__)
125800544 2846.5770.000 29484.2480.000
interpolate.py:443(_evaluate)
125800544 1665.8520.0006000.6030.000 polyint.py:82(_set_yi)
125800544 1039.4550.000 1039.4550.000 {method 'clip' of
'numpy.ndarray' objects}
251601088 944.8480.000 944.8480.000 {method 'reshape' of
'numpy.ndarray' objects}
251601088 922.9280.000 1651.2180.000
numerictypes.py:735(issubdtype)
503202176 897.0440.000 3434.7680.000
numeric.py:392(asarray)
125800544 816.4010.000 32242.4810.000
polyint.py:37(__call__)
251601088 787.5930.000 5338.5330.000 _methods.py:31(_any)
125800544 689.7790.000 1989.1010.000
polyint.py:74(_reshape_yi)
125800544 638.9460.000 638.9460.000 {method
'searchsorted' of 'numpy.ndarray' objects}
125800544 606.7780.000 2257.9960.000
polyint.py:102(_set_dtype)
125800544 598.0000.0006598.6020.000
polyint.py:30(__init__)
629002720 549.3580.000 549.3580.000 {issubclass}
looking at tottime it seems that skymaps mymain() and interpolate take the
same big amount of time...right?
So it's true that I have to slow down mymain() but interpolate is a problem
too!
do you agree with me?
Now I will read Peter Otten's code and run the new simulation with it
thanks
Gabriele
2014-04-12 6:21 GMT-04:00 Peter Otten <__pete...@web.de>:
> Gabriele Brambilla wrote:
>
> > Ok guys, when I wrote that email I was excited for the apparent speed
> > increasing (it was jumping the bottleneck for loop for the reason peter
> > otten outlined).
> > Now, instead the changes, the speed is not improved (the code still
> > running from this morning and it's at one forth of the dataset).
> >
> > What can I do to speed it up?
>
> Not as easy as I had hoped and certainly not as pretty, here's my
> modification of the code you sent me. What makes it messy is that
> I had to inline your kappa() function; my first attempt with
> numpy.vectorize() didn't help much. There is still stuff in the
> 'for gammar...' loop that doesn't belong there, but I decided it
> was time for me to stop ;)
>
> Note that it may still be worthwhile to consult a numpy expert
> (which I'm not!).
>
> from scipy import stats
> import matplotlib.pyplot as plt
> from scipy import optimize
> from matplotlib import colors, ticker, cm
> import numpy as np
>
> phamin = 0
> phamax = 2*pi
> obamin = 0
> obamax = pi
> npha = 100
> nobs = 181
> stepPHA = (phamax-phamin)/npha
> stepOB = (obamax-obamin)/nobs
> freq = 10
> c = 2.9979*(10**(10))
> e = 4.8032*(10**(-10))
> hcut = 1.0546*(10**(-27))
> eVtoErg = 1.6022*(10**(-12))
>
> from math import *
> import numpy as np
> from scipy.interpolate import interp1d
>
> kaparg = [
> -3.0, -2.0, -1.52287875, -1.22184875, -1.0, -0.69897,
> -0.52287875, -0.39794001, -0.30103, -0.22184875,
> -0.15490196, 0.0, 0.30103, 0.60205999, 0.69897,
> 0.77815125, 0.90308999, 1.0]
>
> kapval = [
> -0.6716204 , -0.35163999, -0.21183163, -0.13489603,
> -0.0872467 , -0.04431225, -0.03432803, -0.04335142,
> -0.05998184, -0.08039898, -0.10347378, -0.18641901,
> -0.52287875, -1.27572413, -1.66958623, -2.07314329,
> -2.88941029, -3.7212464 ]
>
> my_inter = interp1d(kaparg, kapval)
>
> def LEstep(n):
> Emin = 10**6
> Emax = 5*(10**10)
> Lemin = log10(Emin)
> Lemax = log10(Emax)
> stepE = (Lemax-Lemin)/n
> return stepE, n, Lemin, Lemax
>
> def mymain(stepENE, nex, Lemin, Lemax, freq):
>
Re: [Tutor] improving speed using and recalling C functions
Ok guys, when I wrote that email I was excited for the apparent speed increasing (it was jumping the bottleneck for loop for the reason peter otten outlined). Now, instead the changes, the speed is not improved (the code still running from this morning and it's at one forth of the dataset). What can I do to speed it up? Thanks Gabriele sent from Samsung Mobile Il giorno 11/apr/2014 17:00, "Danny Yoo" ha scritto: > On Fri, Apr 11, 2014 at 1:01 PM, Gabriele Brambilla > wrote: > > Yes, > > but I want to make a C extension to run faster a function from > > scipy.interpolate (interp1d) > > > Just to emphasis: I believe your goal should be: "I want to make my > program fast." > > Your goal should probably not be: "I want to write a C extension". > I'm not saying that writing a C extension is necessarily wrong, and it > may be that writing a C extension will make your program fast. But > this approach may not be the easiest or most maintainable approach to > improving your program's performance. > > Using C is not without its costs and risks. As soon as you are in C > territory, the seat belts are off. Just recall the craziness that > happened this week with regards to programs written in low-level > languages like C. Explicitly: http://heartbleed.com. If you are > writing with C, you have to be very, very delicate with your code. > Experts get it wrong, with severe consequences. > > This is why the focus on C extensions to get speed disturbs me so > much: it assumes that C is a safe language to use. It's not, > especially for beginners. We should strongly discourage low-level > languages unless there is some overriding concern. For scientific > calculations like the ones you are doing, you should place a premium > on getting a right answer, and not just a fast answer. > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Fwd: improving speed using and recalling C functions
I forget the reply all -- Forwarded message -- From: Gabriele Brambilla Date: 2014-04-11 16:03 GMT-04:00 Subject: Re: [Tutor] improving speed using and recalling C functions To: Peter Otten <__pete...@web.de> you are right. probably this is the problem. thanks Gabriele 2014-04-11 15:35 GMT-04:00 Peter Otten <__pete...@web.de>: Gabriele Brambilla wrote: > > > ok, it seems that the code don't enter in this for loop > > > > for gammar, MYMAP in zip(gmlis, MYMAPS): > > > > I don't understand why. > > You have two variables with similar names, gmlis and gmils: > > >> gmlis = [] > > >> gmils=[my_parts[7], my_parts[8], my_parts[9], > >> my_parts[10], my_parts[11]] > > >> for gammar, MYMAP in zip(gmlis, MYMAPS): > > I assume you wanted > >for gammar, MYMAP in zip(gmils, MYMAPS): > > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] improving speed using and recalling C functions
Yes, but I want to make a C extension to run faster a function from scipy.interpolate (interp1d) It woulldn't change anything? thanks Gabriele 2014-04-11 14:47 GMT-04:00 Alan Gauld : > On 11/04/14 09:59, Peter Otten wrote: > >> Gabriele Brambilla wrote: >> >> Anyway I would like to try to speed it up using C functions >>> >> ... >> >> posted looks like it has great potential for speed-up by replacing the >> inner >> loops with numpy array operations. >> > > And in case its not obvious much(most?) of numPy consists > of C functions. So by using NumPy you are usually using > C code not native Python. > > That's what I alluded to in my first post on this thread: > there are other libraries who have trod this route before > and done the work for you. > > HTH > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.flickr.com/photos/alangauldphotos > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] improving speed using and recalling C functions
this is the profile for a sample of 1000 elements
Fri Apr 11 10:21:21 2014restats
31594963 function calls in 103.708 seconds
Ordered by: internal time
List reduced from 47 to 20 due to restriction <20>
ncalls tottime percall cumtime percall filename:lineno(function)
1 57.133 57.133 103.692 103.692 skymapsI.py:44(mymain)
1768329.8980.000 11.7100.000
interpolate.py:394(_call_linear)
181010007.8080.0007.8080.000 {method 'write' of 'file'
objects}
12378243.7940.0003.7940.000 {numpy.core.multiarray.array}
10010003.6100.000 38.3830.000 instruments.py:10(kappa)
3536643.3140.0003.3140.000 {method 'reduce' of
'numpy.ufunc'
objects}
1768323.1570.0007.4280.000
interpolate.py:454(_check_bounds)
1768322.0740.000 10.2130.000 interpolate.py:330(__init__)
1768322.0530.000 21.5220.000 interpolate.py:443(_evaluate)
1768321.2530.0004.4040.000 polyint.py:82(_set_yi)
1768320.7690.0000.7690.000 {method 'clip' of
'numpy.ndarray'
objects}
3536640.7060.0000.7060.000 {method 'reshape' of
'numpy.ndarra
y' objects}
3536640.6670.0001.2050.000
numerictypes.py:735(issubdtype)
7073280.6370.0002.4510.000 numeric.py:392(asarray)
1768320.6010.000 23.5550.000 polyint.py:37(__call__)
3536640.5690.0003.8830.000 _methods.py:31(_any)
1768320.5040.0001.4290.000 polyint.py:74(_reshape_yi)
1768320.4730.0000.4730.000 {method 'searchsorted' of
'numpy.n
darray' objects}
1768320.4400.0001.6450.000 polyint.py:102(_set_dtype)
176832 0.426 0.000 4.8300.000 polyint.py:30(__init__)
thanks
Gabriele
2014-04-11 10:18 GMT-04:00 Gabriele Brambilla <
gb.gabrielebrambi...@gmail.com>:
> ok
> modifying the for in this way (zipping an array of matrix drive it crazy)
> it works
>
> dko=0
>
> for gammar in gmils:
>
>
> omC = (1.5)*(gammar**3)*c/(rho*rlc)
>
> gig = omC*hcut/eVtoErg
>
>
> #check the single emission
>
>
>
> for w in eel:
>
> omega =
> (10**(w*stepENE+Lemin))*eVtoErg/hcut
>
> x = omega/omC
>
> kap = instruments.kappa(x)
>
> Iom = (1.732050808/c)*(e**2)*gammar*kap
> #jackson dI/domega
>
> P = Iom*(c/(rho*rlc))/(2*pi) #jackson P
>
> phps = P/(hcut*omega) #photons per second
>
> www = phps/(stepPHA*sin(zobs)*stepOB)
>
> MYMAPS[dko][i,j,w] += www
>
> dko += 1
>
>
>
> count = count + 1
>
> Now I will tell you how much it takes.
>
> Thanks
>
> Gabriele
>
>
>
> 2014-04-11 10:05 GMT-04:00 Gabriele Brambilla <
> gb.gabrielebrambi...@gmail.com>:
>
> ok, it seems that the code don't enter in this for loop
>>
>> for gammar, MYMAP in zip(gmlis, MYMAPS):
>>
>> I don't understand why.
>>
>> Thanks
>>
>> Gabriele
>>
>>
>> 2014-04-11 9:56 GMT-04:00 Gabriele Brambilla <
>> gb.gabrielebrambi...@gmail.com>:
>>
>> Hi, I'm sorry but there is a big problem.
>>> the code is producing empty file.dat.
>>>
>>> I think it's because of this that previously I have done that strange
>>> trick of myinternet...
>>>
>>> So:
>>>
>>> for my_line in open('data.dat'):
>>>
>>> myinternet = []
>>>
>>> gmlis = []
>>>
>>> print('reading the line', count, '/599378')
>>>
>>> my_parts = [float(i) for i in my_line.split()]
>>>
>>> phase = my_parts[4]
>>>
>>> zobs = my_parts[5]
>>>
>>> rho = my_parts[6]
>>>
>>>
>>>
>>> gmils=[my_parts[7], my_parts[8], my_parts[9],
>>> my_parts[10], my_parts[11]]
>>>
>>>
>>>
>>> i = int((phase-phamin)/stepPHA)
>>>
>>> j = int((zobs-obamin)/stepOB)
>>>
>>>
>>>
>>>
Re: [Tutor] improving speed using and recalling C functions
ok
modifying the for in this way (zipping an array of matrix drive it crazy)
it works
dko=0
for gammar in gmils:
omC = (1.5)*(gammar**3)*c/(rho*rlc)
gig = omC*hcut/eVtoErg
#check the single emission
for w in eel:
omega =
(10**(w*stepENE+Lemin))*eVtoErg/hcut
x = omega/omC
kap = instruments.kappa(x)
Iom = (1.732050808/c)*(e**2)*gammar*kap
#jackson dI/domega
P = Iom*(c/(rho*rlc))/(2*pi) #jackson P
phps = P/(hcut*omega) #photons per second
www = phps/(stepPHA*sin(zobs)*stepOB)
MYMAPS[dko][i,j,w] += www
dko += 1
count = count + 1
Now I will tell you how much it takes.
Thanks
Gabriele
2014-04-11 10:05 GMT-04:00 Gabriele Brambilla <
gb.gabrielebrambi...@gmail.com>:
> ok, it seems that the code don't enter in this for loop
>
> for gammar, MYMAP in zip(gmlis, MYMAPS):
>
> I don't understand why.
>
> Thanks
>
> Gabriele
>
>
> 2014-04-11 9:56 GMT-04:00 Gabriele Brambilla <
> gb.gabrielebrambi...@gmail.com>:
>
> Hi, I'm sorry but there is a big problem.
>> the code is producing empty file.dat.
>>
>> I think it's because of this that previously I have done that strange
>> trick of myinternet...
>>
>> So:
>>
>> for my_line in open('data.dat'):
>>
>> myinternet = []
>>
>> gmlis = []
>>
>> print('reading the line', count, '/599378')
>>
>> my_parts = [float(i) for i in my_line.split()]
>>
>> phase = my_parts[4]
>>
>> zobs = my_parts[5]
>>
>> rho = my_parts[6]
>>
>>
>>
>> gmils=[my_parts[7], my_parts[8], my_parts[9],
>> my_parts[10], my_parts[11]]
>>
>>
>>
>> i = int((phase-phamin)/stepPHA)
>>
>> j = int((zobs-obamin)/stepOB)
>>
>>
>>
>> for gammar, MYMAP in zip(gmlis, MYMAPS):
>>
>>
>>
>> omC = (1.5)*(gammar**3)*c/(rho*rlc)
>>
>> gig = omC*hcut/eVtoErg
>>
>> #check the single emission
>>
>>
>>
>> for w in eel:
>>
>> omega =
>> (10**(w*stepENE+Lemin))*eVtoErg/hcut
>>
>> x = omega/omC
>>
>> kap = instruments.kappa(x)
>>
>> Iom = (1.732050808/c)*(e**2)*gammar*kap
>> #jackson dI/domega
>>
>> P = Iom*(c/(rho*rlc))/(2*pi) #jackson P
>>
>> phps = P/(hcut*omega) #photons per second
>>
>> www = phps/(stepPHA*sin(zobs)*stepOB)
>>
>> MYMAP[i,j,w] += www
>>
>>
>>
>> count = count + 1
>>
>> when I exit here the MYMAP matrix has all the cells = 0.
>>
>> Now I will try to fiugre it out why.
>>
>> Thanks
>>
>> Gabriele
>>
>>
>>
>> 2014-04-11 9:20 GMT-04:00 Gabriele Brambilla <
>> gb.gabrielebrambi...@gmail.com>:
>>
>> Hi Danny,
>>> I'm quiet impressed.
>>> the program takes near 30 minutes instead of more than 8 hours!
>>>
>>> this is the profile:
>>> Fri Apr 11 09:14:04 2014restats
>>>
>>> 19532732 function calls in 2105.024 seconds
>>>
>>>Ordered by: internal time
>>>
>>>ncalls tottime percall cumtime percall filename:lineno(function)
>>> 1 2087.606 2087.606 2105.006 2105.006 skymapsI.py:44(mymain)
>>> 18101000 12.7570.000 12.7570.000 {method 'write' of 'file'
>>> objects}
>>>
>>>7158533.4730.0003.4730.000 {method 'split' of 'str'
>>> objects}
>>>7158541.1620.0001.1620.000 {zip}
>>> 10.0180.018 2105.024 2105.024 :1()
>>> 60.0060.0010.0060.001 {open}
>>> 50.0020.0000.0020.000 {method
Re: [Tutor] improving speed using and recalling C functions
ok, it seems that the code don't enter in this for loop
for gammar, MYMAP in zip(gmlis, MYMAPS):
I don't understand why.
Thanks
Gabriele
2014-04-11 9:56 GMT-04:00 Gabriele Brambilla :
> Hi, I'm sorry but there is a big problem.
> the code is producing empty file.dat.
>
> I think it's because of this that previously I have done that strange
> trick of myinternet...
>
> So:
>
> for my_line in open('data.dat'):
>
> myinternet = []
>
> gmlis = []
>
> print('reading the line', count, '/599378')
>
> my_parts = [float(i) for i in my_line.split()]
>
> phase = my_parts[4]
>
> zobs = my_parts[5]
>
> rho = my_parts[6]
>
>
>
> gmils=[my_parts[7], my_parts[8], my_parts[9],
> my_parts[10], my_parts[11]]
>
>
>
> i = int((phase-phamin)/stepPHA)
>
> j = int((zobs-obamin)/stepOB)
>
>
>
> for gammar, MYMAP in zip(gmlis, MYMAPS):
>
>
>
> omC = (1.5)*(gammar**3)*c/(rho*rlc)
>
> gig = omC*hcut/eVtoErg
>
> #check the single emission
>
>
>
> for w in eel:
>
> omega =
> (10**(w*stepENE+Lemin))*eVtoErg/hcut
>
> x = omega/omC
>
> kap = instruments.kappa(x)
>
> Iom = (1.732050808/c)*(e**2)*gammar*kap
> #jackson dI/domega
>
> P = Iom*(c/(rho*rlc))/(2*pi) #jackson P
>
> phps = P/(hcut*omega) #photons per second
>
> www = phps/(stepPHA*sin(zobs)*stepOB)
>
> MYMAP[i,j,w] += www
>
>
>
> count = count + 1
>
> when I exit here the MYMAP matrix has all the cells = 0.
>
> Now I will try to fiugre it out why.
>
> Thanks
>
> Gabriele
>
>
>
> 2014-04-11 9:20 GMT-04:00 Gabriele Brambilla <
> gb.gabrielebrambi...@gmail.com>:
>
> Hi Danny,
>> I'm quiet impressed.
>> the program takes near 30 minutes instead of more than 8 hours!
>>
>> this is the profile:
>> Fri Apr 11 09:14:04 2014restats
>>
>> 19532732 function calls in 2105.024 seconds
>>
>>Ordered by: internal time
>>
>>ncalls tottime percall cumtime percall filename:lineno(function)
>> 1 2087.606 2087.606 2105.006 2105.006 skymapsI.py:44(mymain)
>> 18101000 12.7570.000 12.7570.000 {method 'write' of 'file'
>> objects}
>>
>>7158533.4730.0003.4730.000 {method 'split' of 'str'
>> objects}
>>7158541.1620.0001.1620.000 {zip}
>> 10.0180.018 2105.024 2105.024 :1()
>> 60.0060.0010.0060.001 {open}
>> 50.0020.0000.0020.000 {method 'close' of 'file'
>> objects}
>>
>> 10.0000.0000.0000.000 function_base.py:8(linspace)
>> 50.0000.0000.0000.000
>> {numpy.core.multiarray.zeros}
>> 10.0000.0000.0000.000
>> function_base.py:93(logspace)
>> 10.0000.0000.0000.000
>> {numpy.core.multiarray.arange}
>> 30.0000.0000.0000.000 {range}
>> 10.0000.0000.0000.000 {method 'disable' of
>> '_lsprof.Prof
>> iler' objects}
>>
>> I hope to have similar problems in the future to learn better how to do
>> with them!
>> but in the profile I don't see any operation regarding reading the file
>> or the mathematical operations...are them hidden in mymain()?
>>
>> thanks
>>
>> Gabriele
>>
>>
>>
>>
>> 2014-04-10 21:38 GMT-04:00 Danny Yoo :
>>
>> > Comment: You are looping over your sliced eel five times. Do you
>>> >need to? I like eel salad a great deal, as well, but, how about:
>>> >
>>> >
>>> >for k in eel:
>>> >MYMAP1[i, j, k] = MYMAP1[i, j, k] + myinternet[oo]
>>> >MYMAP2[i, j, k] = MYMAP2[i, j, k] + myinternet[oo]
>>> >MYMAP3[i, j, k] = MYMAP3[i, j, k] + myinternet[oo]
>>> >MYMAP4[i, j, k] = MYMAP4[i, j, k] + myinternet[oo]
>>> >MYMAP5[i, j, k] = MYMAP5[i, j, k] + myinternet[oo]
>>> >oo = oo + 1
>>>
>>>
>>> Hi Gabriele,
>>>
>>> Also note that, when Martin looked at this part of the code, he
>>> unfortunately misinterpreted its effect; Martin's proposed rewrite
>>> here does not preserve the meaning of the original code. But rather
>>> than wag my finger at how Martin interpreted the code, I'd rather make
>>> the observation that this is a warning sign that the original code
>>> here was not easy to understand.
>>>
>>
>>
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] improving speed using and recalling C functions
Hi, I'm sorry but there is a big problem.
the code is producing empty file.dat.
I think it's because of this that previously I have done that strange trick
of myinternet...
So:
for my_line in open('data.dat'):
myinternet = []
gmlis = []
print('reading the line', count, '/599378')
my_parts = [float(i) for i in my_line.split()]
phase = my_parts[4]
zobs = my_parts[5]
rho = my_parts[6]
gmils=[my_parts[7], my_parts[8], my_parts[9], my_parts[10],
my_parts[11]]
i = int((phase-phamin)/stepPHA)
j = int((zobs-obamin)/stepOB)
for gammar, MYMAP in zip(gmlis, MYMAPS):
omC = (1.5)*(gammar**3)*c/(rho*rlc)
gig = omC*hcut/eVtoErg
#check the single emission
for w in eel:
omega =
(10**(w*stepENE+Lemin))*eVtoErg/hcut
x = omega/omC
kap = instruments.kappa(x)
Iom = (1.732050808/c)*(e**2)*gammar*kap
#jackson dI/domega
P = Iom*(c/(rho*rlc))/(2*pi) #jackson P
phps = P/(hcut*omega) #photons per second
www = phps/(stepPHA*sin(zobs)*stepOB)
MYMAP[i,j,w] += www
count = count + 1
when I exit here the MYMAP matrix has all the cells = 0.
Now I will try to fiugre it out why.
Thanks
Gabriele
2014-04-11 9:20 GMT-04:00 Gabriele Brambilla :
> Hi Danny,
> I'm quiet impressed.
> the program takes near 30 minutes instead of more than 8 hours!
>
> this is the profile:
> Fri Apr 11 09:14:04 2014restats
>
> 19532732 function calls in 2105.024 seconds
>
>Ordered by: internal time
>
>ncalls tottime percall cumtime percall filename:lineno(function)
> 1 2087.606 2087.606 2105.006 2105.006 skymapsI.py:44(mymain)
> 18101000 12.7570.000 12.7570.000 {method 'write' of 'file'
> objects}
>
>7158533.4730.0003.4730.000 {method 'split' of 'str'
> objects}
>7158541.1620.0001.1620.000 {zip}
> 10.0180.018 2105.024 2105.024 :1()
> 60.0060.0010.0060.001 {open}
> 50.0020.0000.0020.000 {method 'close' of 'file'
> objects}
>
> 10.0000.0000.0000.000 function_base.py:8(linspace)
> 50.0000.0000.0000.000 {numpy.core.multiarray.zeros}
> 10.0000.0000.0000.000 function_base.py:93(logspace)
> 10.0000.0000.0000.000
> {numpy.core.multiarray.arange}
> 30.0000.0000.0000.000 {range}
> 10.0000.0000.0000.000 {method 'disable' of
> '_lsprof.Prof
> iler' objects}
>
> I hope to have similar problems in the future to learn better how to do
> with them!
> but in the profile I don't see any operation regarding reading the file or
> the mathematical operations...are them hidden in mymain()?
>
> thanks
>
> Gabriele
>
>
>
>
> 2014-04-10 21:38 GMT-04:00 Danny Yoo :
>
> > Comment: You are looping over your sliced eel five times. Do you
>> >need to? I like eel salad a great deal, as well, but, how about:
>> >
>> >
>> >for k in eel:
>> >MYMAP1[i, j, k] = MYMAP1[i, j, k] + myinternet[oo]
>> >MYMAP2[i, j, k] = MYMAP2[i, j, k] + myinternet[oo]
>> >MYMAP3[i, j, k] = MYMAP3[i, j, k] + myinternet[oo]
>> >MYMAP4[i, j, k] = MYMAP4[i, j, k] + myinternet[oo]
>> >MYMAP5[i, j, k] = MYMAP5[i, j, k] + myinternet[oo]
>> >oo = oo + 1
>>
>>
>> Hi Gabriele,
>>
>> Also note that, when Martin looked at this part of the code, he
>> unfortunately misinterpreted its effect; Martin's proposed rewrite
>> here does not preserve the meaning of the original code. But rather
>> than wag my finger at how Martin interpreted the code, I'd rather make
>> the observation that this is a warning sign that the original code
>> here was not easy to understand.
>>
>
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] improving speed using and recalling C functions
Hi Danny,
I'm quiet impressed.
the program takes near 30 minutes instead of more than 8 hours!
this is the profile:
Fri Apr 11 09:14:04 2014restats
19532732 function calls in 2105.024 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
1 2087.606 2087.606 2105.006 2105.006 skymapsI.py:44(mymain)
18101000 12.7570.000 12.7570.000 {method 'write' of 'file'
objects}
7158533.4730.0003.4730.000 {method 'split' of 'str'
objects}
7158541.1620.0001.1620.000 {zip}
10.0180.018 2105.024 2105.024 :1()
60.0060.0010.0060.001 {open}
50.0020.0000.0020.000 {method 'close' of 'file'
objects}
10.0000.0000.0000.000 function_base.py:8(linspace)
50.0000.0000.0000.000 {numpy.core.multiarray.zeros}
10.0000.0000.0000.000 function_base.py:93(logspace)
10.0000.0000.0000.000 {numpy.core.multiarray.arange}
30.0000.0000.0000.000 {range}
10.0000.0000.0000.000 {method 'disable' of
'_lsprof.Prof
iler' objects}
I hope to have similar problems in the future to learn better how to do
with them!
but in the profile I don't see any operation regarding reading the file or
the mathematical operations...are them hidden in mymain()?
thanks
Gabriele
2014-04-10 21:38 GMT-04:00 Danny Yoo :
> > Comment: You are looping over your sliced eel five times. Do you
> >need to? I like eel salad a great deal, as well, but, how about:
> >
> >
> >for k in eel:
> >MYMAP1[i, j, k] = MYMAP1[i, j, k] + myinternet[oo]
> >MYMAP2[i, j, k] = MYMAP2[i, j, k] + myinternet[oo]
> >MYMAP3[i, j, k] = MYMAP3[i, j, k] + myinternet[oo]
> >MYMAP4[i, j, k] = MYMAP4[i, j, k] + myinternet[oo]
> >MYMAP5[i, j, k] = MYMAP5[i, j, k] + myinternet[oo]
> >oo = oo + 1
>
>
> Hi Gabriele,
>
> Also note that, when Martin looked at this part of the code, he
> unfortunately misinterpreted its effect; Martin's proposed rewrite
> here does not preserve the meaning of the original code. But rather
> than wag my finger at how Martin interpreted the code, I'd rather make
> the observation that this is a warning sign that the original code
> here was not easy to understand.
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] improving speed using and recalling C functions
I think I have Cython already installed with Anaconda.
How it works?
Thanks
Gabriele
2014-04-11 8:16 GMT-04:00 Albert-Jan Roskam :
>
> > From: Gabriele Brambilla
> >To: Danny Yoo
> >Cc: python tutor
> >Sent: Friday, April 11, 2014 5:30 AM
> >Subject: Re: [Tutor] improving speed using and recalling C functions
> >
> >
> >
> >Hi Danny,
> >I followed your suggestion.
> >Tomorrow morning I will run this new version of the code.
> >
> >
> >Now using a sample of 81 elements (instead of 60) the profile returns:
> >
> >
> >Thu Apr 10 23:25:59 2014restats
> >
> >
> > 18101188 function calls in 1218.626 seconds
> >
> >
> > Ordered by: internal time
> > List reduced from 13 to 10 due to restriction <10>
> >
> >
> > ncalls tottime percall cumtime percall filename:lineno(function)
> >1 1015.803 1015.803 1218.334 1218.334 skymaps5.py:44(mymain)
> > 18101000 202.4900.000 202.4900.000 {method 'write' of 'file'
> objects}
> >
> >
> >10.2920.292 1218.626 1218.626 :1()
> >60.0290.0050.0290.005 {open}
> >50.0100.0020.0100.002 {method 'close' of 'file'
> objects}
> >
> >
> > 810.0020.0000.0020.000 {method 'split' of 'str'
> objects}
> > 820.0010.0000.0010.000 {zip}
> >10.0000.0000.0000.000 function_base.py:8(linspace)
> >10.0000.0000.0000.000
> function_base.py:93(logspace)
> >50.0000.0000.0000.000
> {numpy.core.multiarray.zeros}
> >
> >
> >Anyway I would like to try to speed it up using C functions (and maybe
> comparing the resuts of the two profile in the end)
> >How can I do it now? Can I use Cython?
>
> If you have a compiler installed already it's just easy_install cython.
> Writing Cython is not hard. That is, you easily get speed improvements.
> It's in a .pyx file and once you're done you generate the .c and .so/.dll
> files with a setup.py like below. Reason why I am posting this snippet is
> the way to generate an annotated html file of your cython code. The whiter,
> the more in C, the better. Yellow means stuff might still be improved more.
>
> * setup.py
> from distutils.core import setup
> from distutils.extension import Extension
> from Cython.Distutils import build_ext
> import Cython.Compiler.Options
> Cython.Compiler.Options.annotate = True # < really handy
>
> setup(
> cmdclass = {'build_ext': build_ext},
> ext_modules = [Extension("myModule", ["myModule.pyx"])]
>
> )
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] improving speed using and recalling C functions
Hi Danny,
I followed your suggestion.
Tomorrow morning I will run this new version of the code.
Now using a sample of 81 elements (instead of 60) the profile returns:
Thu Apr 10 23:25:59 2014restats
18101188 function calls in 1218.626 seconds
Ordered by: internal time
List reduced from 13 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 1015.803 1015.803 1218.334 1218.334 skymaps5.py:44(mymain)
18101000 202.4900.000 202.4900.000 {method 'write' of 'file'
objects}
10.2920.292 1218.626 1218.626 :1()
60.0290.0050.0290.005 {open}
50.0100.0020.0100.002 {method 'close' of 'file'
objects}
810.0020.0000.0020.000 {method 'split' of 'str'
objects}
820.0010.0000.0010.000 {zip}
10.0000.0000.0000.000 function_base.py:8(linspace)
10.0000.0000.0000.000 function_base.py:93(logspace)
50.0000.0000.0000.000 {numpy.core.multiarray.zeros}
Anyway I would like to try to speed it up using C functions (and maybe
comparing the resuts of the two profile in the end)
How can I do it now? Can I use Cython?
Thanks
Gabriele
2014-04-10 21:38 GMT-04:00 Danny Yoo :
> > Comment: You are looping over your sliced eel five times. Do you
> >need to? I like eel salad a great deal, as well, but, how about:
> >
> >
> >for k in eel:
> >MYMAP1[i, j, k] = MYMAP1[i, j, k] + myinternet[oo]
> >MYMAP2[i, j, k] = MYMAP2[i, j, k] + myinternet[oo]
> >MYMAP3[i, j, k] = MYMAP3[i, j, k] + myinternet[oo]
> >MYMAP4[i, j, k] = MYMAP4[i, j, k] + myinternet[oo]
> >MYMAP5[i, j, k] = MYMAP5[i, j, k] + myinternet[oo]
> >oo = oo + 1
>
>
> Hi Gabriele,
>
> Also note that, when Martin looked at this part of the code, he
> unfortunately misinterpreted its effect; Martin's proposed rewrite
> here does not preserve the meaning of the original code. But rather
> than wag my finger at how Martin interpreted the code, I'd rather make
> the observation that this is a warning sign that the original code
> here was not easy to understand.
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] improving speed using and recalling C functions
sure.
def mymain():
def LEstep(n):
Emin=10**6
Emax=5*(10**10)
Lemin=log10(Emin)
Lemax=log10(Emax)
stepE=(Lemax-Lemin)/n
return (stepE, n, Lemin, Lemax)
if __name__ == "__main__":
import sys
if len(sys.argv)<=1:
stepENE, nex, Lemin, Lemax = LEstep(200)
elif len(sys.argv)<=2:
stepENE, nex, Lemin, Lemax =
LEstep(int(sys.argv[1]))
else:
stepENE, nex, Lemin, Lemax =
LEstep(int(sys.argv[1]))
freq=float(sys.argv[2])
eel = list(range(nex))
eels = np.logspace(Lemin, Lemax, num=nex, endpoint=False)
indpha = list(range(npha))
indobs = list(range(nobs))
rlc = c/(2*pi*freq)
MYMAP1 = np.zeros([npha, nobs, nex], dtype=float)
MYMAP2 = np.zeros([npha, nobs, nex], dtype=float)
MYMAP3 = np.zeros([npha, nobs, nex], dtype=float)
MYMAP4 = np.zeros([npha, nobs, nex], dtype=float)
MYMAP5 = np.zeros([npha, nobs, nex], dtype=float)
count=0
omegacliston = []
alpha = '60_'
for my_line in open('datasm0_60_5s.dat'):
myinternet = []
gmlis = []
print('reading the line', count, '/599378')
my_parts = [float(i) for i in my_line.split()]
phase = my_parts[4]
zobs = my_parts[5]
rho = my_parts[6]
gammar1 = my_parts[7]
gammar2 = my_parts[8]
gammar3 = my_parts[9]
gammar4 = my_parts[10]
gammar5 = my_parts[11]
gmlis.append(gammar1)
gmlis.append(gammar2)
gmlis.append(gammar3)
gmlis.append(gammar4)
gmlis.append(gammar5)
i = int((phase-phamin)/stepPHA)
j = int((zobs-obamin)/stepOB)
for gammar in gmlis:
omC = (1.5)*(gammar**3)*c/(rho*rlc)
gig = omC*hcut/eVtoErg
omegacliston.append(omC)
for w in eel[:]:
omega =
(10**(w*stepENE+Lemin))*eVtoErg/hcut
x = omega/omC
kap = instruments.kappa(x)
Iom = (1.732050808/c)*(e**2)*gammar*kap
P = Iom*(c/(rho*rlc))/(2*pi)
phps = P/(hcut*omega )
www = phps/(stepPHA*sin(zobs)*stepOB)
myinternet.append(www)
count = count + 1
oo = 0
for k in eel[:]:
MYMAP1[i, j, k] = MYMAP1[i, j, k] + myinternet[oo]
oo = oo + 1
for k in eel[:]:
MYMAP2[i, j, k] = MYMAP2[i, j, k] + myinternet[oo]
oo = oo + 1
for k in eel[:]:
MYMAP3[i, j, k] = MYMAP3[i, j, k] + myinternet[oo]
oo = oo + 1
for k in eel[:]:
MYMAP4[i, j, k] = MYMAP4[i, j, k] + myinternet[oo]
oo = oo + 1
for k in eel[:]:
MYMAP5[i, j, k] = MYMAP5[i, j, k] + myinternet[oo]
oo = oo + 1
BIGMAPS = [MYMAP1, MYMAP2, MYMAP3, MYMAP4, MYMAP5]
sigmas = [1, 3, 5, 10, 30]
fiq1 = plt.figure()
fiq2 = plt.figure()
fiq3 = plt.figure()
fiq4 = plt.figure()
fiq5 = plt.figure()
fiqqs = [fiq1, fiq2, fiq3, fiq4, fiq5]
multis = zip(sigmas, BIGMAPS, fiqqs)
for sigma, MYMAP, fiq in multis:
filename=alpha+'_'+str(sigma)+'_'+str(npha)+'_'+str(phamin)+'_'+str(phamax)+'_'+str(nobs)+'_'+str(obamin)+'_'+str(obamax)+'_'+str(nex)+'_'+str(Lemin)+'_'+str(Lemax)+'_.dat'
MYfile = open(filename, 'a')
for k in eel[:]:
for j in indobs[:]:
for i in indpha[:]:
A=MYMAP[i, j, k]
stringa = str(A) + ','
MYfile.write(stringa)
accapo = '\n'
MYfile.write(accapo)
MYfile.close()
2014-04-10 19:55 GMT-04:00 Danny Yoo :
>
> >ncalls tottime percall cumtime percall filename:lineno(function)
> > 1 149.479 149.479 199.851 199.851 skymaps5.py:16(mymain)
> > 18101000 28.6820.000 28.6820.000 {method 'write' of 'file'
> objects}
> >
> > 33044
Re: [Tutor] improving speed using and recalling C functions
but main is the program that contains everything.
I used the profile in this way:
import cProfile
import pstats
def mymain():
#all the code
#end of main indentation
cProfile.run('mymain()', 'restats', 'time')
p = pstats.Stats('restats')
p.strip_dirs().sort_stats('name')
p.sort_stats('time').print_stats(10)
So all the function I used are contained in main(), so even all the others
that are appeared.
Gabriele
2014-04-10 19:41 GMT-04:00 Martin A. Brown :
>
> Gabriele,
>
>
> 21071736 function calls in 199.883 seconds
>>
>
> The 21 million function calls isn't really a surprise to me, given 18
> million calls to file.write(). Given that the majority of the time is
> still spent in skymaps5.py, I think you'll need to instrument that a bit
> more to figure out where the hotspot is.
>
>Ordered by: internal time
>> List reduced from 188 to 10 due to restriction <10>
>>
>> ncalls tottime percall cumtime percall filename:lineno(function)
>>1 149.479 149.479 199.851 199.851 skymaps5.py:16(mymain)
>> 18101000 28.6820.000 28.6820.000 {method 'write' of 'file'
>> objects}
>>
>> the major time is required by mymain that is the whole program. the write
>> on file is an operation that I do in the end but if I increase the number
>> of data it doesn't increase (I tried doubing the sample of values and I
>> know why it's behaving in this way) the third are interpolate and kappa:
>> the two functions (one called inside the other one)
>>
>
> This is a good finding, in fact. Now, you know which module contains the
> bottleneck. Is your CPU pegged when you run that skymap5.py code? Can you
> run the profiler on the skymaps5.py code, too, to locate the specific
> hotspot? Great to see that you are using the profiler, effectively, too!
>
>
> So they are the ones that are taking time.
>> Now what can I do?
>>
>
> I think you now need to profile skymaps5.py code? Specifically the stuff
> in main().
>
> OK, so I have not myself used scipy.interpolate.interp1d before, but I
> went to have a look at it. So, you feed interp1d() an x and a y (e.g. a
> plot line on a diagram), and it essentially produces its best guess of a
> function which will fit that curve, correct?
>
> Well, this is how it performs with random input and an identity function:
>
> element count 100, duration 0.000
> element count 1000, duration 0.001
> element count 1, duration 0.005
> element count 10, duration 0.055
> element count 100, duration 0.858
> element count 1000, duration 30.404
>
> So, with 10 million inputs on an admittedly brain-dead function, there's
> not a performance bottleneck. If you can find the parts of your
> skymaps5.py code that are the bottleneck, then you could post it here.
>
> I hadn't thought of using interpolate, myself, as I didn't even know it
> existed.
>
> Thanks and good luck,
>
> -Martin
>
>
> --
> Martin A. Brown
> http://linux-ip.net/
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] improving speed using and recalling C functions
Hi,
I get this result:
Thu Apr 10 17:35:53 2014restats
21071736 function calls in 199.883 seconds
Ordered by: internal time
List reduced from 188 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 149.479 149.479 199.851 199.851 skymaps5.py:16(mymain)
18101000 28.6820.000 28.6820.000 {method 'write' of 'file'
objects}
330445.4700.0006.4440.000
interpolate.py:394(_call_linear)
232.2720.000 21.2790.000 instruments.py:10(kappa)
2313282.1200.0002.1200.000 {numpy.core.multiarray.array}
330441.7190.0003.8360.000
interpolate.py:454(_check_bounds)
660881.6110.0001.6110.000 {method 'reduce' of
'numpy.ufunc'
objects}
330441.1460.000 11.6230.000 interpolate.py:443(_evaluate)
330441.1200.0005.5420.000 interpolate.py:330(__init__)
330440.6590.0002.3290.000 polyint.py:82(_set_yi)
the major time is required by mymain that is the whole program.
the write on file is an operation that I do in the end but if I increase
the number of data it doesn't increase (I tried doubing the sample of
values and I know why it's behaving in this way)
the third are interpolate and kappa: the two functions (one called inside
the other one)
So they are the ones that are taking time.
Now what can I do?
thanks
Gabriele
2014-04-10 17:14 GMT-04:00 Danny Yoo :
> Hi Gabriele,
>
>
> I should probably have pointed you to:
>
> https://docs.python.org/2/library/profile.html#instant-user-s-manual
>
> instead.
>
>
> Here is an example that uses the cProfile module. Let's say that I'm
> trying to pinpoint where something is going slow in some_program():
>
> #
> import cProfile
>
> def slow_string_mul(w, n):
> s = ""
> for x in xrange(n):
> s = slow_append(s, w)
> return s
>
> def slow_append(s, w):
> return s + w
>
> def fast_string_mul(w, n):
> return w * n
>
> def some_program(w, n):
> print slow_string_mul(w, n) == fast_string_mul(w, n)
>
> ## Try running the operation, and let cProfile report stats."
> cProfile.run('some_program("testing", 5)', None, 'time')
> #
>
>
> We tell cProfile.run to execute some_program(), sorting its
> measurements by time. You might save the collected statistics to a
> file, but here I have not, and have cProfile.run() just report the
> results after the program finishes.
>
>
>
> Here's what comes back from cProfile's report:
>
> #
> $ python test_profile.py
> True
> 50005 function calls in 1.422 seconds
>
>Ordered by: internal time
>
>ncalls tottime percall cumtime percall filename:lineno(function)
> 51.2250.0001.2250.000
> test_profile.py:9(slow_append)
> 10.1970.1971.4221.422
> test_profile.py:3(slow_string_mul)
> 10.0000.0001.4221.422
> test_profile.py:15(some_program)
> 10.0000.0000.0000.000
> test_profile.py:12(fast_string_mul)
> 10.0000.0001.4221.422 :1()
> 10.0000.0000.0000.000 {method 'disable' of
> '_lsprof.Profiler' objects}
> #
>
> This is telling me that slow_append is being called 5 times, which
> in hindsight sounds right. It's taking the majority of the time of
> this program, which is intuitively true, as I wrote it in a way to do
> a very naive string-appending operation, which gets more expensive the
> longer the string grows.
>
>
>
> The point of examples is to show simple uses of the libraries. But
> you will have to adapt the examples to work for your context.
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] improving speed using and recalling C functions
I'm trying to profile it adding this code:
import cProfile
import re
import pstats
cProfile.run('re.compile("foo|bar")', 'restats')
p = pstats.Stats('restats')
p.strip_dirs().sort_stats('name')
p.sort_stats('time').print_stats(10)
but where I have to add this in my code?
because I obtain
Thu Apr 10 15:23:17 2014restats
194 function calls (189 primitive calls) in 0.001 seconds
Ordered by: internal time
List reduced from 34 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
3/10.0000.0000.0000.000 sre_compile.py:33(_compile)
10.0000.0000.0000.000
sre_compile.py:208(_optimize_chars
et)
3/10.0000.0000.0000.000 sre_parse.py:141(getwidth)
10.0000.0000.0000.000
sre_compile.py:362(_compile_info)
20.0000.0000.0000.000 sre_parse.py:380(_parse)
10.0000.0000.0000.000 sre_parse.py:302(_parse_sub)
10.0000.0000.0010.001 re.py:226(_compile)
100.0000.0000.0000.000 sre_parse.py:183(__next)
10.0000.0000.0010.001 sre_compile.py:496(compile)
150.0000.0000.0000.000 {isinstance}
but my program take more than 0.001 seconds!
I think it's not working as I want.
Gabriele
2014-04-10 15:09 GMT-04:00 Danny Yoo :
> Hi Gabriele,
>
> Have you profiled your program? Please look at:
>
> https://docs.python.org/2/library/profile.html
>
> If you can, avoid guessing what is causing performance to drop.
> Rather, use the tools in the profiling libraries to perform
> measurements.
>
>
> It may be that your program is taking a long time because of something
> obvious, but perhaps there is some other factor that's contributing.
> Please do this. More details would be helpful. Are you using any
> libraries such as Numpy? Just writing something in C doesn't
> magically make it go faster. CPython is written in C, for example,
> and yet people do not say that Python itself is very fast. :P
>
> It may be the case that writing the computations in C will allow you
> to specify enough type information so that the computer can
> effectively run your computations quickly. But if you're using
> libraries like Numpy to do vector parallel operations, I would not be
> surprised if that would outperform native non-parallel C code.
>
> See:
>
>
> http://technicaldiscovery.blogspot.com/2011/06/speeding-up-python-numpy-cython-and.html
>
> which demonstrates that effective use of NumPy may speed up
> computations by a significant order of magnitude, if you use the
> library to take advantage of its vectorizing compuations.
>
>
> More domain-specific details may help folks on the tutor list to give
> good advice here.
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] improving speed using and recalling C functions
Hi, 2014-04-10 13:05 GMT-04:00 Martin A. Brown : > > Hi there Gabriele, > > : I have a program that is reading near 60 elements from a > : file. For each element it performs 200 times a particular > : mathematical operation (a numerical interpolation of a function). > : Now these process takes near 8 hours. > > Sounds fun! Here are some thoughts (though I may not have any solid > answers, I hope these pointers are useful): > > Are you sure (from profiling) that the numerical interpolation > is the bottleneck? > > I see that the time needed by the code increase when I increase the number of these operation. It's the only thing I do in the code. (I'm sorry but I don't know very well what profiling is) > What is the numerical operation? > 1: Is the function implemented in Numpy? [0] > 2: How about SciPy? [1] > > Is interp1d from scipy.interpolate. I think is the best one > Is there a domain-specific Python library that already does the > computation you want to achieve? If so, then you have "only" the > format question--how to put your data into the format required by > the library. If not, then on to the next question you ask ... > > No, interp1d do exactly what I want. > : Creating a C function and calling it from the code could improve > : the speed? It could be useful that it contains both the > : mathematical operation and the for loop or only the mathematical > : operation? > : > : If it is a reasonable solution to try decreasing the > : computational time how can I implement this C function? > > It is certainly an option! In the event that you cannot find > something that is fast enough in Python already and you cannot find > any C library that has Python bindings, then, whether you have to > write your own, or you have a third party C library, you have > several options for writing bindings. > You can write your Python bindings using: > > * ctypes: https://docs.python.org/2/library/ctypes.html > * cffi: http://cffi.readthedocs.org/en/release-0.8/ > * cython: http://www.cython.org/ > * roll your own C! > > Do you know if does a C library exist with an already implemented function like interp1d? I use Anaconda, is there an already implemented mode to do it in Anaconda? Cython? > It might be beyond the scope of this list (certainly beyond my > capabilities) to provide recommendations on which avenue to > take--but if you are asking, you may already have the tools you need > to assess for yourself. There is, however, quite a bit of > experience loitering around this list, so perhaps somebody else will > have a suggestion. > > -Martin > > [0] http://www.numpy.org/ > [1] http://www.scipy.org/ > > -- > Martin A. Brown > http://linux-ip.net/ > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] improving speed using and recalling C functions
Hi, I have a program that is reading near 60 elements from a file. For each element it performs 200 times a particular mathematical operation (a numerical interpolation of a function). Now these process takes near 8 hours. Creating a C function and calling it from the code could improve the speed? It could be useful that it contains both the mathematical operation and the for loop or only the mathematical operation? If it is a reasonable solution to try decreasing the computational time how can I implement this C function? Thanks Gabriele ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] c++ on python
I answer in the text 2014-03-11 9:32 GMT-04:00 ALAN GAULD : > CC'ing the list > Please use ReplyAll when responding. > > > -- > *From:* Gabriele Brambilla > *To:* Alan Gauld > *Sent:* Tuesday, 11 March 2014, 12:54 > *Subject:* Re: [Tutor] c++ on python > > I think (because I've not received the code yet) I will receive the source > code (.c or .cpp file) > and I want to compile it in the way to use it and maybe make small > changes. So I think I want > to embed the code as a Python module (but it's not properly a library). > > What is your experience level with C/C++? > Are you familiar with building C/C++ libraries or even object files? > I know C/C++, I am not able to do everything but I think that I can compile a simple programm with object files and libraries > There are documents and tools to help you turn C code into Python > libraries > but that's really outside the scope of the tutor list. > > About the dependencies I am not so sure as before. > So I mistaken the list? which one is the right one? > > I suspect you may want a different list. But you will need to be clear > about what you are > trying to do. It's still not clear what exactly this source code will be. > Is it a library or a program? > A program. it should use only standard C++ libraries. > Do you think is it better that I install a C compiler and I don't use > python? I use Anaconda... > > You will need a C compiler regardless, if you receive C source code. > Python cannot work with C in source format, only after it is compiled. > But that does not mean you can't use Python to work with it, and that is > probably easier than trying to write your whole application in > C++ - especially if you are not already fluent in C++. > Yes I would prefer to use Python as much as possible. What do you mean about using Python to work with it? How do you usually do? I've no experience of Anaconda but it looks like it might be hard to find > an equivalent > in the C++ world, especially if you have already written a lot of > Python/Anaconda code. > > Alan Gauld > Author of the Learn To Program website > http://www.alan-g.me.uk/ > http://www.flickr.com/photos/alangauldphotos > Thanks Gabriele ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] c++ on python
Hi, in the next days I will receive a c++ code that I would like to run in python (http://docs.python.org/2/extending/index.html). It should be self consistent (no extraroutines). I want to be ready to use it... Has someone some C++ code examples available that I can try to run easily before getting that code? thanks Gabriele ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] numerical problem
for example I read this: On Pythons prior to 2.7 and 3.1, once you start experimenting with floating-point numbers, you're likely to stumble across something that may look a bit odd at first glance: >>> 3.1415 * 2 # repr: as code (Pythons < 2.7 and 3.1) 6.2834 >>> print(3.1415 * 2) # str: user-friendly 6.283 The first result isn't a bug; it's a display issue. It turns out that there are two ways to print every object in Python--with full precision (as in the first result shown here), and in a user-friendly form (as in the second). Formally, the first form is known as an object's as-code repr, and the second is its user-friendly str. In older Pythons, the floating-point repr sometimes displays more precision than you might expect. The difference can also matter when we step up to using classes. For now, if something looks odd, try showing it with a print built-in function call statement. Better yet, upgrade to Python 2.7 and the latest 3.X, where floating-point numbers display themselves more intelligently, usually with fewer extraneous digits--since this book is based on Pythons 2.7 and 3.3, this is the display form I'll be showing throughout this book for floating-point numbers: >>> 3.1415 * 2 # repr: as code (Pythons >= 2.7 and 3.1) 6.283 Besides expressions, there are a handful of useful numeric modules that ship with Python--modules are just packages of additional tools that we import to use: >>> import math >>> math.pi 3.141592653589793 >>> math.sqrt(85) 9.219544457292887 The math module contains more advanced numeric tools as functions, while the ran dom module performs random-number generation and random selections (here, from a Python list coded in square brackets--an ordered collection of other objects to be introduced later in this chapter): >>> import random >>> random.random() 0.7082048489415967 >>> random.choice([1, 2, 3, 4]) 1 Could the problem be something like this? thanks Gabriele 2014-03-03 22:44 GMT-05:00 Gabriele Brambilla < gb.gabrielebrambi...@gmail.com>: > ok, > the things I get is very specific. > my results differs from the other guy's ones with which I compare them for > slightly details. > We fit the data with a function and we obtain different but similar > values. our fit routines on the same data return the same results. So we > produce different data. > the values in www are like 1.01e-134 and also smaller, and I sum values > like this: there could be any problem summing so small numbers? is there a > Python guide that helps in using strange values numbers? > > thanks > > Gabriele > > > > > 2014-03-03 21:02 GMT-05:00 Steven D'Aprano : > > On Mon, Mar 03, 2014 at 08:00:52PM -0500, Gabriele Brambilla wrote: >> > Hi, >> > >> > I'm doing a sum in a for loop: >> > >> > www is the quantity I add. >> > >> > MYMAP[i, j, k] = MYMAP[i, j, k] + www >> > >> > MYMAP is a numpy array >> > >> > I have strong reasons to think that in this operation happens some >> > numerical error...Have you suggestions to discover where it is? >> >> You are almost certainly correct. Unless all the numbers are exact >> integers, nearly every floating point operation will add some error. >> >> Without seeing your code, and some representative sample data, it is >> impossible to tell whether the error is tiny or huge. >> >> Tiny error: >> >> MYMAP[1, j, k] = 1.001 >> www = 0.25 >> >> >> Huge error: >> >> MYMAP[1, j, k] = 1.001e19 >> www = 0.25 >> >> >> >> -- >> Steven >> ___ >> Tutor maillist - Tutor@python.org >> To unsubscribe or change subscription options: >> https://mail.python.org/mailman/listinfo/tutor >> > > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] numerical problem
ok, the things I get is very specific. my results differs from the other guy's ones with which I compare them for slightly details. We fit the data with a function and we obtain different but similar values. our fit routines on the same data return the same results. So we produce different data. the values in www are like 1.01e-134 and also smaller, and I sum values like this: there could be any problem summing so small numbers? is there a Python guide that helps in using strange values numbers? thanks Gabriele 2014-03-03 21:02 GMT-05:00 Steven D'Aprano : > On Mon, Mar 03, 2014 at 08:00:52PM -0500, Gabriele Brambilla wrote: > > Hi, > > > > I'm doing a sum in a for loop: > > > > www is the quantity I add. > > > > MYMAP[i, j, k] = MYMAP[i, j, k] + www > > > > MYMAP is a numpy array > > > > I have strong reasons to think that in this operation happens some > > numerical error...Have you suggestions to discover where it is? > > You are almost certainly correct. Unless all the numbers are exact > integers, nearly every floating point operation will add some error. > > Without seeing your code, and some representative sample data, it is > impossible to tell whether the error is tiny or huge. > > Tiny error: > > MYMAP[1, j, k] = 1.001 > www = 0.25 > > > Huge error: > > MYMAP[1, j, k] = 1.001e19 > www = 0.25 > > > > -- > Steven > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] numerical problem
Hi, I'm doing a sum in a for loop: www is the quantity I add. MYMAP[i, j, k] = MYMAP[i, j, k] + www MYMAP is a numpy array I have strong reasons to think that in this operation happens some numerical error...Have you suggestions to discover where it is? thanks Gabriele ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] from command prompt use interactive python and running script together
Hi, Is possible on python to running scripts from the command prompt (I'm using python on windows) and in the end saving all the variables and continue the analysis in the interactive mode? (the one that you activate typing python in the command prompt?) Or to use python in the interactive mode and in some moments to run scripts, without quit() the interactive mode, and use for the script variables the one you have defined in the interactive mode? thanks Gabriele ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] for: how to skip items
thanks, in the end I'm using something like this and it works: zipPARApha = zip(Pampli, Pgamma, Pecut, Pb, g) for n, (a1,b1,c1,d1,pha) in enumerate(zipPARApha): where the arguments of zip are lists of the same size. Gabriele 2014-02-17 11:19 GMT-05:00 Oscar Benjamin : > On 17 February 2014 16:13, Gabriele Brambilla > wrote: > > No sorry, > > > > it's because my problem is not so simple: > > imagine that in a100 contains not integer sorted in a good way but a > random > > float numbers. > > How could I display only one item every 10? > > for n, a in enumerate(a100): > if n % 10 == 9: > print(a) > > > Oscar > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] for: how to skip items
No sorry, it's because my problem is not so simple: imagine that in a100 contains not integer sorted in a good way but a random float numbers. How could I display only one item every 10? thanks Gabriele 2014-02-17 11:08 GMT-05:00 Oscar Benjamin : > On 17 February 2014 16:05, Gabriele Brambilla > wrote: > > Hi, > > > > I'm wondering how I can (if I can) make a for loop in which I don't use > all > > the elements. > > > > for example > > > > a100 = list(range(100)) > > > > for a in a100: > > print(a) > > > > it print out to me all the numbers from 0 to 99 > > But if I want to display only the numbers 0, 9, 19, 29, 39, ...(one > every 10 > > elements) how can I do it WITHOUT defining a new list (my real case is > not > > so simple) and WITHOUT building a list of indexes? > > for a in a100: > if a % 10 == 9: > print(a) > > Alternatively: > > for a in a100: > if a % 10 == 9: > continue > print(a) > > > Oscar > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] for: how to skip items
Doesn't exist a way in Python to do like in C for i=0, i<100, i=i+10 ? without creating a list of index? Gabriele 2014-02-17 11:15 GMT-05:00 David Palao : > Hi Gabriele, > Without knowing the details of what you are trying, I guess you could > be interested in looking at how to define your own iterators. > > Regards > > 2014-02-17 17:05 GMT+01:00 Gabriele Brambilla < > gb.gabrielebrambi...@gmail.com>: > > Hi, > > > > I'm wondering how I can (if I can) make a for loop in which I don't use > all > > the elements. > > > > for example > > > > a100 = list(range(100)) > > > > for a in a100: > > print(a) > > > > it print out to me all the numbers from 0 to 99 > > But if I want to display only the numbers 0, 9, 19, 29, 39, ...(one > every 10 > > elements) how can I do it WITHOUT defining a new list (my real case is > not > > so simple) and WITHOUT building a list of indexes? > > > > thank you > > > > Gabriele > > > > ___ > > Tutor maillist - Tutor@python.org > > To unsubscribe or change subscription options: > > https://mail.python.org/mailman/listinfo/tutor > > > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] for: how to skip items
Excuse me for the bad english: not "a random float numbers" but "random float numbers" Gabriele 2014-02-17 11:13 GMT-05:00 Gabriele Brambilla < gb.gabrielebrambi...@gmail.com>: > No sorry, > > it's because my problem is not so simple: > imagine that in a100 contains not integer sorted in a good way but a > random float numbers. > How could I display only one item every 10? > > thanks > > Gabriele > > > 2014-02-17 11:08 GMT-05:00 Oscar Benjamin : > > On 17 February 2014 16:05, Gabriele Brambilla >> wrote: >> > Hi, >> > >> > I'm wondering how I can (if I can) make a for loop in which I don't use >> all >> > the elements. >> > >> > for example >> > >> > a100 = list(range(100)) >> > >> > for a in a100: >> > print(a) >> > >> > it print out to me all the numbers from 0 to 99 >> > But if I want to display only the numbers 0, 9, 19, 29, 39, ...(one >> every 10 >> > elements) how can I do it WITHOUT defining a new list (my real case is >> not >> > so simple) and WITHOUT building a list of indexes? >> >> for a in a100: >> if a % 10 == 9: >> print(a) >> >> Alternatively: >> >> for a in a100: >> if a % 10 == 9: >> continue >> print(a) >> >> >> Oscar >> > > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] for: how to skip items
Hi, I'm wondering how I can (if I can) make a for loop in which I don't use all the elements. for example a100 = list(range(100)) for a in a100: print(a) it print out to me all the numbers from 0 to 99 But if I want to display only the numbers 0, 9, 19, 29, 39, ...(one every 10 elements) how can I do it WITHOUT defining a new list (my real case is not so simple) and WITHOUT building a list of indexes? thank you Gabriele ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] strange errors
ok. the problem is that my "errors" are not always the same and sometimes they are only differences in the results displayed (with the same starting data and no random variables used between). so it's difficult for me to understand where the error is (and to give you a piece of code that can run). for example today I got this two warnings and the results displayed were wrong C:\Users\Gabriele\Anaconda1\lib\site-packages\matplotlib\colors.py:918: RuntimeWarning: overflow encountered in true_divide resdat /= (vmax - vmin) C:\Users\Gabriele\Anaconda1\lib\site-packages\matplotlib\colors.py:568: RuntimeWarning: overflow encountered in multiply xa *= self.N and when I search the maximum and the minimum in a matrix it has returned me two nan values. Gabriele 2014-02-03 Jerry Hill : > On Mon, Feb 3, 2014 at 2:17 PM, Gabriele Brambilla > wrote: > > No, i'm not using lowlevel stuff...which part of the script do you want > to > > see? > > Since you asked, what we'd really like to see is a Short, Self > Contained, Compilable Example (see http://sscce.org/). That is, an > example that is short enough to be read quickly, self contained enough > so that we can actually save the code and run it, compilable in that > the code will run, at least to the point where you're having a problem > (it's fine to have code that won't run, as long as the error that pops > up is the error you're actually looking for help with). > > We know that it can be a lot of work to create an example like that, > but if you do there are two things that are likely to happen. First, > probably 75% of the time, you will find your problem yourself without > ever having to post it to the list. The very act of trimming your > code into the piece that demonstrates your problem, will help you find > the problem yourself. > > Second, you are very, very likely to get people to pay attention to > your question and provide advice on how to fix your code when you make > it easy for them to help. > > -- > Jerry > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] strange errors
No, i'm not using lowlevel stuff...which part of the script do you want to see? thanks Gabriele 2014-02-03 Peter Otten <__pete...@web.de>: > Gabriele Brambilla wrote: > > > an example of errors that I obtain is: I build a matrix (SciPy array) > > using the same data. > > I search for the maximum and the minimum with the given function and I > > insert the two values in a tuple that I print. > > sometimes it print to me this: (0.0, 3.1926676650124463e-14) > > sometimes it print to me: (nan, nan) > > Are you messing around with byteorder or other lowlevel stuff? > Can you provide a small script that shows this behaviour? > > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] strange errors
an example of errors that I obtain is: I build a matrix (SciPy array) using the same data. I search for the maximum and the minimum with the given function and I insert the two values in a tuple that I print. sometimes it print to me this: (0.0, 3.1926676650124463e-14) sometimes it print to me: (nan, nan) Thanks Gabriele 2014-02-02 Alan Gauld : > On 02/02/14 02:11, Gabriele Brambilla wrote: > > sometimes when I try to run a program in Python I obtain some errors. >> > > How are you running the program? > > doubly clicking in a file manager/explorer? > Running from an OS command line? > Importing from the Python >>> prompt? > Or using some kind of IDE? (Which?) > > The answer to your question depends a lot on the way you > run the code. > > A sample error and your OS and Python version will help too. > > > How could I avoid this problem? I think that it is because it "remember" >> some variables that I have used... >> > > Could be. If you import a module then change that module then > reimporting alone is not enough. But if you run the code from the OS that > shouldn't be an issue. So how do you run the code? > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.flickr.com/photos/alangauldphotos > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] strange errors
Hi, sometimes when I try to run a program in Python I obtain some errors. The strange thing is that sometimes when I run it a second time or when I turn off the pc and I restart later to try to make it works it gives different errors. How could I avoid this problem? I think that it is because it "remember" some variables that I have used... Thanks Gabriele ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] interactive script
to simplify the question: does a command like cin in C++ or scanf in C exist?? thanks Gabriele 2014-01-31 Gabriele Brambilla : > Hi, > I'm very new to Python (just 5 days!) > is there a possibility to write an interactive script? > in the sense that: > > - you run your script and it do some things that you don't want to type > everytime you run the program > - but at a certain step I want that it ask me question like "which column > of this matrix do you want to use?" and I want to give to it the answer, > and after it that it do what I answered. > - I don't want to use argv method. > > thank you > > Gabriele > > p.s: I'm using Anaconda > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] interactive script
Hi, I'm very new to Python (just 5 days!) is there a possibility to write an interactive script? in the sense that: - you run your script and it do some things that you don't want to type everytime you run the program - but at a certain step I want that it ask me question like "which column of this matrix do you want to use?" and I want to give to it the answer, and after it that it do what I answered. - I don't want to use argv method. thank you Gabriele p.s: I'm using Anaconda ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading files
thanks to everyone, I've used David's method.
Gabriele
2014-01-29 Mark Lawrence
> On 29/01/2014 02:09, Gabriele Brambilla wrote:
>
>> Hi,
>> how could I read float numbers if the data format is like this (using
>> readline):
>>
>> 1.0551951.26758123387023-0.314470329249235
>> -0.293015360064208 6.157957619078221.92919102133526
>> 13.07804596303782.15175351758512e6
>>
>> the numbers aren't equally spaced and they had not the same number of
>> figures...
>>
>> thanks
>>
>> Gabriele
>>
>>
> Something like this, untested:-
>
> floats = []
> with open('myfile') as infile:
> for line in infile:
> floats.extend(float(f) for f in line.split())
>
> --
> My fellow Pythonistas, ask not what our language can do for you, ask what
> you can do for our language.
>
> Mark Lawrence
>
>
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] reading files
Hi, how could I read float numbers if the data format is like this (using readline): 1.0551951.26758123387023-0.314470329249235 -0.293015360064208 6.157957619078221.92919102133526 13.07804596303782.15175351758512e6 the numbers aren't equally spaced and they had not the same number of figures... thanks Gabriele ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
