but main is the program that contains everything.
I used the profile in this way:
import cProfile
import pstats
def mymain():
#all the code
#end of main indentation
cProfile.run('mymain()', 'restats', 'time')
p = pstats.Stats('restats')
p.strip_dirs().sort_stats('name')
p.sort_stats('time').print_stats(10)
So all the function I used are contained in main(), so even all the others
that are appeared.
Gabriele
2014-04-10 19:41 GMT-04:00 Martin A. Brown <mar...@linux-ip.net>:
>
> Gabriele,
>
>
> 21071736 function calls in 199.883 seconds
>>
>
> The 21 million function calls isn't really a surprise to me, given 18
> million calls to file.write(). Given that the majority of the time is
> still spent in skymaps5.py, I think you'll need to instrument that a bit
> more to figure out where the hotspot is.
>
> Ordered by: internal time
>> List reduced from 188 to 10 due to restriction <10>
>>
>> ncalls tottime percall cumtime percall filename:lineno(function)
>> 1 149.479 149.479 199.851 199.851 skymaps5.py:16(mymain)
>> 18101000 28.682 0.000 28.682 0.000 {method 'write' of 'file'
>> objects}
>>
>> the major time is required by mymain that is the whole program. the write
>> on file is an operation that I do in the end but if I increase the number
>> of data it doesn't increase (I tried doubing the sample of values and I
>> know why it's behaving in this way) the third are interpolate and kappa:
>> the two functions (one called inside the other one)
>>
>
> This is a good finding, in fact. Now, you know which module contains the
> bottleneck. Is your CPU pegged when you run that skymap5.py code? Can you
> run the profiler on the skymaps5.py code, too, to locate the specific
> hotspot? Great to see that you are using the profiler, effectively, too!
>
>
> So they are the ones that are taking time.
>> Now what can I do?
>>
>
> I think you now need to profile skymaps5.py code? Specifically the stuff
> in main().
>
> OK, so I have not myself used scipy.interpolate.interp1d before, but I
> went to have a look at it. So, you feed interp1d() an x and a y (e.g. a
> plot line on a diagram), and it essentially produces its best guess of a
> function which will fit that curve, correct?
>
> Well, this is how it performs with random input and an identity function:
>
> element count 100, duration 0.000
> element count 1000, duration 0.001
> element count 10000, duration 0.005
> element count 100000, duration 0.055
> element count 1000000, duration 0.858
> element count 10000000, duration 30.404
>
> So, with 10 million inputs on an admittedly brain-dead function, there's
> not a performance bottleneck. If you can find the parts of your
> skymaps5.py code that are the bottleneck, then you could post it here.
>
> I hadn't thought of using interpolate, myself, as I didn't even know it
> existed.
>
> Thanks and good luck,
>
> -Martin
>
>
> --
> Martin A. Brown
> http://linux-ip.net/
>
_______________________________________________
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
