Hi Sudheer,
Le 21/02/2013 02:22, Sudheer Joseph a écrit :
> Thank you very much Smith and Paul,
> I was away from office due
> to a medical situation. So could not respond and thank you regarding
> the help. I have got the results now and the tips from both of you
> were extremely useful. I am facing an issue with the code when I call
> plt.xcorr, in a loop. it builds up usage of memory by python and
> reaches to the RAM what ever available ( in my 4 GB laptop it reaches
> almost full and in my 24 GB desktop it reaches the available. I
> suspected the plot not being closed during each iteration so have
> given a plt.close('all') in the loop. after which it is taking a good
> time to run the code which was otherwise faster until ram usage
> reaches its maximum.
> Is there a way to get out of this situation?. I am attaching the code
> here and also the link to the data I am using. If possible kindly help.
>Thanks for sharing the code. By a quick look at gen_xcorr_wnd.py, you are generating a quite high number (about len(lons)*len(lats)) of xcorr series over 365 lags. Here are two thoughts about why I would not recommend using xcorr from matplotlib for this job : 1) There is an overhead in creating a plot object which is unnecessary since you're only interested in correlation values 2) internally, plt.xcorr uses numpy.correlate (https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/axes.py#L4319 and https://github.com/numpy/numpy/blob/master/numpy/core/numeric.py#L731) which is quite fast but unfortunately cannot be well tuned in terms of the output length (only three modes : 'valid', 'same' or 'full'. Matplotlib uses 'full' ) All this to say that when you're interested in 365 correlation values, the internal computations takes place on (N+M-1) points (where N, M are the length of the input vectors, i.e. 2189 if I'm right) and so about 90 % of the output is thrown away. This being said, there is a tiny issue : I don't know a good module which has the (x)correlation function. statsmodel has acf (aka correlation) but I don't remember if there is crosscorrelation. For acf has two computation modes : one based on fft, one based on numpy.correlate which suffer from the same problem as matplotlib's xcorr ( https://github.com/statsmodels/statsmodels/blob/master/statsmodels/tsa/stattools.py#L347) best, Pierre
signature.asc
Description: OpenPGP digital signature
------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
