Re: [Matplotlib-users] cross correlation
Thank you Pierre. with best regards, Sudheer From: Pierre Haessig To: Cc: "matplotlib-users@lists.sourceforge.net" Sent: Thursday, 28 February 2013 7:15 PM Subject: Re: [Matplotlib-users] cross correlation Hi, Le 27/02/2013 10:01, Sudheer Joseph a écrit : > I was checking the plt.xcorr and it calls the np.correlate in side it. > It calls np.correlate(ts1,ts2, mode=2). Just as a side note, mode=2 is the old fashioned way to specify mode='full' [1]. This may help in reading the numpy.correlate doc. This being said, I'm really unfamiliar with cross-correlations. I just kind of know the usual 95% confidence interval for autocorrelation at 1.96/sqrt(n). Just as a quick check, this is what R uses by default, but there are options like ci.type get more appropriate intervals for an MA series (http://stat.ethz.ch/R-manual/R-patched/library/stats/html/plot.acf.html) best, Pierre [1] https://github.com/numpy/numpy/blob/master/numpy/core/numeric.py#L678 -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users-- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] cross correlation
Hi, Le 27/02/2013 10:01, Sudheer Joseph a écrit : > I was checking the plt.xcorr and it calls the np.correlate in side it. > It calls np.correlate(ts1,ts2, mode=2). Just as a side note, mode=2 is the old fashioned way to specify mode='full' [1]. This may help in reading the numpy.correlate doc. This being said, I'm really unfamiliar with cross-correlations. I just kind of know the usual 95% confidence interval for autocorrelation at 1.96/sqrt(n). Just as a quick check, this is what R uses by default, but there are options like ci.type get more appropriate intervals for an MA series (http://stat.ethz.ch/R-manual/R-patched/library/stats/html/plot.acf.html) best, Pierre [1] https://github.com/numpy/numpy/blob/master/numpy/core/numeric.py#L678 signature.asc Description: OpenPGP digital signature -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] cross correlation
On Wed, Feb 27, 2013 at 1:01 AM, Sudheer Joseph wrote:
> Dear Pierre,
> I was checking the plt.xcorr and it calls the
> np.correlate in side it. It calls np.correlate(ts1,ts2, mode=2).
> Is there a way to see which vector is sided back in time? ie
> ts1[t1,t2,t3,t4] ts2[t1,t2,t3,t4...] ( ts2[t2] correlated with ts1[t1]
> or ts2[t1] is correlated with ts2[t2] ie {to make out which one is the
> cause and which is effect)
> in case of cross correlation it can be obtained by either sliding ts1 back
> in time or ts2 back in time. Is there a way to know this? I am not able to
> make much from np.correlate. Also is there a way to get 95% significance or
> p value from the xcorr.? in case of matlab? in matlab xcorr can be called
> with option of corcoeff instead of default cross correlation but is there
> similar option for matplotlib?
> sincerely.
> Sudheer
>
>
Sudheer:
It sounds like your needs are beyond the scope of matplotlib. It'll
probably be more productive to check in with the numpy or scipy mailing
lists.
-paul
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb___
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] cross correlation
Dear Pierre,
I was checking the plt.xcorr and it calls the
np.correlate in side it. It calls np.correlate(ts1,ts2, mode=2).
Is there a way to see which vector is sided back in time? ie
ts1[t1,t2,t3,t4] ts2[t1,t2,t3,t4...] ( ts2[t2] correlated with ts1[t1] or
ts2[t1] is correlated with ts2[t2] ie {to make out which one is the cause and
which is effect)
in case of cross correlation it can be obtained by either sliding ts1 back in
time or ts2 back in time. Is there a way to know this? I am not able to make
much from np.correlate. Also is there a way to get 95% significance or p value
from the xcorr.? in case of matlab? in matlab xcorr can be called with option
of corcoeff instead of default cross correlation but is there similar option
for matplotlib?
sincerely.
Sudheer
***
Sudheer Joseph
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.in...@gmail.com;sudheer.jos...@yahoo.com
Web- http://oppamthadathil.tripod.com
***
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb___
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] cross correlation
Le 21/02/2013 17:33, Sudheer Joseph a écrit : > Thank you Pierre, > I will test the other options. I did not > know the number limitation in case of plt.xcorr. > Thanks a lot > with best regards, Just for reference : http://stackoverflow.com/questions/6991471/computing-cross-correlation-function You'll see that (cross)correlation in Python a long ongoing topic. best, Pierre signature.asc Description: OpenPGP digital signature -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] cross correlation
Thank you Pierre,
I will test the other options. I did not know the
number limitation in case of plt.xcorr.
Thanks a lot
with best regards,
Sudheer
***
Sudheer Joseph
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.in...@gmail.com;sudheer.jos...@yahoo.com
Web- http://oppamthadathil.tripod.com
***
From: Pierre Haessig
To: matplotlib-users@lists.sourceforge.net
Sent: Thursday, 21 February 2013 9:52 PM
Subject: Re: [Matplotlib-users] cross correlation
Hi Sudheer,
Le 21/02/2013 02:22, Sudheer Joseph a écrit :
Thank you very much Smith and Paul,
> I was away from office due to a
>medical situation. So could not respond and thank you regarding the help. I
>have got the results now and the tips from both of you were extremely useful.
>I am facing an issue with the code when I call plt.xcorr, in a loop. it
>builds up usage of memory by python and reaches to the RAM what ever available
>( in my 4 GB laptop it reaches almost full and in my 24 GB desktop it reaches
>the available. I suspected the plot not being closed during each iteration so
>have given a plt.close('all') in the loop. after which it is taking a good
>time to run the code which was otherwise faster until ram usage reaches its
>maximum.
>Is there a way to get out of this situation?. I am attaching the code here and
>also the link to the data I am using. If possible kindly help.
>
Thanks for sharing the code. By a quick look at gen_xcorr_wnd.py,
you are generating a quite high number (about len(lons)*len(lats))
of xcorr series over 365 lags. Here are two thoughts about why I
would not recommend using xcorr from matplotlib for this job :
1) There is an overhead in creating a plot object which is
unnecessary since you're only interested in correlation values
2) internally, plt.xcorr uses numpy.correlate
(https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/axes.py#L4319
and https://github.com/numpy/numpy/blob/master/numpy/core/numeric.py#L731)
which is quite fast but unfortunately cannot be well tuned in terms of the
output length (only three modes : 'valid', 'same' or 'full'. Matplotlib uses
'full' )
All this to say that when you're interested in 365 correlation
values, the internal computations takes place on (N+M-1) points
(where N, M are the length of the input vectors, i.e. 2189 if I'm
right) and so about 90 % of the output is thrown away.
This being said, there is a tiny issue : I don't know a good module
which has the (x)correlation function. statsmodel has acf (aka
correlation) but I don't remember if there is crosscorrelation. For
acf has two computation modes : one based on fft, one based on
numpy.correlate which suffer from the same problem as matplotlib's
xcorr (
https://github.com/statsmodels/statsmodels/blob/master/statsmodels/tsa/stattools.py#L347)
best,
Pierre
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
___
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb___
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] cross correlation
Hi Sudheer,
Le 21/02/2013 02:22, Sudheer Joseph a écrit :
> Thank you very much Smith and Paul,
> I was away from office due
> to a medical situation. So could not respond and thank you regarding
> the help. I have got the results now and the tips from both of you
> were extremely useful. I am facing an issue with the code when I call
> plt.xcorr, in a loop. it builds up usage of memory by python and
> reaches to the RAM what ever available ( in my 4 GB laptop it reaches
> almost full and in my 24 GB desktop it reaches the available. I
> suspected the plot not being closed during each iteration so have
> given a plt.close('all') in the loop. after which it is taking a good
> time to run the code which was otherwise faster until ram usage
> reaches its maximum.
> Is there a way to get out of this situation?. I am attaching the code
> here and also the link to the data I am using. If possible kindly help.
>
Thanks for sharing the code. By a quick look at gen_xcorr_wnd.py, you
are generating a quite high number (about len(lons)*len(lats)) of xcorr
series over 365 lags. Here are two thoughts about why I would not
recommend using xcorr from matplotlib for this job :
1) There is an overhead in creating a plot object which is unnecessary
since you're only interested in correlation values
2) internally, plt.xcorr uses numpy.correlate
(https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/axes.py#L4319
and
https://github.com/numpy/numpy/blob/master/numpy/core/numeric.py#L731)
which is quite fast but unfortunately cannot be well tuned in terms of
the output length (only three modes : 'valid', 'same' or 'full'.
Matplotlib uses 'full' )
All this to say that when you're interested in 365 correlation values,
the internal computations takes place on (N+M-1) points (where N, M are
the length of the input vectors, i.e. 2189 if I'm right) and so about 90
% of the output is thrown away.
This being said, there is a tiny issue : I don't know a good module
which has the (x)correlation function. statsmodel has acf (aka
correlation) but I don't remember if there is crosscorrelation. For acf
has two computation modes : one based on fft, one based on
numpy.correlate which suffer from the same problem as matplotlib's xcorr
(
https://github.com/statsmodels/statsmodels/blob/master/statsmodels/tsa/stattools.py#L347)
best,
Pierre
signature.asc
Description: OpenPGP digital signature
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb___
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] cross correlation
Thank you very much Smith and Paul,
I was away from office due to a
medical situation. So could not respond and thank you regarding the help. I
have got the results now and the tips from both of you were extremely useful. I
am facing an issue with the code when I call plt.xcorr, in a loop. it builds
up usage of memory by python and reaches to the RAM what ever available ( in my
4 GB laptop it reaches almost full and in my 24 GB desktop it reaches the
available. I suspected the plot not being closed during each iteration so have
given a plt.close('all') in the loop. after which it is taking a good time to
run the code which was otherwise faster until ram usage reaches its maximum.
Is there a way to get out of this situation?. I am attaching the code here and
also the link to the data I am using. If possible kindly help.
ftp ftpser.incois.gov.in
user temp
password incoistemp
cd /home0/temp/comp
bin
mget qu_test.nc.gz gunzip qu_test.nc.gz
***
Sudheer Joseph
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.in...@gmail.com;sudheer.jos...@yahoo.com
Web- http://oppamthadathil.tripod.com
***
From: Sterling Smith
To: Sudheer Joseph
Cc: Paul Hobson ; "matplotlib-users@lists.sourceforge.net"
Sent: Friday, 8 February 2013 10:23 PM
Subject: Re: [Matplotlib-users] cross correlation
Sudheer,
For the documentation you are looking for
print ax1.xcorr.__doc__
(Paul tried to give you the IPython method of getting that documentation which
is by typing a ? (or ??) after the desired object.)
In the documentation (at the link you gave
http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.xcorr), it says
that there are three objects returned by xcorr:
Return value is a tuple (*lags*, *c*, *line*) where:
- *lags* are a length ``2*maxlags+1`` lag vector
- *c* is the ``2*maxlags+1`` auto correlation vector
- *line* is a :class:`~matplotlib.lines.Line2D` instance
returned by :func:`~matplotlib.pyplot.plot`.
So the error you were getting is due to the fact that you have only specified
two variables to hold the three returned objects.
Try:
lags,c,line = ax1.xcorr .
(Note that you have xcorr and lags backwards in your attempt.)
-Sterling
On Feb 8, 2013, at 1:56AM, Sudheer Joseph wrote:
> Thank you verymuch Hobson,
> However I think I did not understand
>the suggestion by you fully( pardon my ignorance). I use the below test code
>from matplotlib site. How does one make a call to get lags and correlation
>corresponding to the x and y values in the plot. a Print command of
> In [23]: print ax1.xcorr
> 0x44c1410>>
> results as above. Is it possible to assign the xcorr,lags=ax1.xcorr(x, y,
> usevlines=True, maxlags=50, normed=True, lw=2) ? with a different syntax? I
> get below error when I try the above .
> In [27]: xcorr,lags=ax1.xcorr(x, y, usevlines=True, maxlags=50, normed=True,
> lw=2)
> ---
> ValueError Traceback (most recent call last)
> /home/sjo/work/PY_WORK/stats/ in ()
> > 1 xcorr,lags=ax1.xcorr(x, y, usevlines=True, maxlags=50, normed=True,
> lw=2)
>
> ValueError: too many values to unpack
>
>
>
> import matplotlib.pyplot as plt
> import numpy as np
> x,y = np.random.randn(2,100)
> fig = plt.figure()
> ax1 = fig.add_subplot(211)
> ax1.xcorr(x, y, usevlines=True, maxlags=50, normed=True, lw=2)
> ax1.grid(True)
> ax1.axhline(0, color='black', lw=2)
> ax2 = fig.add_subplot(212, sharex=ax1)
> ax2.acorr(x, usevlines=True, normed=True, maxlags=50, lw=2)
> ax2.grid(True)
> ax2.axhline(0, color='black', lw=2)
> plt.show()
>
>
> From: Paul Hobson
> To: Sudheer Joseph
> Cc: "matplotlib-users@lists.sourceforge.net"
>
> Sent: Thursday, 7 February 2013 10:31 PM
> Subject: Re: [Matplotlib-users] cross correlation
>
>
>
>
> On Thu, Feb 7, 2013 at 3:24 AM, Sudheer Joseph
> wrote:
> Dear Users,
> I am relatively new to Matplotlib. I wanted to find cross
>correlation between 2 time series for my research and was looking at options
>available with python and found
>http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.xcorr . However I
>wanted to save the results in a netcdf file for further use. ie the
>correlat
Re: [Matplotlib-users] cross correlation
Sudheer, For the documentation you are looking for print ax1.xcorr.__doc__ (Paul tried to give you the IPython method of getting that documentation which is by typing a ? (or ??) after the desired object.) In the documentation (at the link you gave http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.xcorr), it says that there are three objects returned by xcorr: Return value is a tuple (*lags*, *c*, *line*) where: - *lags* are a length ``2*maxlags+1`` lag vector - *c* is the ``2*maxlags+1`` auto correlation vector - *line* is a :class:`~matplotlib.lines.Line2D` instance returned by :func:`~matplotlib.pyplot.plot`. So the error you were getting is due to the fact that you have only specified two variables to hold the three returned objects. Try: lags,c,line = ax1.xcorr . (Note that you have xcorr and lags backwards in your attempt.) -Sterling On Feb 8, 2013, at 1:56AM, Sudheer Joseph wrote: > Thank you verymuch Hobson, > However I think I did not understand > the suggestion by you fully( pardon my ignorance). I use the below test code > from matplotlib site. How does one make a call to get lags and correlation > corresponding to the x and y values in the plot. a Print command of > In [23]: print ax1.xcorr > 0x44c1410>> > results as above. Is it possible to assign the xcorr,lags=ax1.xcorr(x, y, > usevlines=True, maxlags=50, normed=True, lw=2) ? with a different syntax? I > get below error when I try the above . > In [27]: xcorr,lags=ax1.xcorr(x, y, usevlines=True, maxlags=50, normed=True, > lw=2) > --- > ValueErrorTraceback (most recent call last) > /home/sjo/work/PY_WORK/stats/ in () > > 1 xcorr,lags=ax1.xcorr(x, y, usevlines=True, maxlags=50, normed=True, > lw=2) > > ValueError: too many values to unpack > > > > import matplotlib.pyplot as plt > import numpy as np > x,y = np.random.randn(2,100) > fig = plt.figure() > ax1 = fig.add_subplot(211) > ax1.xcorr(x, y, usevlines=True, maxlags=50, normed=True, lw=2) > ax1.grid(True) > ax1.axhline(0, color='black', lw=2) > ax2 = fig.add_subplot(212, sharex=ax1) > ax2.acorr(x, usevlines=True, normed=True, maxlags=50, lw=2) > ax2.grid(True) > ax2.axhline(0, color='black', lw=2) > plt.show() > > > From: Paul Hobson > To: Sudheer Joseph > Cc: "matplotlib-users@lists.sourceforge.net" > > Sent: Thursday, 7 February 2013 10:31 PM > Subject: Re: [Matplotlib-users] cross correlation > > > > > On Thu, Feb 7, 2013 at 3:24 AM, Sudheer Joseph > wrote: > Dear Users, > I am relatively new to Matplotlib. I wanted to find cross > correlation between 2 time series for my research and was looking at options > available with python and found > http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.xcorr . However I > wanted to save the results in a netcdf file for further use. ie the > correlation, lags and significance if possible. Is there a way to get the > corr and lags from the axis.xcorr ?? any help in this matter will be greatly > appreciated. > Sudheer > > Sudheer, > > A call to axes.xcorr returns the lags, correlation (from np.correlate) and > the line artists on the figure. > > In IPython, doing "plt.xcorr??" should provide sufficient information. It's a > pretty simple method. > -paul > > > -- > Free Next-Gen Firewall Hardware Offer > Buy your Sophos next-gen firewall before the end March 2013 > and get the hardware for free! Learn more. > http://p.sf.net/sfu/sophos-d2d-feb___ > Matplotlib-users mailing list > Matplotlib-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] cross correlation
Thank you verymuch Hobson, However I think I did not understand the suggestion by you fully( pardon my ignorance). I use the below test code from matplotlib site. How does one make a call to get lags and correlation corresponding to the x and y values in the plot. a Print command of In [23]: print ax1.xcorr > results as above. Is it possible to assign the xcorr,lags=ax1.xcorr(x, y, usevlines=True, maxlags=50, normed=True, lw=2) ? with a different syntax? I get below error when I try the above . In [27]: xcorr,lags=ax1.xcorr(x, y, usevlines=True, maxlags=50, normed=True, lw=2) --- ValueError Traceback (most recent call last) /home/sjo/work/PY_WORK/stats/ in () > 1 xcorr,lags=ax1.xcorr(x, y, usevlines=True, maxlags=50, normed=True, lw=2) ValueError: too many values to unpack import matplotlib.pyplot as plt import numpy as np x,y = np.random.randn(2,100) fig = plt.figure() ax1 = fig.add_subplot(211) ax1.xcorr(x, y, usevlines=True, maxlags=50, normed=True, lw=2) ax1.grid(True) ax1.axhline(0, color='black', lw=2) ax2 = fig.add_subplot(212, sharex=ax1) ax2.acorr(x, usevlines=True, normed=True, maxlags=50, lw=2) ax2.grid(True) ax2.axhline(0, color='black', lw=2) plt.show() From: Paul Hobson To: Sudheer Joseph Cc: "matplotlib-users@lists.sourceforge.net" Sent: Thursday, 7 February 2013 10:31 PM Subject: Re: [Matplotlib-users] cross correlation On Thu, Feb 7, 2013 at 3:24 AM, Sudheer Joseph wrote: Dear Users, > I am relatively new to Matplotlib. I wanted to find cross >correlation between 2 time series for my research and was looking at options >available with python and found >http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.xcorr . However I >wanted to save the results in a netcdf file for further use. ie the >correlation, lags and significance if possible. Is there a way to get the corr >and lags from the axis.xcorr ?? any help in this matter will be greatly >appreciated. >Sudheer Sudheer, A call to axes.xcorr returns the lags, correlation (from np.correlate) and the line artists on the figure. In IPython, doing "plt.xcorr??" should provide sufficient information. It's a pretty simple method. -paul-- Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] cross correlation
On Thu, Feb 7, 2013 at 3:24 AM, Sudheer Joseph wrote: > Dear Users, > I am relatively new to Matplotlib. I wanted to find cross > correlation between 2 time series for my research and was looking at > options available with python and found > http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.xcorr . > However I wanted to save the results in a netcdf file for further use. ie > the correlation, lags and significance if possible. Is there a way to get > the corr and lags from the axis.xcorr ?? any help in this matter will be > greatly appreciated. > Sudheer > Sudheer, A call to axes.xcorr returns the lags, correlation (from np.correlate) and the line artists on the figure. In IPython, doing "plt.xcorr??" should provide sufficient information. It's a pretty simple method. -paul -- Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Cross-correlation?
> So ... does matplotlib have something akin to what I'm after or is > there an extention that might have it? Perhaps the 'correlate' function in NumPy is what you're looking for? Joris Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Cross-correlation?
Ciao Sarah > cross_correlate, but given two arrays, that function returns a single > scalar value (whereas I was expecting a list of correlation > coefficients corresponding to how well the two signals match on > successive lags). That's not really a mpl question, more a 'numerix' one: what does your numerix stand for ? numpy ? numarray ? For numpy, I guess that the corresponding function is 'correlate', which you must use with the 'full' parameter: >>>corxy = N.correlate(x, y, 'full') Unless I'm mistaken, the lags are then 1-n, 2-n,..., -1, 0, 1, ..., n-1 Note that you may want to use anomalies (ie, x-x.mean() and y-y.mean()), and divide by the variances to get proper coefficients A second possibility is to use FFTs, >>>Fx = N.fft.fft(x, npad, axis=axis) >>>Fy = N.fft.fft(y, npad, axis=axis) >>>iFxy = N.fft.ifft(Fx.conj()*Fy,n=npad,axis=axis).real with npad=x.size+y.size The lags are then 0,1,2,...,n,1-n,...,-1 If you're interested, you can use these two functions that I had written #.. def ccf(x, y, axis=None): """Computes the cross-correlation function of two series `x` and `y`. Note that the computations are performed on anomalies (deviations from average). Returns the values of the cross-correlation at different lags. Lags are given as [0,1,2,...,n,n-1,n-2,...,-2,-1]. :Parameters: `x` : 1D MaskedArray Time series. `y` : 1D MaskedArray Time series. `axis` : integer *[None]* Axis along which to compute (0 for rows, 1 for cols). If `None`, the array is flattened first. """ assert(x.ndim == y.ndim, "Inconsistent shape !") #assert(x.shape == y.shape, "Inconsistent shape !") if axis is None: if x.ndim > 1: x = x.ravel() y = y.ravel() npad = x.size + y.size xanom = (x - x.mean(axis=None)) yanom = (y - y.mean(axis=None)) Fx = N.fft.fft(xanom, npad, ) Fy = N.fft.fft(yanom, npad, ) iFxy = N.fft.ifft(Fx.conj()*Fy).real varxy = N.sqrt(N.inner(xanom,xanom) * N.inner(yanom,yanom)) else: npad = x.shape[axis] + y.shape[axis] if axis == 1: if x.shape[0] != y.shape[0]: raise ValueError, "Arrays should have the same length!" xanom = (x - x.mean(axis=1)[:,None]) yanom = (y - y.mean(axis=1)[:,None]) varxy = N.sqrt((xanom*xanom).sum(1) * (yanom*yanom).sum(1)) [:,None] else: if x.shape[1] != y.shape[1]: raise ValueError, "Arrays should have the same width!" xanom = (x - x.mean(axis=0)) yanom = (y - y.mean(axis=0)) varxy = N.sqrt((xanom*xanom).sum(0) * (yanom*yanom).sum(0)) Fx = N.fft.fft(xanom, npad, axis=axis) Fy = N.fft.fft(yanom, npad, axis=axis) iFxy = N.fft.ifft(Fx.conj()*Fy,n=npad,axis=axis).real # return iFxy/varxy #.. def ccf1d(x,y): """Computes the crosscorrelation of two flat arrays `x` and `y`, with the numpy.correlate function. Note that the computations are performed on anomalies (deviations from average). """ if x.ndim > 1: x = x.ravel() if y.ndim > 1: y = y.ravel() (xanom, yanom) = (x-x.mean(), y-y.mean()) corxy = N.correlate(xanom, yanom, 'full') n = min(x.size, y.size) #return N.r_[ xc[len(yf)-1:], 0, xc[:len(yf)-1] ] corxy = N.r_[ corxy[:n][::-1], 0, corxy[n:][::-1] ] varxy = N.sqrt(N.inner(xanom,xanom) * N.inner(yanom,yanom)) return corxy/varxy > > So ... does matplotlib have something akin to what I'm after or is > there an extention that might have it? > > Very many thanks, > > Sarah > > (*) > > http://www.mathworks.com/access/helpdesk/help/toolbox/garch/index.html?/acc >ess/helpdesk/help/toolbox/garch/crosscorr.html > http://www.mathworks.com/access/helpdesk/help/toolbox/signal/index.html?/ac >cess/helpdesk/help/toolbox/signal/xcorr.html - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
