Re: [Numpy-discussion] Updating cython directory to pxd usage: objections?
On Wed, Jun 18, 2008 at 9:54 PM, Robert Kern [EMAIL PROTECTED] wrote: On Wed, Jun 18, 2008 at 23:48, Fernando Perez [EMAIL PROTECTED] wrote: Stefan, core team? Knock yourself out. Thanks. Committed in r5298. Cheers, f ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Updating cython directory to pxd usage: objections?
2008/6/19 Fernando Perez [EMAIL PROTECTED]: after I updated the pyrex code to use cython, a bit of confusion led me to using .pxi includes instead of .pxd files and cimport. That turned out to be mistaken, as later clarified by the Cython team: http://codespeak.net/pipermail/cython-dev/2008-April/000628.html You weren't the only one confused. In fact, reading the FAQ didn't help me: http://wiki.cython.org/FAQ#head-a3d09805a03c014080feff45fe8e22024d473d62 The NumPy definitions are all external, so why shouldn't they go in a .pxi? Some other questions: - There's a bug in the latest Cython annotation, preventing the example from compiling. Workaround: replace cython -a with cython. - We could also add a scons buildfile, to show how extensions are built with and without numscons. - import_array(). Can we put it at the bottom of c_numpy.pxd, so that users don't have to import it every time? - Author names: should we remove author names from module headers and add them in THANKS.txt or some other contributor list? Regards Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange behavior for argsort() and take()
2008/6/18 Charles R Harris [EMAIL PROTECTED]:
On Wed, Jun 18, 2008 at 11:48 AM, Anne Archibald [EMAIL PROTECTED]
wrote:
2008/6/18 Stéfan van der Walt [EMAIL PROTECTED]:
2008/6/18 Anne Archibald [EMAIL PROTECTED]:
In [7]: x.take(x.argsort())
Out[7]: array([ 0. , 0.1, 0.2, 0.3])
If you would like to think of it more mathematically, when you feed
np.argsort() an array that represents a permutation of the numbers
0,1,...,n-1, you get back the inverse permutation. When you pass a
permutation as the argument to x.take(), you apply the permutation to
x. (You can also invert a permutation by feeding it as indices to
arange(n)).
I have been tempted to write some support functions for manipulating
permutations, but I'm not sure how generally useful they would be.
Do we have an easy way of grabbing the elements out of an array that
correspond to
argmax(x, axis) ?
`take` won't do the job; there `axis` has a different meaning.
Well, here's a quick hack:
def take_axis(X, ix, axis):
XX = np.rollaxis(X,axis)
s = XX.shape
return XX[(ix,)+tuple(np.indices(s[1:]))]
And a version that works for argsort()ing:
def take_axis_argsort(X, ix, axis):
XX = np.rollaxis(X,axis)
ixix = np.rollaxis(ix,axis)
s = XX.shape
return np.rollaxis(XX[(ixix,)+tuple(np.indices(s)[1:])],0,axis+1)
I'm always a little bit leery of using indices(), since it can produce
very large arrays:
And fancy indexing too, as it tends to be slow. The axis=None case also
needs to be handled. I think this is best done in C where the code stacks
for argsort and arg{max,min} are good starting points. The possibility of
different integer types for the index array adds a bit of complication.
Actually, apart from the sizes of the index arrays, I don't really
think fancy indexing can be beat. I suspect that one could write
tuple_indices that used broadcasting to get the index arrays, if you
were willing to return a tuple. That would bring these up to nearly C
speed, I think.
Of course they're not general routines, but I think with a bit of work
they might serve.
Anne
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] nose changes checked in
On Wed, Jun 18, 2008 at 5:24 PM, Robert Kern [EMAIL PROTECTED] wrote: - The signature of numpy.test in 1.2 will be backward compatible with 1.1, and it will at least return some indication of failure (if not the same object as in 1.1). This will, by the way, make it different from the signature and behavior of scipy.test. scipy.test() should be made to match numpy.test(). scipy.testing was a staging ground for the nose changes in numpy, not a separate branch of development. Ok. Jarrod mentioned that the intent is to change SciPy over to use numpy.testing (and remove scipy.testing) once NumPy's nose transition is complete. Is that something that something that will happen simultaneously in the next release, or will SciPy lag behind one cycle? Either way, at some point in the next few weeks I'll try making that change locally just to see if it turns up any surprising requirements in the NumPy side. For my preference, we should accept the old argument signature with a deprecation warning but prefer the current signature. This is a little hairy, but such is life with deprecations. Do you want a blanket warning, or only a warning if test() gets keyword arguments from the old signature? Also, since the numpy.testing.ScipyTestCase and ScipyTest classes already had deprecation warnings in 1.1, I'll leave them out in 1.2, if that's ok. Thanks, Alan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] CFLAGS= causes bad build
Having an empty CFLAGS environment variable and running $ python setup.py build emits tons of these messages: numpy/core/src/scalartypes.inc.src: In function 'scalar_value': numpy/core/src/scalartypes.inc.src:77: warning: dereferencing type-punned pointer will break strict-aliasing rules The compile flags are missing -fno-strict-aliasing. I don't know if this is a problem with the python distutils or what, but it would be nice if it didn't occur. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Fancy index assign ignores extra assignees
Hi all, Is this supposed to be like that, i.e. is the fancy __setitem__ supposed to not complain about unused assignees? v = zeros((10,)) z = [1,2,5] v[z] = [1,2,4,5] v array([ 0., 1., 2., 0., 0., 4., 0., 0., 0., 0.]) Contrast with: v[1:3] = [1,2,3,4] Traceback (most recent call last): File console, line 1, in module ValueError: shape mismatch: objects cannot be broadcast to a single shape Andreas signature.asc Description: This is a digitally signed message part. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] nose changes checked in
On Thu, Jun 19, 2008 at 11:59, Alan McIntyre [EMAIL PROTECTED] wrote: On Wed, Jun 18, 2008 at 5:24 PM, Robert Kern [EMAIL PROTECTED] wrote: - The signature of numpy.test in 1.2 will be backward compatible with 1.1, and it will at least return some indication of failure (if not the same object as in 1.1). This will, by the way, make it different from the signature and behavior of scipy.test. scipy.test() should be made to match numpy.test(). scipy.testing was a staging ground for the nose changes in numpy, not a separate branch of development. Ok. Jarrod mentioned that the intent is to change SciPy over to use numpy.testing (and remove scipy.testing) once NumPy's nose transition is complete. Is that something that something that will happen simultaneously in the next release, or will SciPy lag behind one cycle? No scipy was released with the new scipy.testing stuff, so scipy itself is the only customer. We can transition ourselves. Either way, at some point in the next few weeks I'll try making that change locally just to see if it turns up any surprising requirements in the NumPy side. For my preference, we should accept the old argument signature with a deprecation warning but prefer the current signature. This is a little hairy, but such is life with deprecations. Do you want a blanket warning, or only a warning if test() gets keyword arguments from the old signature? The latter, please. I know it's tricky, but it is doable. Also, since the numpy.testing.ScipyTestCase and ScipyTest classes already had deprecation warnings in 1.1, I'll leave them out in 1.2, if that's ok. Yes. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Updating cython directory to pxd usage: objections?
On Thu, Jun 19, 2008 at 2:31 AM, Stéfan van der Walt [EMAIL PROTECTED] wrote: You weren't the only one confused. In fact, reading the FAQ didn't help me: http://wiki.cython.org/FAQ#head-a3d09805a03c014080feff45fe8e22024d473d62 The NumPy definitions are all external, so why shouldn't they go in a .pxi? I'm not up to speed on all the details, but the cython guys were pretty clear on the final piece of advice, so I'm just taking it at face value here. I haven't followed the discussion on pxi vs pxd in detail (there have been several on their list), but the '.pxi files are mostly a historical artifact' argument led me to stop caring and start using pxds for good. Some other questions: - There's a bug in the latest Cython annotation, preventing the example from compiling. Workaround: replace cython -a with cython. Mmh, I have 0.9.8 (their last release) and it works for me... - We could also add a scons buildfile, to show how extensions are built with and without numscons. Yup. - import_array(). Can we put it at the bottom of c_numpy.pxd, so that users don't have to import it every time? Sounds fine to me, but I don't know if there could be a subtle problem with pxds used by multiple pyx files. I'm not comfortable enough with cython to answer the question. - Author names: should we remove author names from module headers and add them in THANKS.txt or some other contributor list? I left them as they were. If we want to revisit the discussion on per-file author tags vs per-project files that's fine, but it's a separate discussion from the cython technical questions. Cheers, f ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Updating cython directory to pxd usage: objections?
Hi, - There's a bug in the latest Cython annotation, preventing the example from compiling. Workaround: replace cython -a with cython. Mmh, I have 0.9.8 (their last release) and it works for me... I've got the same version, works for me too... Matthew ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Updating cython directory to pxd usage: objections?
2008/6/19 Matthew Brett [EMAIL PROTECTED]: - There's a bug in the latest Cython annotation, preventing the example from compiling. Workaround: replace cython -a with cython. Mmh, I have 0.9.8 (their last release) and it works for me... I've got the same version, works for me too... I'm referring to the latest version from hg. Regards Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] embedded arrays - continued
Hi,
I am still tracking down the memory bug in my C module and I have found a odd
inconsistency.
I call import_array in the init function of the module and I want to check in
the later code if numpy has been initialised before by checking the
PyArray_API
value if it is NULL.
I have found out that PyArray_API in the init function is in a different memory
location than for my later call. Thus I assume that PyArray_API is not static.
Could someone please explain me the code snipplet of the PyArray_API definition
in __multiarray_api.h? What do the constants represent?
#if defined(NO_IMPORT) || defined(NO_IMPORT_ARRAY)
extern void **PyArray_API;
#else
#if defined(PY_ARRAY_UNIQUE_SYMBOL)
void **PyArray_API;
#else
static void **PyArray_API=NULL;
#endif
#endif
Best,
Thomas
-Ursprüngliche Nachricht-
Von: [EMAIL PROTECTED] im Auftrag von Robert Kern
Gesendet: Fr 06.06.2008 15:27
An: Discussion of Numerical Python
Betreff: Re: [Numpy-discussion] embedded arrays
On Fri, Jun 6, 2008 at 17:10, Thomas Hrabe [EMAIL PROTECTED] wrote:
Hi all,
while writing a extension module in C++ for python numpy, I find a strange
error.
I can send and retrieve numpy arrays to and from my module.
But python stops if I do the following:
a = numpy.array([[1.1,2,3],[4,5,6]])
PM.put(a,'a') //send a to the module
b = PM.get('a') //get a identical copy from the module
print b
array([[ 1.1, 2. , 3. ],
[ 4. , 5. , 6. ]])
b.__class__
Out[36]: type 'numpy.ndarray'
perfect, until
a == b
the interpreter does not continue from here...
I can add values to to b, everything, but a == b simply crashes ...?
Does anybody have a clue for this problem?
Not really. It probably depends on some details with your interfacing.
Since we don't have access to your code, we don't have much to go on.
You might have buggy reference counting or perhaps you gave the numpy
ndarray ownership of the array's memory when it actually shouldn't.
Memory issues can be a bit finicky where everything will work for a
while, then crash.
Try running your program under a C debugger like gdb so we can get a
backtrace. That might give us some more ideas about exactly where
problems are occurring.
--
Robert Kern
I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
-- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
winmail.dat___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Cython headers in numpy include
Hi, Following on from Fernando's post about his Cython example, I would like to suggest adding his .pxd files to the standard numpy include directory, fetched with np.get_include() Why: Because anyone writing a Cython extension for numpy will need these files. At the moment, this means that everyone will have need a copy of these files in their package directory or similar, and it will be less likely that people will share any improvements, fixes or extensions. Alternatively, or additionally, add np.get_pxd_include() Any thoughts? Matthew ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is there a function to calculate ecnomic beta coefficient in numpy given two time series data.
I took the following code and applied it to aapl and time series (see attached file): import numpy as np lstsq = np.linalg.lstsq from numpy import float64, extract aapl_array = np.array([row[0] for row in stock_and_market_values]) _array = np.array([row[1] for row in stock_and_market_values]) A = np.ones((len(_array), 2), dtype=float64) A[:,0] = aapl_array result = lstsq(A, _array) print result The result is: (array([ 0.13851625, 29.57888955]), array([ 144.23291488]), 2, array([ 639.591 08529,0.94451427])) And the beta comes out to be 0.138 which is a low. It should be closer to 2. Any idea on what I'm doing wrong. Vineet -Original Message- From: Vineet Jain (gmail) [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 04, 2008 9:24 PM To: 'Discussion of Numerical Python' Subject: RE: [Numpy-discussion] Is there a function to calculate ecnomic beta coefficient in numpy given two time series data. Thanks Keith! -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Keith Goodman Sent: Wednesday, June 04, 2008 9:04 PM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] Is there a function to calculate ecnomic beta coefficient in numpy given two time series data. On Wed, Jun 4, 2008 at 5:39 PM, Vineet Jain (gmail) [EMAIL PROTECTED] wrote: Timeseries1 = daily or weekly close of stock a Timeseries2 = daily or weekly close of market index (spx, , etc) Beta of stock a is what I would like to compute as explained in this article on Wikipedia: http://en.wikipedia.org/wiki/Beta_coefficient I'm trying to compute the beta of entire stock market (about 15,000 instruments) one stock at a time and would like to use the spiders and to represent the overall market. Unless you run out of memory (or if you want to handle missing returns which may occur on different dates in each series) there is no need to do it one stock at a time: import numpy.matlib as mp mrkt = mp.randn(250,1) # - 250 days of returns stocks = mp.randn(250, 4) # 4 stocks beta, resids, rank, s = mp.linalg.lstsq(mrkt, stocks) beta matrix([[-0.01701467, 0.11242168, 0.00207398, 0.03920687]]) And you can use mp.log to convert the price ratios to log returns. It might also be useful to shuffle (mp.random.shuffle) the market returns and repeat the beta calculation many times to estimate the noise level of your beta estimates. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion beta.py Description: Binary data ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ma's maximum_fill_value function in 1.1.0
This is just a minor question/problem with the new numpy.ma in version 1.1.0. Because maximum and minimum in ma lack an accumulate attribute, I've duplicated the functionality using the maximum_fill_value/minimum_fill_value functions and doing something like: np.ma.masked_array(np.maximum.accumulate(np.ma.filled(x, np.ma.maximum_fill_value(x))), x.mask) In version 1.0.4 of numpy, maximum_fill_value/minimum_fill_value were located in ma's namespace so the above line would work. In the latest version, the ma module has an __all__ which does not include maximum_fill_value/minimum_fill_value and I'm forced to get them from numpy.ma.core (which doesn't seem like a very clean way of doing it). So I guess my question is: was this done intentionally and if so why, or can these functions be added to __all__ to get the old behavior back. Thanks for your help with this. -- Reggie Dugard [EMAIL PROTECTED] Merfin, LLC ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Cython headers in numpy include
On Thu, Jun 19, 2008 at 17:46, Matthew Brett [EMAIL PROTECTED] wrote: Hi, Following on from Fernando's post about his Cython example, I would like to suggest adding his .pxd files to the standard numpy include directory, fetched with np.get_include() I am a bit hesitant to add them to a standard public path until they are close to complete. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Cython headers in numpy include
Following on from Fernando's post about his Cython example, I would like to suggest adding his .pxd files to the standard numpy include directory, fetched with np.get_include() I am a bit hesitant to add them to a standard public path until they are close to complete. As usual, fair comment. Do we then have the goal before us of inclusion in a public path when they are close to complete? Matthew ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is there a function to calculate ecnomic beta coefficient in numpy given two time series data.
On Thu, Jun 19, 2008 at 17:48, Vineet Jain (gmail) [EMAIL PROTECTED] wrote: I took the following code and applied it to aapl and time series (see attached file): import numpy as np lstsq = np.linalg.lstsq from numpy import float64, extract aapl_array = np.array([row[0] for row in stock_and_market_values]) _array = np.array([row[1] for row in stock_and_market_values]) A = np.ones((len(_array), 2), dtype=float64) A[:,0] = aapl_array result = lstsq(A, _array) print result The result is: (array([ 0.13851625, 29.57888955]), array([ 144.23291488]), 2, array([ 639.591 08529,0.94451427])) And the beta comes out to be 0.138 which is a low. It should be closer to 2. Any idea on what I'm doing wrong. Beta is supposed to be calculated on returns, not prices. aapl_ret = np.log(aapl_array[1:] / aapl_array[:-1]) _ret = np.log(_array[1:] / _array[:-1]) -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Cython headers in numpy include
On Thu, Jun 19, 2008 at 18:02, Matthew Brett [EMAIL PROTECTED] wrote: Following on from Fernando's post about his Cython example, I would like to suggest adding his .pxd files to the standard numpy include directory, fetched with np.get_include() I am a bit hesitant to add them to a standard public path until they are close to complete. As usual, fair comment. Do we then have the goal before us of inclusion in a public path when they are close to complete? Cython looks in the include_dirs for .pxd files, right? Then yes, I would support putting them alongside arrayobject.h. If possible, we should be careful to make sure that the .pxds work with Pyrex, too. I got burned recently by a change to .pxd semantics in Pyrex 0.9.8, so this could be tricky. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Cython headers in numpy include
Hi, Cython looks in the include_dirs for .pxd files, right? Then yes, I would support putting them alongside arrayobject.h. Yes, right - Fernando just pointed me to the reference - quoting him: As indicated here: http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/version/Doc/Manual/sharing.html#mozTocId559554 the search path for 'cimport foo' to find foo.pxd is the -I search path of the compiler If possible, we should be careful to make sure that the .pxds work with Pyrex, too. I got burned recently by a change to .pxd semantics in Pyrex 0.9.8, so this could be tricky. I don't have deep enough understanding of the differences to give a reliable answer as to whether this is practical. If it wasn't practical, we could either standardize to Cython, or (less likely surely) Pyrex, or have two sets of pxd files, Matthew ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is there a function to calculate ecnomic beta coefficient in numpy given two time series data.
Thanks. With your changes, I'm getting a beta of 0.23 which I think is still wrong. I would have expected a value closer to 2. Vineet -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Robert Kern Sent: Thursday, June 19, 2008 7:08 PM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] Is there a function to calculate ecnomic beta coefficient in numpy given two time series data. On Thu, Jun 19, 2008 at 17:48, Vineet Jain (gmail) [EMAIL PROTECTED] wrote: I took the following code and applied it to aapl and time series (see attached file): import numpy as np lstsq = np.linalg.lstsq from numpy import float64, extract aapl_array = np.array([row[0] for row in stock_and_market_values]) _array = np.array([row[1] for row in stock_and_market_values]) A = np.ones((len(_array), 2), dtype=float64) A[:,0] = aapl_array result = lstsq(A, _array) print result The result is: (array([ 0.13851625, 29.57888955]), array([ 144.23291488]), 2, array([ 639.591 08529,0.94451427])) And the beta comes out to be 0.138 which is a low. It should be closer to 2. Any idea on what I'm doing wrong. Beta is supposed to be calculated on returns, not prices. aapl_ret = np.log(aapl_array[1:] / aapl_array[:-1]) _ret = np.log(_array[1:] / _array[:-1]) -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is there a function to calculate ecnomic beta coefficient in numpy given two time series data.
On Thu, Jun 19, 2008 at 18:18, Vineet Jain (gmail) [EMAIL PROTECTED] wrote: Thanks. With your changes, I'm getting a beta of 0.23 which I think is still wrong. I would have expected a value closer to 2. To get the beta of AAPL versus the index , you need to swap aapl_ret and _ret when you construct the arguments to lstsq(). That gives a beta of 1.0-1.2 depending on whether or not you omit the outlier. Google Finance reports a beta of 2.4 for AAPL recently, but this is probably against the SP 500 rather than a tech index like . Try computing the betas of AAPL and against the SP 500. import numpy as np lstsq = np.linalg.lstsq from numpy import float64, extract aapl_array = np.array([row[0] for row in stock_and_market_values]) aapl_ret = np.log(aapl_array[1:] / aapl_array[:-1]) _array = np.array([row[1] for row in stock_and_market_values]) _ret = np.log(_array[1:] / _array[:-1]) # There's a big outlier in the AAPL returns. AAPL did really well one day. #m = aapl_ret 0.1 #aapl_ret = aapl_ret[m] #_ret = _ret[m] A = np.ones((len(_ret), 2), dtype=float64) A[:,0] = _ret result = lstsq(A, aapl_ret) print result -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Updating cython directory to pxd usage: objections?
2008/6/19 Fernando Perez [EMAIL PROTECTED]: - import_array(). Can we put it at the bottom of c_numpy.pxd, so that users don't have to import it every time? Sounds fine to me, but I don't know if there could be a subtle problem with pxds used by multiple pyx files. I'm not comfortable enough with cython to answer the question. From what I understand, a .pxd file is loaded only once -- is that correct? Either way, I don't think executing import_array more than once could hurt (or could it -- Robert?). Regards Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Updating cython directory to pxd usage: objections?
On Thu, Jun 19, 2008 at 19:02, Stéfan van der Walt [EMAIL PROTECTED] wrote: 2008/6/19 Fernando Perez [EMAIL PROTECTED]: - import_array(). Can we put it at the bottom of c_numpy.pxd, so that users don't have to import it every time? Sounds fine to me, but I don't know if there could be a subtle problem with pxds used by multiple pyx files. I'm not comfortable enough with cython to answer the question. From what I understand, a .pxd file is loaded only once -- is that correct? Either way, I don't think executing import_array more than once could hurt (or could it -- Robert?). I don't think it would cause bad things to happen. It is slightly suboptimal, but only just. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ma's maximum_fill_value function in 1.1.0
Reggie, Good to hear from you. There are no particular reason why maximum/minimum/default_fill_value functions should stay in limbo, I'll put them in __all__. I'll also try to implement the accumulate/reduceat functions for maximum/minimum, using the work you've done on _extrema_functions. Your question raises a good point: is there any consensus on using __all__ instead of the module namespace ? P. On Thursday 19 June 2008 18:55:59 Reggie Dugard wrote: This is just a minor question/problem with the new numpy.ma in version 1.1.0. Because maximum and minimum in ma lack an accumulate attribute, I've duplicated the functionality using the maximum_fill_value/minimum_fill_value functions and doing something like: np.ma.masked_array(np.maximum.accumulate(np.ma.filled(x, np.ma.maximum_fill_value(x))), x.mask) In version 1.0.4 of numpy, maximum_fill_value/minimum_fill_value were located in ma's namespace so the above line would work. In the latest version, the ma module has an __all__ which does not include maximum_fill_value/minimum_fill_value and I'm forced to get them from numpy.ma.core (which doesn't seem like a very clean way of doing it). So I guess my question is: was this done intentionally and if so why, or can these functions be added to __all__ to get the old behavior back. Thanks for your help with this. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Nose, doctests and extension modules
Hi all, as we transition over to nose in full (which I'm very happy to see), we'll want to have a good solution to finding doctests in extension modules. My initial efforts haven't been very fruitful on this front, and if any of you has any insights, I'd appreciate to hear them. I started the conversation on the testing in python mailing list: http://lists.idyll.org/pipermail/testing-in-python/2008-June/000701.html but I'm convinced (for the reasons given there) that the solution I got won't work in general, even if it did work for that one little case. Any insights appreciated. Cheers, f ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ma's maximum_fill_value function in 1.1.0
On Thu, Jun 19, 2008 at 19:15, Pierre GM [EMAIL PROTECTED] wrote: Your question raises a good point: is there any consensus on using __all__ instead of the module namespace ? I'm not sure what you mean. Can you clarify? -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Updating cython directory to pxd usage: objections?
On Thu, Jun 19, 2008 at 19:18, Fernando Perez [EMAIL PROTECTED] wrote: On Thu, Jun 19, 2008 at 5:06 PM, Robert Kern [EMAIL PROTECTED] wrote: On Thu, Jun 19, 2008 at 19:02, Stéfan van der Walt [EMAIL PROTECTED] wrote: 2008/6/19 Fernando Perez [EMAIL PROTECTED]: - import_array(). Can we put it at the bottom of c_numpy.pxd, so that users don't have to import it every time? Sounds fine to me, but I don't know if there could be a subtle problem with pxds used by multiple pyx files. I'm not comfortable enough with cython to answer the question. From what I understand, a .pxd file is loaded only once -- is that correct? Either way, I don't think executing import_array more than once could hurt (or could it -- Robert?). I don't think it would cause bad things to happen. It is slightly suboptimal, but only just. As long as it doesn't cause problems, then I'd vote for doing it. This only happens at module import time, so even if several modules using cython call it, it's only when they get imported. I think the API simplification is worth it (minor, but still nice). I just tested the changes and there seem to be no ill effects. Should I go ahead and commit it? I'll do it in a single commit with no other changes so it's easy to revert should it prove to be a bad idea. Sure. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ma's maximum_fill_value function in 1.1.0
On Thursday 19 June 2008 20:28:46 Robert Kern wrote: On Thu, Jun 19, 2008 at 19:15, Pierre GM [EMAIL PROTECTED] wrote: Your question raises a good point: is there any consensus on using __all__ instead of the module namespace ? I'm not sure what you mean. Can you clarify? __all__ is used on a regular basis in numpy instead of using modules namespace: why ? ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ma's maximum_fill_value function in 1.1.0
On Thu, Jun 19, 2008 at 19:38, Pierre GM [EMAIL PROTECTED] wrote: On Thursday 19 June 2008 20:28:46 Robert Kern wrote: On Thu, Jun 19, 2008 at 19:15, Pierre GM [EMAIL PROTECTED] wrote: Your question raises a good point: is there any consensus on using __all__ instead of the module namespace ? I'm not sure what you mean. Can you clarify? __all__ is used on a regular basis in numpy instead of using modules namespace: why ? The contents of numpy.core.foo, etc., are typically exposed at a higher level, like in numpy.core.__init__ and numpy.__init__. Defining numpy.core.foo.__all__ lets the __init__.py files just from numpy.core.foo import * and have a nearly-sensible namespace. Even if you aren't catering to the from ... import * use case, including a name in __all__ is a strong and useful signal about what is part of the public API for your module. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Updating cython directory to pxd usage: objections?
Hi, As long as it doesn't cause problems, then I'd vote for doing it. This only happens at module import time, so even if several modules using cython call it, it's only when they get imported. I think the API simplification is worth it (minor, but still nice). I just tested the changes and there seem to be no ill effects. Should I go ahead and commit it? I'll do it in a single commit with no other changes so it's easy to revert should it prove to be a bad idea. I may be missing something, but given cython / pyrex are generating c files, as long as you don't call import_array in your pyx file as well as your pxd file, I don't think it could make any difference that several cython modules use the same pxd file. Matthew ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Find groups of true values or sequence of values
2008/6/18 bevan [EMAIL PROTECTED]: Hello, I am looking for some pointers that will hopefully get me a round an issue I have hit. I have a timeseries of river flow and would like to carry out some analysis on the recession periods. That is anytime the values are decreasing. I would like to restrict (mask) my data to any values that are in a continuous sequence of 3 or more (in the example below), that are decreasing in value. Hopefully this example helps: import numpy as np Flow = np.array([15.4,20.5,19.4,18.7,18.6,35.5,34.8,25.1,26.7]) FlowDiff = np.diff(Flow) boolFlowdiff = FlowDiff0 MaskFlow = np.ma.masked_array(Flow[1:],boolFlowdiff) print MaskFlow [-- 19.4 18.7 18.6 -- 34.8 25.1 --] The output I would like is [-- 19.4 18.7 18.6 -- -- -- --] Where the second groups is blanked because the sequence only has 2 members. I would tackle this in steps: find the decreasing pairs, then find places where two of them occur in a row, then construct your mask. Finding decreases: d = diff(X)0 finding two decreases in a row: t = d[1:] d[:-1] creating the right mask: m = np.zeros(n,dtype=np.bool) m[:n-2] |= t m[1:n-1] |= t m[2:n] |= t For finding longer runs you would want other tricks. Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] CFLAGS= causes bad build
Charles R Harris wrote: The compile flags are missing -fno-strict-aliasing. I don't know if this is a problem with the python distutils or what, but it would be nice if it didn't occur. Yes. That's one of the reasons why I started numscons. The current behavior in numpy.distutils is for the same reason as for LDFLAGS: sometimes, this behavior is necessary for unsupported compiler/platform configurations. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Updating cython directory to pxd usage: objections?
On Thu, Jun 19, 2008 at 5:29 PM, Robert Kern [EMAIL PROTECTED] wrote: I just tested the changes and there seem to be no ill effects. Should I go ahead and commit it? I'll do it in a single commit with no other changes so it's easy to revert should it prove to be a bad idea. Sure. Thanks. Done in r5301. Cheers, f ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
