[Numpy-discussion] Does Numpy support CGI-scripting?`
Does Numpy Support CGI scripting? DO scipy and matplotlib also support? Regards ~ymk ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Import error in builds of 7726
On Mon, Dec 14, 2009 at 5:29 AM, Chris wrote: > Chris gmail.com> writes: > >> Here it is: >> >> http://files.me.com/fonnesbeck/6ezhy5 >> > > Sorry, that link should be: > > http://files.me.com/fonnesbeck/qv8o59 Ok, so the undefined functions all indicate that the most recently implemented ones are not included. I really cannot see any other explanation that having a discrepancy between the source tree, build tree and installation. Sometimes, svn screw things up when switching between branches in my experience, so that's something to check for as well. Could you give us the generated config.h (somewhere in build/src.*/numpy/core/), just in case ? David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] structured array from ordinary array
On Dec 13, 2009, at 8:54 PM, Ernest Adrogué wrote:
> Hi,
>
> How does one generate a structured array from a normal
> array?
>
> I use the 'view' method, but this way I get a superfluous
> dimension that I do not want. Example:
>
> In [619]: a = np.array([[1,2,3],[1,2,3],[1,2,4]],int)
>
> In [620]: struct = np.dtype([('a',int),('b',int),('c',int)])
>
> In [621]: a.view(struct)
...
> I'd like the last array to have shape (3,).
> What am I doing wrong??
Put a .squeeze() after your view()
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Problem with set_fill_value for masked structured array
On Dec 13, 2009, at 6:18 PM, Thomas Robitaille wrote:
> Hi,
>
> The following code doesn't seem to work:
>
> import numpy.ma as ma
>
> t = ma.array(zip([1,2,3],[4,5,6]),dtype=[('a',int),('b',int)])
> print repr(t['a'])
> t['a'].set_fill_value(10)
> print repr(t['a'])
>
> As the output is
>
> masked_array(data = [1 2 3],
> mask = [False False False],
>fill_value = 99)
>
> masked_array(data = [1 2 3],
> mask = [False False False],
>fill_value = 99)
>
> (and no exception is raised)
>
> Am I doing something wrong?
Well, that's a problem indeed, and I'd put that as a bug.
However, you can use that syntax instead:
>>> t.fill_value['a']=10
or set all the fields at once:
>>>t.fill_value=(10,99)
I gonna try to see what I can do, but don't expect it in 1.4.x
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] indices of values contained in a list
12/12/09 @ 08:16 (-0800), thus spake Keith Goodman: > If a and b are as short as in your example, which I doubt, here's a faster > way: > > >> timeit np.nonzero(reduce(np.logical_or, [a == i for i in b])) > 10 loops, best of 3: 14 µs per loop > >> timeit [i for i, z in enumerate(a) if z in b] > 10 loops, best of 3: 3.43 µs per loop > > Looping over a instead of b is faster if len(a) is much less than len(b): > > >> a = np.random.randint(0,100,1) > >> b = tuple(set(a[:50].tolist())) > >> len(b) >41 > >> timeit np.nonzero(reduce(np.logical_or, [a == i for i in b])) > 100 loops, best of 3: 2.65 ms per loop > >> timeit [i for i, z in enumerate(a) if z in b] > 10 loops, best of 3: 37.7 ms per loop Nice. Well speed was not critical in this case, sometimes is good to know alternative ways of doing things. Thanks for your suggestions. Ernest ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] structured array from ordinary array
Hi,
How does one generate a structured array from a normal
array?
I use the 'view' method, but this way I get a superfluous
dimension that I do not want. Example:
In [619]: a = np.array([[1,2,3],[1,2,3],[1,2,4]],int)
In [620]: struct = np.dtype([('a',int),('b',int),('c',int)])
In [621]: a.view(struct)
Out[621]:
array([[(1, 2, 3)],
[(1, 2, 3)],
[(1, 2, 4)]],
dtype=[('a', 'http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Import error in builds of 7726
Chris gmail.com> writes: > Here it is: > > http://files.me.com/fonnesbeck/6ezhy5 > Sorry, that link should be: http://files.me.com/fonnesbeck/qv8o59 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] [Ann] EuroScipy 2010
== Announcing EuroScipy 2010 == --- The 3rd European meeting on Python in Science --- **Paris, Ecole Normale Supérieure, July 8-11 2010** We are happy to announce the 3rd EuroScipy meeting, in Paris, July 2010. The EuroSciPy meeting is a cross-disciplinary gathering focused on the use and development of the Python language in scientific research. This event strives to bring together both users and developers of scientific tools, as well as academic research and state of the art industry. Important dates == == === **Registration opens** Sunday March 29 **Paper submission deadline**Sunday May 9 **Program announced**Sunday May 22 **Tutorials tracks** Thursday July 8 - Friday July 9 **Conference track** Saturday July 10 - Sunday July 11 == === Tutorial = There will be two tutorial tracks at the conference, an introductory one, to bring up to speed with the Python language as a scientific tool, and an advanced track, during which experts of the field will lecture on specific advanced topics such as advanced use of numpy, scientific visualization, software engineering... Main conference topics We will be soliciting talks on the follow topics: - Presentations of scientific tools and libraries using the Python language, including but not limited to: - Vector and array manipulation - Parallel computing - Scientific visualization - Scientific data flow and persistence - Algorithms implemented or exposed in Python - Web applications and portals for science and engineering - Reports on the use of Python in scientific achievements or ongoing projects. - General-purpose Python tools that can be of special interest to the scientific community. Keynote Speaker: Hans Petter Langtangen == We are excited to welcome Hans Petter Langtangen as our keynote speaker. - Director of scientific computing and bio-medical research at Simula labs, Oslo - Author of the famous book Python scripting for computational science http://www.springer.com/math/cse/book/978-3-540-73915-9 -- Gaël Varoquaux, conference co-chair Nicolas Chauvat, conference co-chair Program committee . Romain Brette (ENS Paris, DEC) Mike Müller (Python Academy) Christophe Pradal (CIRAD/INRIA, DigiPlantes team) Pierre Raybault (CEA, DAM) Jarrod Millman (UC Berkeley, Helen Wills NeuroScience institute) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Problem with set_fill_value for masked structured array
Hi,
The following code doesn't seem to work:
import numpy.ma as ma
t = ma.array(zip([1,2,3],[4,5,6]),dtype=[('a',int),('b',int)])
print repr(t['a'])
t['a'].set_fill_value(10)
print repr(t['a'])
As the output is
masked_array(data = [1 2 3],
mask = [False False False],
fill_value = 99)
masked_array(data = [1 2 3],
mask = [False False False],
fill_value = 99)
(and no exception is raised)
Am I doing something wrong?
Thanks in advance for any help,
Thomas
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Import error in builds of 7726
David Cournapeau gmail.com> writes: > could you show the output from nm on umath.so, to check what > symbols are missing. Maybe seeing the whole list would bring > something. Here it is: http://files.me.com/fonnesbeck/6ezhy5 The symbol in question is in there, but I see that it does not have a value. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question on timeseries, for financial application
On Sun, Dec 13, 2009 at 9:27 AM, Robert Ferrell wrote: > > On Dec 13, 2009, at 7:07 AM, josef.p...@gmail.com wrote: > >> On Sun, Dec 13, 2009 at 3:31 AM, Pierre GM >> wrote: >>> On Dec 13, 2009, at 12:11 AM, Robert Ferrell wrote: Have you considered creating a TimeSeries for each data series, and then putting them all together in a dict, keyed by symbol? >>> >>> That's an idea >> >> As far as I understand, that's what pandas.DataFrame does. >> pandas.DataMatrix used 2d array to store data >> >>> One disadvantage of one big monster numpy array for all the series is that not all series may have a full set of 1800 data points. So the array isn't really nicely rectangular. >>> >>> Bah, there's adjust_endpoints to take care of that. >>> Not sure exactly what kind of analysis you want to do, but grabbing a series from a dict is quite fast. >>> >>> Thomas, as robert F. pointed out, everything depends on the kind of >>> analysis you want. If you want to normalize your series, having all >>> of them in a big array is the best (plain array, not structured, so >>> that you can apply .mean and .std directly without having to loop >>> on the series). If you need to apply the same function over all the >>> series, here again having a big ndarray is easiest. Give us an >>> example of what you wanna do. >> >> Or a structured array with homogeneous type that allows fast creation >> of views for data analysis. > > These kinds of financial series don't have that much data (speaking > from the early 21st century point of view). The OP says 1000 series, > 1800 observations per series. Maybe 5 data items per observation, 4 > bytes each. That's well under 50MB. I've found it satisfactory to > keep the data someplace that's handy to get at, and easy to use. When > I want to do analysis I pull it into whatever format is best for that > analysis. Depending on the needs, it may not be necessary to try to > arrange the data so you can get a view for analysis - the time for a > copy can be negligible if the analysis takes a while. > > -r > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > As Josef mentioned, the pandas library is designed for the problem we're discussing-- i.e. working with collections of time series or cross-sectional data. The basic DataFrame object accepts a dict of pandas.Series objects (or a dict of equal-length ndarrays and an array of labels / dates) and provides slicing, reindexing, aggregation, other conveniences. I have not made an official release of the library yet but it is quite robust and suitable for general use (I use it actively in proprietary applications). HTML documentation is also not available yet, but the docstrings are reasonably good. I hope to make an official release by the end of the year with documentation, etc. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] A discrepancy between NumPy documentation and recomendations for beginers
Here
http://www.scipy.org/NumPy_for_Matlab_Users
a recommendation to use scipy.integrate.ode(...) with parameters
"method='bdf', order=15" instead of the ode15s function (from Matlab) is
given. But from the documentation for the scipy.integrate.ode(...) one can
find out that the accuracy order ("order") of the BDF method
("method='bdf'") is no greater than 5. What is the correct value of the
highest order of the BDF method?
I think the answer for this question is necessary for practical use of
scipy.integrate.ode(...).
I tried to migrate my numerical research from Matlab to NumPy and first
tried scipy.integrate.ode(...) function because it supports single step
calculation. I used the default parameters listed in this guide
http://www.scipy.org/NumPy_for_Matlab_Users
and found the calculation process extremely slow. It was unusable because it
selected too small integration step and I could complete only few percents
of my numerical task (a stiff ODE integration) in an admissible time (tens
of minutes). Than I switched my program to use the simpler
scipy.integrate.odeint instead. The scipy.integrate.odeint function gave me
an excellent solution in several seconds. But it does not support single
step integrations and thus is not a good replacement for the ode15s function
(Matlab).
What is the difference between scipy.integrate.ode and
scipy.integrate.odeint? How to select correct parameters for the
scipy.integrate.ode function? After all it seems to me that there might be a
bug in scipy.integrate.ode function implementation...
--
View this message in context:
http://old.nabble.com/A-discrepancy-between-NumPy-documentation-and-recomendations-for-beginers-tp26765626p26765626.html
Sent from the Numpy-discussion mailing list archive at Nabble.com.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ma feature request (log2)
Hi, Could we have a ma aware numpy.ma.log2 please, similar to np.ma.log and np.ma.log10? I think it should be as simple as the patch below but perhaps I've missed something: Thanks, Robin --- core.py.orig2009-12-13 15:14:14.0 + +++ core.py 2009-12-13 15:14:53.0 + @@ -66,7 +66,7 @@ 'identity', 'ids', 'indices', 'inner', 'innerproduct', 'isMA', 'isMaskedArray', 'is_mask', 'is_masked', 'isarray', 'left_shift', 'less', 'less_equal', 'load', 'loads', 'log', 'log10', - 'logical_and', 'logical_not', 'logical_or', 'logical_xor', + 'log2', 'logical_and', 'logical_not', 'logical_or', 'logical_xor', 'make_mask', 'make_mask_descr', 'make_mask_none', 'mask_or', 'masked', 'masked_array', 'masked_equal', 'masked_greater', 'masked_greater_equal', 'masked_inside', 'masked_invalid', @@ -1124,6 +1124,8 @@ _DomainGreater(0.0)) log10 = _MaskedUnaryOperation(umath.log10, 1.0, _DomainGreater(0.0)) +log2 = _MaskedUnaryOperation(umath.log2, 1.0, + _DomainGreater(0.0)) tan = _MaskedUnaryOperation(umath.tan, 0.0, _DomainTan(1e-35)) arcsin = _MaskedUnaryOperation(umath.arcsin, 0.0, ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question on timeseries, for financial application
On Dec 13, 2009, at 7:07 AM, josef.p...@gmail.com wrote: > On Sun, Dec 13, 2009 at 3:31 AM, Pierre GM > wrote: >> On Dec 13, 2009, at 12:11 AM, Robert Ferrell wrote: >>> Have you considered creating a TimeSeries for each data series, and >>> then putting them all together in a dict, keyed by symbol? >> >> That's an idea > > As far as I understand, that's what pandas.DataFrame does. > pandas.DataMatrix used 2d array to store data > >> >>> One disadvantage of one big monster numpy array for all the series >>> is >>> that not all series may have a full set of 1800 data points. So the >>> array isn't really nicely rectangular. >> >> Bah, there's adjust_endpoints to take care of that. >> >>> >>> Not sure exactly what kind of analysis you want to do, but >>> grabbing a >>> series from a dict is quite fast. >> >> Thomas, as robert F. pointed out, everything depends on the kind of >> analysis you want. If you want to normalize your series, having all >> of them in a big array is the best (plain array, not structured, so >> that you can apply .mean and .std directly without having to loop >> on the series). If you need to apply the same function over all the >> series, here again having a big ndarray is easiest. Give us an >> example of what you wanna do. > > Or a structured array with homogeneous type that allows fast creation > of views for data analysis. These kinds of financial series don't have that much data (speaking from the early 21st century point of view). The OP says 1000 series, 1800 observations per series. Maybe 5 data items per observation, 4 bytes each. That's well under 50MB. I've found it satisfactory to keep the data someplace that's handy to get at, and easy to use. When I want to do analysis I pull it into whatever format is best for that analysis. Depending on the needs, it may not be necessary to try to arrange the data so you can get a view for analysis - the time for a copy can be negligible if the analysis takes a while. -r ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question on timeseries, for financial application
On Dec 13, 2009, at 1:31 AM, Pierre GM wrote: > On Dec 13, 2009, at 12:11 AM, Robert Ferrell wrote: >> Have you considered creating a TimeSeries for each data series, and >> then putting them all together in a dict, keyed by symbol? > > That's an idea > >> One disadvantage of one big monster numpy array for all the series is >> that not all series may have a full set of 1800 data points. So the >> array isn't really nicely rectangular. > > Bah, there's adjust_endpoints to take care of that. Maybe this will work for the OP. In my work, if a series is missing data the desirable thing is to use the data I have. I don't' want to truncate existing series to fit the short ones, nor pad to fit the long ones. Really depends on the analysis the OP is trying to do. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question on timeseries, for financial application
On Sun, Dec 13, 2009 at 3:31 AM, Pierre GM wrote: > On Dec 13, 2009, at 12:11 AM, Robert Ferrell wrote: >> Have you considered creating a TimeSeries for each data series, and >> then putting them all together in a dict, keyed by symbol? > > That's an idea As far as I understand, that's what pandas.DataFrame does. pandas.DataMatrix used 2d array to store data > >> One disadvantage of one big monster numpy array for all the series is >> that not all series may have a full set of 1800 data points. So the >> array isn't really nicely rectangular. > > Bah, there's adjust_endpoints to take care of that. > >> >> Not sure exactly what kind of analysis you want to do, but grabbing a >> series from a dict is quite fast. > > Thomas, as robert F. pointed out, everything depends on the kind of analysis > you want. If you want to normalize your series, having all of them in a big > array is the best (plain array, not structured, so that you can apply .mean > and .std directly without having to loop on the series). If you need to apply > the same function over all the series, here again having a big ndarray is > easiest. Give us an example of what you wanna do. Or a structured array with homogeneous type that allows fast creation of views for data analysis. Josef > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Slicing slower than matrix multiplication?
Bruce Southey wrote: > Really I would suggest asking the list for the real problem because it > is often amazing what solutions have been given. So far this is the fastest code I've got: import numpy as np nmax = 100 def minover(Xi,S): P,N = Xi.shape SXi = Xi.copy() for i in xrange(0,P): SXi[i] *= S[i] SXi2 = np.dot(SXi,SXi.T) SXiSXi2divN = np.concatenate((SXi,SXi2),axis=1)/N w = np.random.standard_normal((N)) E = np.dot(SXi,w) wE = np.concatenate((w,E)) for s in xrange(0,nmax*P): mu = wE[N:].argmin() wE += SXiSXi2divN[mu] # E' = dot(SXi,w') #= dot(SXi,w + SXi[mu,:]/N) #= dot(SXi,w) + dot(SXi,SXi[mu,:])/N #= E + dot(SXi,SXi.T)[:,mu]/N #= E + dot(SXi,SXi.T)[mu,:]/N return wE[:N] I am particularly interested in cleaning up the initialization part, but any suggestions for improving the overall performance are of course appreciated. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Import error in builds of 7726
On Sun, Dec 13, 2009 at 12:55 PM, Robert Kern wrote: >> I don't see any build error on this log ? > > See earlier in the thread. The error occurs at runtime: Right. Chris, could you show the output from nm on umath.so, to check what symbols are missing. Maybe seeing the whole list would bring something. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question on timeseries, for financial application
On Dec 13, 2009, at 12:11 AM, Robert Ferrell wrote: > Have you considered creating a TimeSeries for each data series, and > then putting them all together in a dict, keyed by symbol? That's an idea > One disadvantage of one big monster numpy array for all the series is > that not all series may have a full set of 1800 data points. So the > array isn't really nicely rectangular. Bah, there's adjust_endpoints to take care of that. > > Not sure exactly what kind of analysis you want to do, but grabbing a > series from a dict is quite fast. Thomas, as robert F. pointed out, everything depends on the kind of analysis you want. If you want to normalize your series, having all of them in a big array is the best (plain array, not structured, so that you can apply .mean and .std directly without having to loop on the series). If you need to apply the same function over all the series, here again having a big ndarray is easiest. Give us an example of what you wanna do. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
