Re: [Numpy-discussion] [SciPy-dev] NumPy/SciPy + MPI for Python

[eric]

Brian Granger wrote:
> Just as a data point.
>
> I have used mpi4py before and have built it on many systems ranging
> from my macbook to NERSC supercomputers.  In my opinion it is
> currently the best python mpi bindings available.  Lisandro has done a
> fantastic job with this.  Also Fernando and I have worked hard to make
> sure that mpi4py works with the new parallel capabilities of IPython.
>
> I would love to see mpi4py hosted in a public repository for others to
> contribute.  I think this would really solidify mpi4py as a top notch
> mpi interface.  But, my only concern is that there might be many folks
> who want to use mpi4py who don't need scipy.  I am one of those folks
> - I don't necessarily need scipy on the NERSC supercomputers, but I do
> need mpi4py.  Because of this, I would probably still recommend
> keeping mpi4py as a separate project.  Is there any chance it could be
> hosted at mip4py.scipy.org?
>   
Fine from our side...

eric

> I strongly encourage others to try it out.  Installation is is easy.
>
> Brian Granger
>
> On 10/13/06, Lisandro Dalcin <[EMAIL PROTECTED]> wrote:
>   
>> This post is surely OT, but I cannot imagine a better place to contact
>> people about this subject. Please, don't blame me.
>>
>> Any people here interested in NumPy/SciPy + MPI? From some time ago,
>> I've been developing mpi4py (first release at SF) and I am really near
>> to release a new version.
>>
>> This package exposes an API almost identical to MPI-2 C++ bindings.
>> Almost all MPI-1 and MPI-2 features (even one-sided communications and
>> parallel I/O) are fully supported for any object exposing
>> single-segment buffer interface, an only some of them for
>> communication of general Python objects (with the help of
>> pickle/marshal).
>>
>> The posibility of constructing any user-defined MPI datatypes, as well
>> as virtual topologies (specially cartesian), can be really nice for
>> anyone interested in parallel multidimensional array procesing.
>>
>> Before the next release, I would like to wait for any comment, You can
>> contact me via private mail to get a tarbal with latest developments,
>> or we can have some discussion here, if many of you consider this a
>> good idea. In the long term, I would like to see mpi4py integrated as
>> a subpackage of SciPy.
>>
>> Regards,
>>
>> --
>> Lisandro Dalcín
>> ---
>> Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
>> Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
>> Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
>> PTLC - Güemes 3450, (3000) Santa Fe, Argentina
>> Tel/Fax: +54-(0)342-451.1594
>> ___
>> Scipy-dev mailing list
>> [EMAIL PROTECTED]
>> http://projects.scipy.org/mailman/listinfo/scipy-dev
>>
>> 
>
> -
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> ___
> Numpy-discussion mailing list
> Numpy-discussion@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>   


-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] [SciPy-dev] NumPy/SciPy + MPI for Python

[Brian Granger]

Just as a data point.

I have used mpi4py before and have built it on many systems ranging
from my macbook to NERSC supercomputers.  In my opinion it is
currently the best python mpi bindings available.  Lisandro has done a
fantastic job with this.  Also Fernando and I have worked hard to make
sure that mpi4py works with the new parallel capabilities of IPython.

I would love to see mpi4py hosted in a public repository for others to
contribute.  I think this would really solidify mpi4py as a top notch
mpi interface.  But, my only concern is that there might be many folks
who want to use mpi4py who don't need scipy.  I am one of those folks
- I don't necessarily need scipy on the NERSC supercomputers, but I do
need mpi4py.  Because of this, I would probably still recommend
keeping mpi4py as a separate project.  Is there any chance it could be
hosted at mip4py.scipy.org?

I strongly encourage others to try it out.  Installation is is easy.

Brian Granger

On 10/13/06, Lisandro Dalcin <[EMAIL PROTECTED]> wrote:
> This post is surely OT, but I cannot imagine a better place to contact
> people about this subject. Please, don't blame me.
>
> Any people here interested in NumPy/SciPy + MPI? From some time ago,
> I've been developing mpi4py (first release at SF) and I am really near
> to release a new version.
>
> This package exposes an API almost identical to MPI-2 C++ bindings.
> Almost all MPI-1 and MPI-2 features (even one-sided communications and
> parallel I/O) are fully supported for any object exposing
> single-segment buffer interface, an only some of them for
> communication of general Python objects (with the help of
> pickle/marshal).
>
> The posibility of constructing any user-defined MPI datatypes, as well
> as virtual topologies (specially cartesian), can be really nice for
> anyone interested in parallel multidimensional array procesing.
>
> Before the next release, I would like to wait for any comment, You can
> contact me via private mail to get a tarbal with latest developments,
> or we can have some discussion here, if many of you consider this a
> good idea. In the long term, I would like to see mpi4py integrated as
> a subpackage of SciPy.
>
> Regards,
>
> --
> Lisandro Dalcín
> ---
> Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
> Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
> Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
> PTLC - Güemes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
> ___
> Scipy-dev mailing list
> [EMAIL PROTECTED]
> http://projects.scipy.org/mailman/listinfo/scipy-dev
>

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] rcond in polyfit

[A. M. Archibald]

On 13/10/06, Tim Hochberg <[EMAIL PROTECTED]> wrote:
> Charles R Harris wrote:

> > That sounds good, but how to do it? Should I raise an exception?
> Use the warnings framework:
>
>  >>> import warnings
>  >>> warnings.warn("condition number is BAD")
> __main__:1: UserWarning: condition number is BAD
>
> The user can turn warnings on or off or turned in exceptions based on a
> variety of criteria. Look for the warnings filter in the docs.
>
> Which brings up a question: do we want to have a FloatingPointWarning or
> some such? Currently, if you use set the error handling to warn using
> seterr a runtime warning is issued:
>
>  >>> np.seterr(all='warn')
> {'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under':
> 'ignore'}
>  >>> np.arange(1) / 0
> __main__:1: RuntimeWarning: divide by zero encountered in divide
>
>
> On the other hand if error handling is set to 'raise', then a
> FloatingPointError is issued. Is a FloatingPointWarning in order to
> mirror the FloatingPointError? And if so, would it be appropriate to use
> for condition number?

I submitted a patchto use warnings for several functions in scipy a
while ago, and the approach I took was to create a ScipyWarning, from
which more specific warnings were derived (IntegrationWarning, for
example). That was perhaps a bit short-sighted.

I'd suggest a FloatingPointWarning as a base class, with
IllConditionedMatrix as a subclass (it should include the condition
number, but probably not the matrix itself unless it's small, as
debugging information).

The warnings module is frustratingly non-reentrant, unfortunately,
which makes writing tests very awkward.

> > I would also have to modify lstsq so it returns the degree of the fit
> > which would mess up the current  interface.
> One approach would be to write lstsqcond (or a better name) that returns
> both the fit and the condition number. listsq could then be just a
> wrapper over that which dumped the condition number.  IIRC, the
> condition number is available, but we're not returning it.

This is a very good idea. scipy.integrate.quad returns a pair (result,
error_estimate) and every time I use it I trip over that. (Perhaps if
I were a fine upstanding numerical analyst I would be checking the
error estimate every time, but it is a pain.) Another option would be
a "full_output" optional argument.

A. M. Archibald

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] rcond in polyfit

[Tim Hochberg]

Charles R Harris wrote:
>
>
> On 10/13/06, *A. M. Archibald* <[EMAIL PROTECTED] 
> > wrote:
>
> On 12/10/06, Charles R Harris <[EMAIL PROTECTED]
> > wrote:
> > Hi all,
> >
> > I note that polyfit looks like it should work for single and
> double, real
> > and complex, polynomials. On the otherhand, the default rcond
> needs to
> > depend on the underlying precision. On the other, other hand,
> all the svd
> > computations are done with dgelsd or zgelsd, i.e., double
> precision. Even so
> > problems can arise from inherent errors of the input data if it
> is single
> > precision to start with. I also think the final degree of the
> fit should be
> > available somewhere if wanted, as it is an indication of what is
> going on.
> > Sooo, any suggestions as to what to do? My initial impulse would
> be to set
> > rcond=1e-6 for single, 1e-14 for double, make rcond a keyword,
> and kick the
> > can down the road on returning the actual degree of the fit.
>
> I'd also be inclined to output a warning (which the user can ignore,
> read or trap as necessary) if the condition number is too bad or they
> supplied an rcond that is too small for the precision of their data. 
>
>
> That sounds good, but how to do it? Should I raise an exception?
Use the warnings framework:

 >>> import warnings
 >>> warnings.warn("condition number is BAD")
__main__:1: UserWarning: condition number is BAD

The user can turn warnings on or off or turned in exceptions based on a 
variety of criteria. Look for the warnings filter in the docs.

Which brings up a question: do we want to have a FloatingPointWarning or 
some such? Currently, if you use set the error handling to warn using 
seterr a runtime warning is issued:

 >>> np.seterr(all='warn')
{'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under':
'ignore'}
 >>> np.arange(1) / 0
__main__:1: RuntimeWarning: divide by zero encountered in divide


On the other hand if error handling is set to 'raise', then a 
FloatingPointError is issued. Is a FloatingPointWarning in order to 
mirror the FloatingPointError? And if so, would it be appropriate to use 
for condition number?

> I would also have to modify lstsq so it returns the degree of the fit 
> which would mess up the current  interface.
One approach would be to write lstsqcond (or a better name) that returns 
both the fit and the condition number. listsq could then be just a 
wrapper over that which dumped the condition number.  IIRC, the 
condition number is available, but we're not returning it.

-tim




-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] Polyfit

[Greg Willden]

On 10/13/06, A. M. Archibald <[EMAIL PROTECTED]> wrote:
In any case, all this is outside the purview of numpy (as is polyfit, frankly).Great.  Thanks for the ideas of other algorithms/functions to look at.
Greg-- Linux.  Because rebooting is for adding hardware.
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] rcond in polyfit

[A. M. Archibald]

On 13/10/06, Charles R Harris <[EMAIL PROTECTED]> wrote:
>
>
>
> On 10/13/06, A. M. Archibald <[EMAIL PROTECTED]> wrote:
> > On 12/10/06, Charles R Harris <[EMAIL PROTECTED]> wrote:
> > > Hi all,
> > >
> > > I note that polyfit looks like it should work for single and double,
> real
> > > and complex, polynomials. On the otherhand, the default rcond needs to
> > > depend on the underlying precision. On the other, other hand, all the
> svd
> > > computations are done with dgelsd or zgelsd, i.e., double precision.
> Even so
> > > problems can arise from inherent errors of the input data if it is
> single
> > > precision to start with. I also think the final degree of the fit should
> be
> > > available somewhere if wanted, as it is an indication of what is going
> on.
> > > Sooo, any suggestions as to what to do? My initial impulse would be to
> set
> > > rcond=1e-6 for single, 1e-14 for double, make rcond a keyword, and kick
> the
> > > can down the road on returning the actual degree of the fit.
> >
> > I'd also be inclined to output a warning (which the user can ignore,
> > read or trap as necessary) if the condition number is too bad or they
> > supplied an rcond that is too small for the precision of their data.
>
> That sounds good, but how to do it? Should I raise an exception? I would
> also have to modify lstsq so it returns the degree of the fit which would
> mess up the current  interface.

Python's warnings module is a decent solution for providing this
information. Goodness-of-fit worries me less than ill-conditioning -
users are going to expect the curve to deviate from their function
(and an easy reliable way to get goodness of fit is
sqrt(sum(abs(f(xs)-polynomial(xs))**2)); this is certain to take into
account any roundoff errors introduced anywhere). But they may well
have no idea they should be worried about the condition number of some
matrix they've never heard of.

A. M. Archibald

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] Polyfit

[A. M. Archibald]

On 13/10/06, Greg Willden <[EMAIL PROTECTED]> wrote:

> What about including multiple algorithms each returning a figure of fit?
> Then I could try two or three different algorithms and then use the one that
> works best for my data.

The basic problem is that X^n is rarely a good basis for the functions
on [a,b]. So if you want it to return the coefficients of a
polynomial, you're basically stuck. If you *don't* want that, there's
a whole bestiary of other options.

If you're just looking to put a smooth curve through a bunch of data
points (perhaps with known uncertainties), scipy.interpolate includes
some nice spline fitting functions.

If you're looking for polynomials, orthogonal polynomials may serve as
a better basis for your interval; you can look in scipy.special for
them (and leastsq will fit them to your points). Extracting their
coefficients is possible but will bring you back to numerical
instabilities.

In any case, all this is outside the purview of numpy (as is polyfit, frankly).

A. M. Archibald

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] rcond in polyfit

[Charles R Harris]

On 10/13/06, A. M. Archibald <[EMAIL PROTECTED]> wrote:
On 12/10/06, Charles R Harris <[EMAIL PROTECTED]> wrote:> Hi all,>> I note that polyfit looks like it should work for single and double, real
> and complex, polynomials. On the otherhand, the default rcond needs to> depend on the underlying precision. On the other, other hand, all the svd> computations are done with dgelsd or zgelsd, i.e., double precision. Even so
> problems can arise from inherent errors of the input data if it is single> precision to start with. I also think the final degree of the fit should be> available somewhere if wanted, as it is an indication of what is going on.
> Sooo, any suggestions as to what to do? My initial impulse would be to set> rcond=1e-6 for single, 1e-14 for double, make rcond a keyword, and kick the> can down the road on returning the actual degree of the fit.
I'd also be inclined to output a warning (which the user can ignore,read or trap as necessary) if the condition number is too bad or theysupplied an rcond that is too small for the precision of their data.
That sounds good, but how to do it? Should I raise an exception? I would also have to modify lstsq so it returns the degree of the fit which would mess up the current  interface. Chuck

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] Polyfit

[Tim Hochberg]

Greg Willden wrote:
> On 10/13/06, *A. M. Archibald* <[EMAIL PROTECTED] 
> > wrote:
>
> At this point you might as well use a polynomial class that can
> accomodate a variety of bases for the space of polynomials - X^n,
> (X-a)^n, orthogonal polynomials (translated and scaled as needed),
> what have you.
>
> I think I vote for polyfit that is no more clever than it has to be
> but which warns the user when the fit is bad.
>
>
>
> What about including multiple algorithms each returning a figure of fit?
> Then I could try two or three different algorithms and then use the 
> one that works best for my data.
A simple, "stupid" curve fitting algorithm may be appropriate for numpy, 
but once your getting into multiple algorithms it's time to move it to a 
package in scipy IMO (and it would be good to find someone who cares, 
and knows, about curve fitting to adopt it).

-tim


-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] Polyfit

[Greg Willden]

On 10/13/06, A. M. Archibald <[EMAIL PROTECTED]> wrote:
At this point you might as well use a polynomial class that canaccomodate a variety of bases for the space of polynomials - X^n,(X-a)^n, orthogonal polynomials (translated and scaled as needed),what have you.
I think I vote for polyfit that is no more clever than it has to bebut which warns the user when the fit is bad.What about including multiple algorithms each returning a figure of fit?
Then I could try two or three different algorithms and then use the one that works best for my data.Greg-- Linux.  Because rebooting is for adding hardware.
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] Testing numpy without doing an installation?

[Lisandro Dalcin]

On 10/13/06, Francesc Altet <[EMAIL PROTECTED]> wrote:
> Is it possible to test a numpy version directly from the source
> directory without having to install it?

I usually do:

$ python setup.py build
$ python setup.py install --home=/tmp
$ export PYTHONPATH=/tmp/lib/python

and then

$ python -c 'import numpy; numpy.test()'

and finally,  if all was right,

su -c 'python setup.py install'

or

python setup.py install --home=$HOME

-- 
Lisandro Dalcín
---
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

[Numpy-discussion] NumPy/SciPy + MPI for Python

[Lisandro Dalcin]

This post is surely OT, but I cannot imagine a better place to contact
people about this subject. Please, don't blame me.

Any people here interested in NumPy/SciPy + MPI? From some time ago,
I've been developing mpi4py (first release at SF) and I am really near
to release a new version.

This package exposes an API almost identical to MPI-2 C++ bindings.
Almost all MPI-1 and MPI-2 features (even one-sided communications and
parallel I/O) are fully supported for any object exposing
single-segment buffer interface, an only some of them for
communication of general Python objects (with the help of
pickle/marshal).

The posibility of constructing any user-defined MPI datatypes, as well
as virtual topologies (specially cartesian), can be really nice for
anyone interested in parallel multidimensional array procesing.

Before the next release, I would like to wait for any comment, You can
contact me via private mail to get a tarbal with latest developments,
or we can have some discussion here, if many of you consider this a
good idea. In the long term, I would like to see mpi4py integrated as
a subpackage of SciPy.

Regards,

-- 
Lisandro Dalcín
---
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] Polyfit

[A. M. Archibald]

On 13/10/06, Charles R Harris <[EMAIL PROTECTED]> wrote:

> You can also get *much* better results if you scale the x interval to [0,1]
> as the problem will be better posed. For instance, with your data and a
> degree 10 fit I get  a condition number of about 2e7 when x is scaled to
> [0,1], as opposed to about 1e36 when left as is. The former yields a
> perfectly useable fit while the latter blows up. I suppose this could be
> built into the polyfit routine if one were only interested in polynomial
> fits of some sort, but the polynomial would have to carry around an offset
> and scale factor to make evaluation work.

[-1,1] would probably be even better, no?

> If Travis is interested in such a thing we could put together some variant
> of the polynomials that includes the extra data.

At this point you might as well use a polynomial class that can
accomodate a variety of bases for the space of polynomials - X^n,
(X-a)^n, orthogonal polynomials (translated and scaled as needed),
what have you.

I think I vote for polyfit that is no more clever than it has to be
but which warns the user when the fit is bad.

A. M. Archibald

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] rcond in polyfit

[A. M. Archibald]

On 12/10/06, Charles R Harris <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I note that polyfit looks like it should work for single and double, real
> and complex, polynomials. On the otherhand, the default rcond needs to
> depend on the underlying precision. On the other, other hand, all the svd
> computations are done with dgelsd or zgelsd, i.e., double precision. Even so
> problems can arise from inherent errors of the input data if it is single
> precision to start with. I also think the final degree of the fit should be
> available somewhere if wanted, as it is an indication of what is going on.
> Sooo, any suggestions as to what to do? My initial impulse would be to set
> rcond=1e-6 for single, 1e-14 for double, make rcond a keyword, and kick the
> can down the road on returning the actual degree of the fit.

I'd also be inclined to output a warning (which the user can ignore,
read or trap as necessary) if the condition number is too bad or they
supplied an rcond that is too small for the precision of their data.

A. M. Archibald

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] Polyfit

[Charles R Harris]

On 10/12/06, Greg Willden <[EMAIL PROTECTED]> wrote:
On 10/12/06, Charles R Harris <[EMAIL PROTECTED]
> wrote:
And here is the location of the problem in numpy/linalg/linalg.py :def lstsq(a, b, rcond=1.e-10): The 1e-10 is a bit conservative. On the other hand, I will note that the condition number of the dot(V^T ,V) matrix is somewhere around 1e22, which means in general terms that you need around 22 digits of accuracy. Inverting it only works sorta by accident in the current case. Generally, using Vandermonde matrices and polynomial fits it a bad idea when the dynamic range of the interval gets large and the degree gets up around 4-5 as it leads to ill conditioned sets of equations. When you really need the best start with chebychev polynomials or, bestest, compute a set of polynomials orthogonal over the sample points. Anyway, I think rcond should be something like 1e-12 or 1e-13 by default and be available as a keyword in the polyfit function. If no one complains I will make this change, although it is just a bandaid and things will fall apart again as soon as you call polyfit(x,y,4).
Hey that's great.  I'm glad you tracked it down.Pardon my ignorance of polyfit algorithm details.Is there a way of choosing rcond based on N that would give sensible defaults for a variety of N?
GregYou can also get *much* better results if you scale the x interval to [0,1] as the problem will be better posed. For instance, with your data and a degree 10 fit I get  a condition number of about 2e7 when x is scaled to [0,1], as opposed to about 1e36 when left as is. The former yields a perfectly useable fit while the latter blows up. I suppose this could be built into the polyfit routine if one were only interested in polynomial fits of some sort, but the polynomial would have to carry around an offset and scale factor to make evaluation work.
If Travis is interested in such a thing we could put together some variant of the polynomials that includes the extra data.Chuck
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] Library problem on installation

[Michael Subotin]

Following up on my own message for archival purposes, after getting
local help. If you're having a problem like this, read the file
called  INSTALL.txt.  The current NumPy tarball doesn't have
this file, but the SciPy tarball does. You may need to reinstall
Atlas/Lapack libraries using a different compiler. 

MichaelOn 10/11/06, Michael Subotin <[EMAIL PROTECTED]> wrote:
Hi,

I know that it's a perennial topic on the list, but I haven't been able
to find my answer in the archives. After running the installation on a
RedHat Linux machine, I'm getting the import error:
"/usr/lib/libblas.so.3: undefined symbol: e_wsfe". Judging from earlier
exchanges here, it seems that I need to add libg2c (which this machine
does have in /usr/lib, unlike g2c) somewhere between 'f77blas' and
'cblas', but I'm not sure where I should make this change. Not being
well versed in Python distributions, I tried my luck with a few
candidates and the import error remains. The machine should be running
gcc.

Thanks for any help.

Michael 


-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

[Numpy-discussion] dtype always copies

[Stefan van der Walt]

Hi all,

I've noticed that 'astype' always forces a copy.  Is this
behaviour intended?  It seems to conflict with 'asarray', that
tries to avoid a copy.

For example, when wrapping code in ctypes, the following snippet
would have been useful:

def foo(x):
# ensure x is an array of the right type
x = N.ascontiguousarray(x).astype(N.intc)

but that will cause a copy, so you'll have to do

def foo(x):
try:
x = N.ascontiguousarray(x,N.intc)
except:
x = N.ascontiguousarray(x).astype(N.intc)

Maybe I'm missing something obvious here -- any pointers?

Thanks
Stéfan

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

[Numpy-discussion] Testing numpy without doing an installation?

[Francesc Altet]

Hi,

Is it possible to test a numpy version directly from the source
directory without having to install it? I mean, if I compile the sources
and try to use the package directly from there, I get unexpected
results. For example:

$ export PYTHONPATH=/home/faltet/python.nobackup/numpy/trunk
$ python2.4 -c "import numpy;print numpy.dtype([('col1', '(1,)i4')])"
Running from numpy source directory.
Traceback (most recent call last):
  File "", line 1, in ?
AttributeError: 'module' object has no attribute 'dtype'

It would be nice to have a way of testing a recently built version of
numpy prior to install it.

Thanks,

-- 
>0,0<   Francesc Altet http://www.carabos.com/
V   V   Cárabos Coop. V.   Enjoy Data
 "-"

"Be careful about using the following code --
 I've only proven that it works, I haven't tested it." -- Donald Knuth


-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] How to make sqrt(-1) be 1j

[Tim Hochberg]

Bill Baxter wrote:
> On 10/13/06, Tim Hochberg <[EMAIL PROTECTED]> wrote:
>   
>> For this sort of thing, I
>> would just make a new module to pull together the function I want and
>> use that instead. It's then easy to explain that this new module bbeconf
>> (Bill Baxter's Excellent Collection Of Numeric Functions) is actually an
>> amalgamation of stuff from multiple sources.
>>
>> # bbeconf.py
>> from numpy import *
>> fromnumpy.scimath import sqrt
>> # possibly some other stuff to correctly handle subpackages...
>> 
>
> That does sound like a good way to do it.
> Then you just tell your users to import 'eduNumpy'  rather than numpy,
> and you're good to go.
> Added that suggestion to http://www.scipy.org/NegativeSquareRoot
>
> I'd like to ask one basic Python question related my previous
> suggestion of doing things like "numpy.sqrt = numpy.lib.scimath.sqrt":
> In python does that make it so that any module importing numpy in the
> same program will now see the altered sqrt function?  E.g. in my
> program I do "import A,B".  Module A alters numpy.sqrt.  Does that
> also modify how module B sees numpy.sqrt?
>   
Indeed it does. Module imports are cached in sys.modules, so numpy is 
only imported once. (With some effort, you can usually get your own 
private copy of a module, that you could mess with to your hearts 
content, but I generally wouldn't recommend it).
> If so then that's a very good reason not to do it that way.
>
> I've heard people using the term "monkey-patch" before.  Is that what that is?
>   
I believe that is what the term refers to although I'm not absolutely 
certain.

-tim




-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

[Numpy-discussion] SVN checkout problem

[Greg Willden]

Hi, I just tried to checkout numpy and scipy to another machine and got the following errors:$ svn co http://svn.scipy.org/svn/numpy/trunk numpysvn: REPORT request failed on '/svn/numpy/!svn/vcc/default'
svn: REPORT of '/svn/numpy/!svn/vcc/default': 400 Bad Request (http://svn.scipy.org)$ svn co http://svn.scipy.org/svn/scipy/trunk
 scipysvn: REPORT request failed on '/svn/scipy/!svn/vcc/default'svn: REPORT of '/svn/scipy/!svn/vcc/default': 400 Bad Request (http://svn.scipy.org)Any ideas?
Greg-- Linux.  Because rebooting is for adding hardware.
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

[Numpy-discussion] Those young people taking the position you deserve because you lack a Degree?

[[EMAIL PROTECTED]]

Yeah, it does, said Tonks decisively. She screwed up


Ill come and help you, said Tonks brightly.eye remained focused on the ceiling. Damn it, he added  restless energy that made him unable to settle to anything, duringGet me a glass of water, would you, Harry, requested Moody.Can you learn how to be a Metamorphmagus? Harry 
 whats going on, I havent heard anything from anyone, whats Vol—?Back in the kitchen Moody had replaced his eye, which was This is Alastor Moody, Harry Lupin continued, pointing towards Moody. Lower and lower they flew, until Harry could see individual headlights and  teeth and clenching his fists, casting angry looks out at the empty, 
 when Harry had last said goodbye to him and his robes wereHarrys eyes watered in the chill as they soared upwards; he could see . Shivering, Harry looked around. The grimy fronts of the surrounding Come here, boy, said Moody gruffly, beckoning Harry She followed Harry back into the hall and up the stairs,
She followed Harry back into the hall and up the stairs, window and Dudley had hit him were throbbing painfully. I want three hundred and sixty degrees visibility on the return journey.collection of lights he had yet seen, a huge, sprawling crisscrossing mass,the real Potter would know. Unless anyone brought any Veritaserum?
 wand like a conductors baton, Tonks made the trunk hover staring at Harrys midriff. movement over the floor.Can you learn how to be a Metamorphmagus? Harry Harry touched down right behind her and dismounted on a patch of unkempt
and every thought of the Ministry hearing was swept from his mind Er - yeah, said Harry. Look - he turned back to Lupin,quite blank, when his uncle entered his bedroom. Harry looked slowly still eyeing Harry curiously. Too risky. Weve set up Disillusionment Charm, said Moody, raising his wand. 
Harrys eyes watered in the chill as they soared upwards; he could see Stop being so cheerful, Mad-Eye, hell think were not taking this  And I saw that, he added irritably, as the woman rolledCan you learn how to be a Metamorphmagus? Harry read Harrys mind; the corners of his mouth twitched slightly.
he was going home… for a few glorious moments, all his problems seemed to full of rage about the non-existent Lawn Competition… and Harry laughed felt a curious sensation as though Moody had just Well, youll have to learn the hard way, Im afraid, dead clumsy, did you hear me break that plate when we 
 dead clumsy, did you hear me break that plate when we Borrowed it from Dumbledore, growled Moody, pocketing the Put-Outer.Hedwig gave a muffled hooting noise, her beak still full of frog. to land him in a cell in Azkaban? Whenever this thought occurred, it, so as to illuminate the writing. Read quickly and memorise.
slightly in the moonlight… now Emmeline Vance was on his right, her wand out,being this cold on a broom only once before, during the Quidditch sliding off the bed on to his feet - but a split second later it occurred  Lawn Competition.Time to start the descent! came Lupins voice. Follow Tonks, Harry!


-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion