Re: [Numpy-discussion] linear algebra help
Alternatively, to solve A x = b you could do import numpy import numpy.linalg B = numpy.dot(A.T, A) c = numpy.dot(A.T, b) x = numpy.linalg(B,c) This is not the most efficient way to do it but at least you know exactly what's going on in your code. On Sun, May 17, 2009 at 7:21 PM, josef.p...@gmail.com wrote: On Sun, May 17, 2009 at 12:14 PM, Quilby qui...@gmail.com wrote: Right the dimensions I gave were wrong. What do I need to do for m=n (more rows than columns)? Can I use the same function? When I run the script written by Nils (thanks!) I get: from numpy.random import rand, seed ImportError: No module named random But importing numpy works ok. What do I need to install? This should be working without extra install. You could run the test suite, numpy.test(), to see whether your install is ok. Otherwise, you would need to provide more information, numpy version, np.lstsq works for mn, mn and m=n, however the meaning of the solution is different in the 3 cases. mn (more observations than parameters) is the standard least squares estimation problem. Josef Thanks again! On Sun, May 17, 2009 at 1:51 AM, Alan G Isaac ais...@american.edu wrote: On 5/16/2009 9:01 AM Quilby apparently wrote: Ax = y Where A is a rational m*n matrix (m=n), and x and y are vectors of the right size. I know A and y, I don't know what x is equal to. I also know that there is no x where Ax equals exactly y. If m=n, that can only be true if there are not m linearly independent columns of A. Are you sure you have the dimensions right? Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] linear algebra help
2009/5/18 Sebastian Walter sebastian.wal...@gmail.com: B = numpy.dot(A.T, A) This multiplication should be avoided whenever possible -- you are effectively squaring your condition number. In the case where you have more rows than columns, use least squares. For square matrices use solve. For large sparse matrices, use GMRES or any of the others available in scipy.sparse.linalg. Regards Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] linear algebra help
2009/5/18 Stéfan van der Walt ste...@sun.ac.za: 2009/5/18 Sebastian Walter sebastian.wal...@gmail.com: B = numpy.dot(A.T, A) This multiplication should be avoided whenever possible -- you are effectively squaring your condition number. Indeed. In the case where you have more rows than columns, use least squares. For square matrices use solve. For large sparse matrices, use GMRES or any of the others available in scipy.sparse.linalg. It is my impression that this is a linear algebra and not a numerics question. Regards Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] linear algebra help
2009/5/18 Stéfan van der Walt ste...@sun.ac.za 2009/5/18 Sebastian Walter sebastian.wal...@gmail.com: B = numpy.dot(A.T, A) This multiplication should be avoided whenever possible -- you are effectively squaring your condition number. Although the condition number doesn't mean much unless the columns are normalized. Having badly scaled columns can lead to problems with lstsq because of its default cutoff based on the condition number. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] linear algebra help
On Mon, May 18, 2009 at 10:55 AM, Charles R Harris charlesr.har...@gmail.com wrote: 2009/5/18 Stéfan van der Walt ste...@sun.ac.za 2009/5/18 Sebastian Walter sebastian.wal...@gmail.com: B = numpy.dot(A.T, A) This multiplication should be avoided whenever possible -- you are effectively squaring your condition number. Although the condition number doesn't mean much unless the columns are normalized. Having badly scaled columns can lead to problems with lstsq because of its default cutoff based on the condition number. Chuck Do you know if any of the linalg methods, np.linalg.lstsq or scipy.linalg.lstsq, do any normalization internally to improve numerical accuracy? I saw automatic internal normalization (e.g. rescaling) for some econometrics methods, and was wondering whether we should do this also in stats.models or whether scipy.linalg is already taking care of this. I have only vague knowledge of the numerical precision of different linear algebra methods. Thanks, Josef ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] linear algebra help
On Mon, May 18, 2009 at 9:35 AM, josef.p...@gmail.com wrote: On Mon, May 18, 2009 at 10:55 AM, Charles R Harris charlesr.har...@gmail.com wrote: 2009/5/18 Stéfan van der Walt ste...@sun.ac.za 2009/5/18 Sebastian Walter sebastian.wal...@gmail.com: B = numpy.dot(A.T, A) This multiplication should be avoided whenever possible -- you are effectively squaring your condition number. Although the condition number doesn't mean much unless the columns are normalized. Having badly scaled columns can lead to problems with lstsq because of its default cutoff based on the condition number. Chuck Do you know if any of the linalg methods, np.linalg.lstsq or scipy.linalg.lstsq, do any normalization internally to improve numerical accuracy? - They don't. Although, IIRC, lapack provides routines for doing so. Maybe there is another least squares routine that does the scaling. I saw automatic internal normalization (e.g. rescaling) for some econometrics methods, and was wondering whether we should do this also in stats.models or whether scipy.linalg is already taking care of this. I have only vague knowledge of the numerical precision of different linear algebra methods. It's a good idea. Otherwise the condition number depends on choice of units and other such extraneous things. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] linear algebra help
Right the dimensions I gave were wrong. What do I need to do for m=n (more rows than columns)? Can I use the same function? When I run the script written by Nils (thanks!) I get: from numpy.random import rand, seed ImportError: No module named random But importing numpy works ok. What do I need to install? Thanks again! On Sun, May 17, 2009 at 1:51 AM, Alan G Isaac ais...@american.edu wrote: On 5/16/2009 9:01 AM Quilby apparently wrote: Ax = y Where A is a rational m*n matrix (m=n), and x and y are vectors of the right size. I know A and y, I don't know what x is equal to. I also know that there is no x where Ax equals exactly y. If m=n, that can only be true if there are not m linearly independent columns of A. Are you sure you have the dimensions right? Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] linear algebra help
On Sun, May 17, 2009 at 12:14 PM, Quilby qui...@gmail.com wrote: Right the dimensions I gave were wrong. What do I need to do for m=n (more rows than columns)? Can I use the same function? When I run the script written by Nils (thanks!) I get: from numpy.random import rand, seed ImportError: No module named random But importing numpy works ok. What do I need to install? This should be working without extra install. You could run the test suite, numpy.test(), to see whether your install is ok. Otherwise, you would need to provide more information, numpy version, np.lstsq works for mn, mn and m=n, however the meaning of the solution is different in the 3 cases. mn (more observations than parameters) is the standard least squares estimation problem. Josef Thanks again! On Sun, May 17, 2009 at 1:51 AM, Alan G Isaac ais...@american.edu wrote: On 5/16/2009 9:01 AM Quilby apparently wrote: Ax = y Where A is a rational m*n matrix (m=n), and x and y are vectors of the right size. I know A and y, I don't know what x is equal to. I also know that there is no x where Ax equals exactly y. If m=n, that can only be true if there are not m linearly independent columns of A. Are you sure you have the dimensions right? Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] linear algebra help
Hi- This is what I need to do- I have this equation- Ax = y Where A is a rational m*n matrix (m=n), and x and y are vectors of the right size. I know A and y, I don't know what x is equal to. I also know that there is no x where Ax equals exactly y. I want to find the vector x' such that Ax' is as close as possible to y. Meaning that (Ax' - y) is as close as possible to (0,0,0,...0). I know that I need to use either the lstsq function: http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#lstsq or the svd function: http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#svd I don't understand the documentation at all. Can someone please show me how to use these functions to solve my problem. Thanks a lot!!! -quilby ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] linear algebra help
On Sat, 16 May 2009 16:01:00 +0300 Quilby qui...@gmail.com wrote: Hi- This is what I need to do- I have this equation- Ax = y Where A is a rational m*n matrix (m=n), and x and y are vectors of the right size. I know A and y, I don't know what x is equal to. I also know that there is no x where Ax equals exactly y. I want to find the vector x' such that Ax' is as close as possible to y. Meaning that (Ax' - y) is as close as possible to (0,0,0,...0). I know that I need to use either the lstsq function: http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#lstsq or the svd function: http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#svd I don't understand the documentation at all. Can someone please show me how to use these functions to solve my problem. Thanks a lot!!! -quilby ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion I guess you meant a rectangular matrix http://mathworld.wolfram.com/RectangularMatrix.html from numpy.random import rand, seed from numpy import dot, shape from numpy.linalg import lstsq, norm seed(1) m = 10 n = 20 A = rand(m,n) # random matrix b = rand(m) # rhs x,residues,rank,s = lstsq(A,b) print 'Singular values',s print 'Numerical rank of A',rank print 'Solution',x r=dot(A,x)-b print 'residual',norm(r) Cheers, Nils ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] linear algebra help
On Sat, May 16, 2009 at 9:01 AM, Quilby qui...@gmail.com wrote: Hi- This is what I need to do- I have this equation- Ax = y Where A is a rational m*n matrix (m=n), and x and y are vectors of the right size. I know A and y, I don't know what x is equal to. I also know that there is no x where Ax equals exactly y. I want to find the vector x' such that Ax' is as close as possible to y. Meaning that (Ax' - y) is as close as possible to (0,0,0,...0). I know that I need to use either the lstsq function: http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#lstsq or the svd function: http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#svd I don't understand the documentation at all. Can someone please show me how to use these functions to solve my problem. Hi, The new docs are more informative and are being improved in the online editor see http://docs.scipy.org/doc/ and http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.lstsq.html#numpy.linalg.lstsq any comments and improvement to the docs are very welcome. Josef ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] linear algebra help
On 5/16/2009 9:01 AM Quilby apparently wrote: Ax = y Where A is a rational m*n matrix (m=n), and x and y are vectors of the right size. I know A and y, I don't know what x is equal to. I also know that there is no x where Ax equals exactly y. If m=n, that can only be true if there are not m linearly independent columns of A. Are you sure you have the dimensions right? Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
