On Mon, Jun 8, 2009 at 15:21, <josef.p...@gmail.com> wrote: > 2009/6/8 Stéfan van der Walt <ste...@sun.ac.za>: >> 2009/6/8 Robert Kern <robert.k...@gmail.com>: >>>> Remember, the example is a **teaching** example. >>> >>> I know. Honestly, I would prefer that teachers skip over the normal >>> equations entirely and move directly to decomposition approaches. If >>> you are going to make them implement least-squares from more basic >>> tools, I think it's more enlightening as a student to start with the >>> SVD than the normal equations. >> >> I agree, and I wish our cirriculum followed that route. In linear >> algebra, I also don't much like the way eigenvalues are taught, where >> students have to solve characteristic polynomials by hand. When I >> teach the subject again, I'll pay more attention to these books: >> >> Numerical linear algebra by Lloyd Trefethen >> http://books.google.co.za/books?id=bj-Lu6zjWbEC >> >> (e.g. has SVD in Lecture 4) >> >> Applied Numerical Linear Algebra by James Demmel >> http://books.google.co.za/books?id=lr8cFi-YWnIC >> >> (e.g. has perturbation theory on page 4) >> >> Regards >> Stéfan > > Ok, I also have to give my 2 cents > > Any basic econometrics textbook warns of multicollinearity. Since, > economists are mostly interested in the parameter estimates, the > covariance matrix needs to have little multicollinearity, otherwise > the standard errors of the parameters will be huge. > > If I use automatically pinv or lstsq, then, unless I look at the > condition number and singularities, I get estimates that look pretty > nice, even they have an "arbitrary" choice of the indeterminacy. > > So in economics, I never worried too much about the numerical > precision of the inverse, because, if the correlation matrix is close > to singular, the model is misspecified, or needs reparameterization or > the data is useless for the question. > > Compared to endogeneity bias for example, or homoscedasticy > assumptions and so on, the numerical problem is pretty small. > > This doesn't mean matrix decomposition methods are not useful for > numerical calculations and efficiency, but I don't think the numerical > problem deserves a lot of emphasis in a basic econometrics class.
Actually, my point is a bit broader. Numerics aside, if you are going to bother peeking under the hood of least-squares at all, I think the student gets a better understanding of least-squares via one of the decomposition methods rather than the normal equations. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
