Re: [R] Non-positive definite cross-covariance matrices
Giovanni, Both matrices describing the points (A and B in my example) are the same size, so the resulting matrix will always be square. Also, the equation I'm using is essentially the following identity: Var(A + B) = Var(A) + Var(B) + Cov(A, B) + Cov(B, A) All the covariance matrices that result from the Var() terms should be positive definite, and while it seems possible that either of those resulting from the Cov() terms may not be, the sum of the two should. Do you agree? Of course, the above identity only holds if the data is normally distributed. The Mardia test for multivariate normality in fact shows that my data is not. This may ultimately be my problem. So maybe I should be asking if you could point me toward some packages that can transform my data so that it is normally distributed. - Jeff On Mon, Nov 15, 2010 at 3:56 PM, Giovanni Petris gpet...@uark.edu wrote: What made you think that a cross-covariance matrix should be positive definite? Id does not even need to be a square matrix, or symmetric. Giovanni Petris On Mon, 2010-11-15 at 12:58 -0500, Jeff Bassett wrote: I am creating covariance matrices from sets of points, and I am having frequent problems where I create matrices that are non-positive definite. I've started using the corpcor package, which was specifically designed to address these types of problems. It has solved many of my problems, but I still have one left. One of the matrices I need to calculate is a cross-covariance matrix. In other words, I need to calculate cov(A, B), where A and B are each a matrix defining a set of points. The corpcor package does not seem to be able to perform this operation. Can anyone suggest a way to create cross-covariance matrices that are guaranteed (or at least likely) to be positive definite, either using corpcor or another package? I'm using R 2.8.1 and corpcor 1.5.2 on Mac OS X 10.5.8. - Jeff __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Non-positive definite cross-covariance matrices
On Tue, Nov 16, 2010 at 9:40 AM, Jeff Bassett jbass...@cs.gmu.edu wrote: Giovanni, Both matrices describing the points (A and B in my example) are the same size, so the resulting matrix will always be square. Also, the equation I'm using is essentially the following identity: Var(A + B) = Var(A) + Var(B) + Cov(A, B) + Cov(B, A) All the covariance matrices that result from the Var() terms should be positive definite, and while it seems possible that either of those resulting from the Cov() terms may not be, the sum of the two should. Do you agree? Of course, the above identity only holds if the data is normally distributed. Hi Jeff, as far as I can see, the identity above is an identity and holds irrespective of how your data is distributed (just write out the difinition of var and cov and you will see that the equation is always true). It is easy to come up with examples where Cov(A, B) + Cov(B, A) is not positive definite. As an extreme example, consider a matrix A (say 10 columns, 100 rows) such that the off-diagonal covariances are all zero and the columns are standardized, so cov(A) = diag(1, 1, 1, ...). Then take B = -A, so cov(A, B) = cov(B, A) = diag(-1, -1, -1, ...). Obviously, cov(A, B) + cov(B, A) is not positively definite, in fact it is negative definite. If the matrices A and B are completely independent, it is not very likely that cov(A,B) + cov(B,A) will be positive definite. For example, if A and B had just one column, there's a 50-50 chance that cov(A, B) is negative (single number). When you have more than one column, the chance is even less than 50-50 because all eigenvalues would have to be positive. You may be able to generate matrices whose cov(A, B) is positive definite by starting with a matrix A, then generating a random matrix B0, subtracting from the columns of B0 the projections of columns of B0 into the columns of A, then adding a small multiple of A to get B. But this may not be what you want or need. Alternatively (and more likely), something in your approach may need re-thinking. HTH, Peter - Jeff On Mon, Nov 15, 2010 at 3:56 PM, Giovanni Petris gpet...@uark.edu wrote: What made you think that a cross-covariance matrix should be positive definite? Id does not even need to be a square matrix, or symmetric. Giovanni Petris On Mon, 2010-11-15 at 12:58 -0500, Jeff Bassett wrote: I am creating covariance matrices from sets of points, and I am having frequent problems where I create matrices that are non-positive definite. I've started using the corpcor package, which was specifically designed to address these types of problems. It has solved many of my problems, but I still have one left. One of the matrices I need to calculate is a cross-covariance matrix. In other words, I need to calculate cov(A, B), where A and B are each a matrix defining a set of points. The corpcor package does not seem to be able to perform this operation. Can anyone suggest a way to create cross-covariance matrices that are guaranteed (or at least likely) to be positive definite, either using corpcor or another package? I'm using R 2.8.1 and corpcor 1.5.2 on Mac OS X 10.5.8. - Jeff __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Non-positive definite cross-covariance matrices
On Tue, Nov 16, 2010 at 1:49 PM, Peter Langfelder peter.langfel...@gmail.com wrote: It is easy to come up with examples where Cov(A, B) + Cov(B, A) is not positive definite. As an extreme example, consider a matrix A (say 10 columns, 100 rows) such that the off-diagonal covariances are all zero and the columns are standardized, so cov(A) = diag(1, 1, 1, ...). Then take B = -A, so cov(A, B) = cov(B, A) = diag(-1, -1, -1, ...). Obviously, cov(A, B) + cov(B, A) is not positively definite, in fact it is negative definite. Peter, I see your point. As it turns out though, what I'm trying to calculate is heritability using a slightly modified version of an equation from multivariate quantitative genetics. Theoretically I suppose a heritability matrix could be non-positive definite, but in practice it almost never is, at least from what I understand. I think I've found a solution to my problem though. The equation I showed before can be rearranged so that the cross-covariance terms are described in terms of the Var() terms. Cov(A, B) + Cov(B, A) = Var(A) + Var(B) - Var(A + B) Since the corpcor package can calculate positive definite versions of all the Var() terms, I can then calculate the sum of the cross-covariance terms from those. I've done some preliminary tests, and it seems to be working quite well. Thanks for all the help, - Jeff __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Non-positive definite cross-covariance matrices
Peter, I see your point. As it turns out though, what I'm trying to calculate is heritability using a slightly modified version of an equation from multivariate quantitative genetics. Theoretically I suppose a heritability matrix could be non-positive definite, but in practice it almost never is, at least from what I understand. I think I've found a solution to my problem though. The equation I showed before can be rearranged so that the cross-covariance terms are described in terms of the Var() terms. Cov(A, B) + Cov(B, A) = Var(A) + Var(B) - Var(A + B) Since the corpcor package can calculate positive definite versions of all the Var() terms, I can then calculate the sum of the cross-covariance terms from those. I've done some preliminary tests, and it seems to be working quite well. Hi Jeff, well, if it works for you, then use it (note that the right hand side of the equation should have reversed sign), although I have to say it's not clear to me how the rearrangement helps. Even if all three variance matrices are positive definite, their sum need not be. Peter Peter __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Non-positive definite cross-covariance matrices
Date: Tue, 16 Nov 2010 17:39:57 -0800 From: peter.langfel...@gmail.com To: jbass...@cs.gmu.edu CC: r-help@r-project.org Subject: Re: [R] Non-positive definite cross-covariance matrices Peter, I see your point. As it turns out though, what I'm trying to calculate is heritability using a slightly modified version of an equation from multivariate quantitative genetics. Theoretically I suppose a heritability matrix could be non-positive definite, but in practice it almost never is, at least from what I understand. I think I've found a solution to my problem though. The equation I showed before can be rearranged so that the cross-covariance terms are described in terms of the Var() terms. Cov(A, B) + Cov(B, A) = Var(A) + Var(B) - Var(A + B) Since the corpcor package can calculate positive definite versions of all the Var() terms, I can then calculate the sum of the cross-covariance terms from those. I've done some preliminary tests, and it seems to be working quite well. Hi Jeff, well, if it works for you, then use it (note that the right hand side of the equation should have reversed sign), although I have to say it's not clear to me how the rearrangement helps. Even if all three variance matrices are positive definite, their sum need not be. Do you have a link to a paper on what you are trying to do? It is quite common in many related expressions to have quadratic terms and then get cross terms that don't act the same way. The elements of A and B are ordered numbers right rather than codes for bases? Are these the formula you are talking about? With scalars order not matter http://mathworld.wolfram.com/Covariance.html usually you expect matricies to be about the same and often are for many special cases ( usually things like AB=BA etc) . In this case, if you look at the defintions, it seems the cross terms differ just in being transpose of each other, http://en.wikipedia.org/wiki/Covariance_matrix Positive-sort-of-definite would seem to imply something about the data. I guess one question might be that given what you think this thing measures and given what you think your data is, should it have some properties like being positive? What does positive mean in the system under consideration? Peter Peter __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Non-positive definite cross-covariance matrices
What made you think that a cross-covariance matrix should be positive definite? Id does not even need to be a square matrix, or symmetric. Giovanni Petris On Mon, 2010-11-15 at 12:58 -0500, Jeff Bassett wrote: I am creating covariance matrices from sets of points, and I am having frequent problems where I create matrices that are non-positive definite. I've started using the corpcor package, which was specifically designed to address these types of problems. It has solved many of my problems, but I still have one left. One of the matrices I need to calculate is a cross-covariance matrix. In other words, I need to calculate cov(A, B), where A and B are each a matrix defining a set of points. The corpcor package does not seem to be able to perform this operation. Can anyone suggest a way to create cross-covariance matrices that are guaranteed (or at least likely) to be positive definite, either using corpcor or another package? I'm using R 2.8.1 and corpcor 1.5.2 on Mac OS X 10.5.8. - Jeff __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
