On Sat, Mar 3, 2012 at 2:36 PM, Peter Langfelder <peter.langfel...@gmail.com> wrote:
> 3. Instead of calculating the correlations one-by-one, calculate them > in small blocks (if you have enough memory and you run a 64-bit R). > With 900M rows, you will only be able to put a 900Mx2 into an R > object, but if you have two such standardized matrices loaded in g1, > g2, you can get their (2x2) correlation matrix by t(g1) %*% g2. This > 2x2 matrix you can use to fill the appropriate components of the > result matrix. Or split it the other way. Compute the covariance of all 9000 variables on, say, 50k observations and store it. Repeat 180 times, then add up the covariances and scale to a correlation. -thomas -- Thomas Lumley Professor of Biostatistics University of Auckland ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.