Actually I still get an error for the case of equal correlation. Below is the code
> m <- 100000 > n <- 500 > m1 <- 0.5*m > mu <- c(rep(2, m1), rep(0, m-m1)) > rho <- 0.5 > x.mod1 <- matrix(rnorm(n*m, sd=sqrt(1-rho)), nrow=n, ncol=m)+rnorm(n, sd=sqrt(rho))+t(replicate(n,mu)) Error: cannot allocate vector of size 381.5 Mb > > R(173,0xa0a7b540) malloc: *** mmap(size=400003072) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug R(173,0xa0a7b540) malloc: *** mmap(size=400003072) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug R(173,0xa0a7b540) malloc: *** mmap(size=400003072) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug ÔÚ 2012Äê2ÔÂ19ÈÕ ÉÏÎç10:53£¬li li <hannah....@gmail.com>дµÀ£º > > Petr, > Thanks for the help. That certainly makes sense and solves my current > problem. But, in general, for arbitrary covariance matrix (instead of the > special equi-correlation case), it there a way to generate numbers from > multivariate normal when the dimension is very high? > Thanks. > Hannah > > ÔÚ 2012Äê2ÔÂ19ÈÕ ÉÏÎç5:01£¬Petr Savicky <savi...@cs.cas.cz>дµÀ£º > > On Sat, Feb 18, 2012 at 06:00:53PM -0500, li li wrote: >> > Dear all, >> > I need to generate numbers from multivariate normal with large >> dimensions >> > (5,000,000). >> > Below is my code and the error I got from R. Sigma in the code is the >> > covariance >> > matrix. Can anyone give some idea on how to take care of this error. >> Thank >> > you. >> > Hannah >> > >> > > m <- 5000000 >> > > m1 <- 0.5*m >> > > rho <- 0.5 >> > > Sigma <- rho* matrix(1, m, m)+diag(1-rho, m) >> > Error in matrix(1, m, m) : too many elements specified >> >> Hi. >> >> The matrix of dimension m times m does not fit into memory, >> since it requires 8*m^2 = 2e+14 bytes = 2e+05 GB. >> >> Generating a multivariate normal with a covariance matrix >> with 1 on the diagonal and rho outside of the diagonal may >> be done also as follows. >> >> m <- 10 # can be 5000000 >> rho <- 0.5 >> # single vector >> x <- rnorm(1, sd=sqrt(rho)) + rnorm(m, sd=sqrt(1 - rho)) >> >> # several vectors >> a <- t(replicate(10000, rnorm(1, sd=sqrt(rho)) + rnorm(m, sd=sqrt(1 - >> rho)))) >> >> # check the sample covariance matrix if m is not too large >> sigma <- cov(a) >> range(diag(sigma)) # elements on the diagonal >> >> [1] 0.9963445 1.0196015 >> >> diag(sigma) <- NA >> range(sigma, na.rm=TRUE) # elements outside of the diagonal >> >> [1] 0.4935129 0.5162836 >> >> Generating several vectors using replicate() may not be efficient. >> The following can be used instead. >> >> n <- 10000 >> a <- matrix(rnorm(n*m, sd=sqrt(1 - rho)), nrow=n, ncol=m) + rnorm(n, >> sd=sqrt(rho)) >> >> Note that the size of "a" is n times m and it should fit into the memory. >> >> Hope this helps. >> >> Petr Savicky. >> >> ______________________________________________ >> R-help@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> > > [[alternative HTML version deleted]]
______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
