> On 01/07/2008, at 5:07 AM, Antonio P. Ramos wrote: > >> Thanks for the comments. >> >> What I'm doing is very simple: I'm running an one dimensional item >> response model, similar to the ones use in psychology and educational >> testing data via Markov Chain Monte Carlo Methods. >> >> >> model_m12<- ideal(rollcall_m2, maxiter = 500 000 000, thin = 1000, >> burnin = 5000, >> store.item = TRUE, normalize=T, >> priors=list(xp=1e-12,xpv=1e-12,bp=1e-12,bpv=1e-12), verbose=T) >> >> # my data matrix is provided by the rollcall object, but it has just >> 155*17 dimensions: rollcall_m2 >> >> # the number of interactions is maxiter/thin= 500,000 >> >> # store.item=true is the main source of the problem: it's store the >> discrimination >> # parameter, which consumer a large amount of memory. >> Unfortunately, I >> need this information. >> >> >> So, if R can access up to 3.5 Gb how can I fix the problem. I'm sure >> lot's of mac users will be also interested in increasing its memory >> allocation capabilities in R. >> >> > > I would wonder about the need for 500000 MCMC samples. > > Also I assume 17 discriminant parameters so 500000*17*8=68M or 136M > with the difficulty, which seems much less than 1.2G. > > Ken
Hi I am the author of the pscl package in which the ideal() function resides. I agree that 5e8 iterations seems like an awful lot, even thinned by 1e3 down to 5e5 stored iterations. I'd try thinning that out some more, if you really need that many iterations. I've seen some of these rollcall data that do require many iterations to get the standard MCMC convergence diagnostics to look reasonable: e.g., the item parameters for very lop-sided rollcalls, or the latent trait for a legislator/test-taker with an "extreme" voting history/response profile. But even there I'd say "many" \approx 1e6, and there is certainly no need to save them all. also, do note that there is an option in the ideal() function to dump the output of the MCMC algorithm to file, rather than trying to store it all in memory: see ?ideal and the file argument. If you are generating a massive amount of MCMC output this option might help you manage it in a more sane way (say, summarizing it in chunks, either of the parameters or of the saved iterations, etc). and not a R-sig-mac point, but alternative parameterizations can help with these very slow mixing analyses: but at that point you're looking at BUGS/JAGS and friends. Regards -- Simon Jackman Professor Simon Jackman Dept of Political Science, and (by courtesy) Statistics, Director, Political Science Computational Laboratory, Director, Methods of Analysis Program in the Social Sciences, Stanford University, Stanford CA 94305-6044 http://jackman.stanford.edu [[alternative HTML version deleted]] _______________________________________________ R-SIG-Mac mailing list R-SIG-Mac@stat.math.ethz.ch https://stat.ethz.ch/mailman/listinfo/r-sig-mac
