> someone else on this list may be able to give you a ballpark estimate > of how much RAM this merge would require.
I don't have an absolute estimate, but try data.table::merge, as it needs less working memory than base::merge. 20 million rows of 5 columns isn't beyond 32bit : (1*4 + 4*8)*19758564/1024^3 = 0.662GB Also try sqldf to do the join. Matthew "Sharpie" <ch...@sharpsteen.net> wrote in message news:1270102758449-1747733.p...@n4.nabble.com... > > > Janet Choate-2 wrote: >> >> Thanx for clarification on stating my problem, Charlie. >> >> I am attempting to merge to files, i.e.: >> hi39 = merge(comb[,c("hillID","geo")], hi.h39, by=c("hillID")) >> >> if this is relevant or helps to explain: >> the file 'comb' is 3 columns and 1127 rows >> the file 'hi.h39' is 5 columns and 19758564 rows >> >> i started a new clean R session in which i was able to read those 2 files >> in, but get the following error when i try to merge them: >> >> R(2175) malloc: *** mmap(size=79036416) failed (error code=12) >> *** error: can't allocate region >> *** set a breakpoint in malloc_error_break to debug >> R(2175) malloc: *** mmap(size=79036416) failed (error code=12) >> *** error: can't allocate region >> *** set a breakpoint in malloc_error_break to debug >> R(2175) malloc: *** mmap(size=158068736) failed (error code=12) >> *** error: can't allocate region >> *** set a breakpoint in malloc_error_break to debug >> R(2175) malloc: *** mmap(size=158068736) failed (error code=12) >> *** error: can't allocate region >> *** set a breakpoint in malloc_error_break to debug >> R(2175) malloc: *** mmap(size=158068736) failed (error code=12) >> *** error: can't allocate region >> *** set a breakpoint in malloc_error_break to debug >> Error: cannot allocate vector of size 150.7 Mb >> >> so the final error is "Cannot allocate vector of size 150.7 Mb", as >> suggested when R runs out of memory. >> >> i am running R version 2.9.2, on mac os X 10.5 - leopard. >> >> any suggestion on how to increase R's memory on a mac? >> thanx for any much needed help! >> Janet >> > > Ah, so it is indeed a shortage of memory problem. With R 2.9.2, you are > likely running a 32 bit version of R which will be limited to accessing at > most 4 GB of RAM. You may want to try the newest version of R, 2.10.1, as > it includes a 64 bit version that will allow you to access significantly > more memory- provided you have the RAM installed on your system. > > I'm not too hot on memory usage calculation, but someone else on this list > may be able to give you a ballpark estimate of how much RAM this merge > would > require. If it turns out to be a ridiculous amount, you will need to > consider breaking the merge up into chunks or finding an out-of-core (i.e. > not dependent on RAM for storage) merge tool. > > Hope this helps! > > -Charlie > > ----- > Charlie Sharpsteen > Undergraduate-- Environmental Resources Engineering > Humboldt State University > -- > View this message in context: > http://n4.nabble.com/memory-error-tp1747357p1747733.html > Sent from the R help mailing list archive at Nabble.com. > ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.