Thanks lcn, I will try to read data from different chunks.
Best, Kevin On Fri, Apr 26, 2013 at 3:05 PM, lcn <lcn...@gmail.com> wrote: > Do you really have the need loading all the data into memory? > > Mostly for large data set, people would just read a chunk of it for > developing analysis pipeline, and when that's done, the ready script would > just iterate through the entire data set. For example, the read.table > function has 'nrow' and 'skip' parameters to control the reading of data > chunks. > > read.table(file, nrows = -1, skip = 0, ...) > > And another tip here is, you can split the large file into smaller ones. > > > > On Fri, Apr 26, 2013 at 8:09 AM, Kevin Hao <rfans4ch...@gmail.com> wrote: > >> Hi all scientists, >> >> Recently, I am dealing with big data ( >3G txt or csv format ) in my >> desktop (windows 7 - 64 bit version), but I can not read them faster, >> thought I search from internet. [define colClasses for read.table, cobycol >> and limma packages I have use them, but it is not so fast]. >> >> Could you share your methods to read big data to R faster? >> >> Though this is an odd question, but we need it really. >> >> Any suggest appreciates. >> >> Thank you very much. >> >> >> kevin >> >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> > > [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.