I am attempting to perform some simple data manipulation on a large data set. I have a snippet of the whole data set, and my small snippet is 2GB in CSV.
Is there a way I can read my csv, select a few columns, and write it to an output file in real time? This is what I do right now to a small test file: data <- read.csv('data.csv', header = FALSE) data_filter <- data[c(1,3,4)] write.table(data_filter, file = "filter_data.csv", sep = ",", row.names = FALSE, col.names = FALSE) This test file writes the three columns to my desired output file. Can I do this while bypassing the storage of the entire array in memory? Thank you very much for the help. -- Jason [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.