Hello,
I'm trying to read a file rows at a time, so as to not read the entire file
into memory. When reading the "connections" and "readLines" help, and "R help
archive," it seems this should be possible with read.csv and a file connection,
making use of the "nrows" argument, and checking where the "nrow()" of the new
batch is zero rows.
>From certain posts, it seemed that read.csv should return "character(0)" when
>the end of file is reached, and there are no more rows to read. Instead, I
>get an error there are "no lines available for input." Have I made a mistake
>with the file, or calling read.csv?
What is the proper way to check the end-of-file condition with read.csv, such
that I could break a while loop reading the data in?
#example, make a test file
con <- file("test.csv","wt")
cat("a,b,c\n", "1,2,3\n", "4,5,6\n", "7,6,5\n", "4,3,2\n", "3,2,1\n",file=con)
unlink(con)
#show the file is valid
con <- file("test.csv","rt")
read.csv(con,header=T)
unlink(con)
#show that readLines ends with "character(0)", like expected
con <- file("test.csv","rt")
readLines(con,n=10)
readLines(con,n=10)
unlink(con)
#show that read.csv end with error
con <- file("test.csv","rt")
read.csv(con,header=T,nrows=10)
read.csv(con,header=F,nrows=10)
unlink(con)
Sincerely,
Stephen Collins
Predictive Modeler
Allstate Insurance Company
> sessionInfo()
R version 2.15.0 (2012-03-30)
Platform: i386-pc-mingw32/i386 (32-bit)
locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] tools_2.15.0
Stephen Collins
Predictive Modeler
Quantitative Research & Analytics
Allstate Insurance Company
2775 Sanders Road, Suite D2W
Northbrook, IL 60062
t: 1+ 847 402 1465
e: stephen.coll...@allstate.com
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
