Re: [R] Problem reading mixed CSV file

2012-03-26 Thread Ashish Agarwal
Why does the output in the following say 2 and not 6? count.fields(textConnection(LL1532Ap,ABC# Depot-A+,,1971,8,2 + LL1532Ap,ABC# Depot-A+,Bhutan,1971,6,1 + LL1532Ap,ABC# Depot-A+,China,1971,17,1 + LL1532Ap,ABC# Depot-A+,China,1971,33,1 + LL1532Ap,ABC# Depot-A+,HongKong,1971,16,2 +

Re: [R] Problem reading mixed CSV file

2012-03-26 Thread Berend Hasselman
On 26-03-2012, at 08:16, Ashish Agarwal wrote: Why does the output in the following say 2 and not 6? count.fields(textConnection(LL1532Ap,ABC# Depot-A+,,1971,8,2 + LL1532Ap,ABC# Depot-A+,Bhutan,1971,6,1 + LL1532Ap,ABC# Depot-A+,China,1971,17,1 + LL1532Ap,ABC# Depot-A+,China,1971,33,1 +

Re: [R] Problem reading mixed CSV file

2012-03-26 Thread Ashish Agarwal
OMG. I think it uses comment character # as default in the argument. comment.char = # How do I turn it off? On Mon, Mar 26, 2012 at 11:56 AM, Berend Hasselman b...@xs4all.nl wrote: On 26-03-2012, at 08:16, Ashish Agarwal wrote: Why does the output in the following say 2 and not 6?

Re: [R] Problem reading mixed CSV file

2012-03-26 Thread Berend Hasselman
On 26-03-2012, at 08:26, Berend Hasselman wrote: On 26-03-2012, at 08:16, Ashish Agarwal wrote: Why does the output in the following say 2 and not 6? count.fields(textConnection(LL1532Ap,ABC# Depot-A+,,1971,8,2 + LL1532Ap,ABC# Depot-A+,Bhutan,1971,6,1 + LL1532Ap,ABC#

Re: [R] Problem reading mixed CSV file

2012-03-26 Thread Ashish Agarwal
comment.char = NULL does not work. Is there any way to make it NULL rather than having a specific character like '%'? On Mon, Mar 26, 2012 at 12:06 PM, Berend Hasselman b...@xs4all.nl wrote: comment.char = # How do I turn it off? ??? How about comment.car=% for example? Berend

Re: [R] Problem reading mixed CSV file

2012-03-26 Thread Berend Hasselman
On 26-03-2012, at 08:40, Ashish Agarwal wrote: comment.char = NULL does not work. Is there any way to make it NULL rather than having a specific character like '%'? Why don't you try something? comment.char= looks quite obvious and worth a try. Berend

Re: [R] Problem reading mixed CSV file

2012-03-26 Thread Berend Hasselman
On 26-03-2012, at 08:33, Ashish Agarwal wrote: OMG. I think it uses comment character # as default in the argument. comment.char = # How do I turn it off? ??? How about comment.car=% for example? Berend __ R-help@r-project.org mailing

Re: [R] Problem reading mixed CSV file

2012-03-26 Thread Ashish Agarwal
thanks. works very well. On Mon, Mar 26, 2012 at 12:12 PM, Berend Hasselman b...@xs4all.nl wrote: On 26-03-2012, at 08:40, Ashish Agarwal wrote: comment.char = NULL does not work. Is there any way to make it NULL rather than having a specific character like '%'? Why don't you try

Re: [R] Problem reading mixed CSV file

2012-03-26 Thread Ashish Agarwal
Need to fix up the file having 6 and 7 columns to be read as 6 columns only. Here is the working. Can somebody please let me know how do I maintain the order in which rows were read and append the two files into one: count.fields(textConnection(LL1532Ap,ABC# Depot-A+,,1971,8,2 LL1532Ap,ABC#

Re: [R] Problem reading mixed CSV file

2012-03-20 Thread Ashish Agarwal
The file is 20MB having 2 Million rows. I understand that I two different formats - 6 columns and 7 columns. How do I read chunks to different files by using scan with modifying skip and nlines parameters? On Mon, Mar 19, 2012 at 3:59 PM, Petr PIKAL petr.pi...@precheza.cz wrote: I would follow

Re: [R] Problem reading mixed CSV file

2012-03-20 Thread jim holtman
use 'count.fields' to determine which line have 6 and 7 fields in them. then use 'readLines' to read in the entire file and the use the data from count.fields to write out to separate files x - count.fields(...) input - readLines(..) writeLines(input[x == 6], file = '6fields.csv')

Re: [R] Problem reading mixed CSV file

2012-03-20 Thread Ashish Agarwal
Given x- count.fields(..) could you pls help in following: 1. how to create a data vector with data being line numbers of original file where x==6? 2. what is the way to read only the nth line (only) of an input file into a data vector with first three attributes to be read as string, 4th being

Re: [R] Problem reading mixed CSV file

2012-03-20 Thread jim holtman
On Tue, Mar 20, 2012 at 3:17 PM, Ashish Agarwal ashish.agarw...@gmail.com wrote: Given x- count.fields(..) could you pls help in following: 1. how to create a data vector with data being line numbers of original file where x==6? That is what the expression: writeLines(input[x == 6], file =

Re: [R] Problem reading mixed CSV file

2012-03-19 Thread Ashish Agarwal
This is quite a CPu consuming process. My system got hung up for the big file I have. Within the for loop that you have suggested, can't I have a case statement for different value of nfields to be read and specify what format does the variable needs to be read? something like case # input format

Re: [R] Problem reading mixed CSV file

2012-03-19 Thread Jim Holtman
How big is the file? In the example I sent I waa using 'textConnection' to reread the input. If the file is large, this can be slow. You will have better luck writing the converted data outmto a temporarynfile and reading it right back in. I am not such exactly what you are asking. You can

Re: [R] Problem reading mixed CSV file

2012-03-19 Thread Petr PIKAL
Hi This is quite a CPu consuming process. My system got hung up for the big file I have. Within the for loop that you have suggested, can't I have a case statement for different value of nfields to be read and specify what format does the variable needs to be read? something like case

[R] Problem reading mixed CSV file

2012-03-16 Thread Ashish Agarwal
I am having trouble reading this CSV file in R. There are six attributes that I need to read - CVar1, CVar2, Location, Year, Nvar3, Nvar4. Can somebody help in reading this file? On line 10 it has city and state separated by comma. I had been a user of SAS where I can use different format to read

Re: [R] Problem reading mixed CSV file

2012-03-16 Thread jim holtman
What do you mean by Mixed? If a field has a comma, then it is supposed be to enclosed in quotes. You could preprocess the file looking for cases where there are more fields than there there are supposed to be, and if they are always in the same place, you could enclose them in quotes and then

Re: [R] Problem reading mixed CSV file

2012-03-16 Thread Ashish Agarwal
I want to import this CSV file into R. The CSV file is ,,,1968,21,0 ,,Boston,1968,13,0 ,,Boston,1968,18,0 ,,Chicago,1967,44,0 ,,Providence,1968,17,0 ,,Providence,1969,48,0 ,,Binky,1968,24,0 ,,Chicago,1968,23,0 ,,Dally,1968,7,0 ,,Raleigh, North Carol,1968,25,0 Addy ABC-Dogs

Re: [R] Problem reading mixed CSV file

2012-03-16 Thread David Winsemius
On Mar 16, 2012, at 1:11 PM, Ashish Agarwal wrote: I want to import this CSV file into R. The CSV file is ,,,1968,21,0 ,,Boston,1968,13,0 ,,Boston,1968,18,0 ,,Chicago,1967,44,0 ,,Providence,1968,17,0 ,,Providence,1969,48,0 ,,Binky,1968,24,0 ,,Chicago,1968,23,0 ,,Dally,1968,7,0 ,,Raleigh,

Re: [R] Problem reading mixed CSV file

2012-03-16 Thread Ashish Agarwal
Line 10 has City and State that too separated by comma. For line 10 how can I read differently as compared to the other lines? On Fri, Mar 16, 2012 at 10:59 PM, David Winsemius dwinsem...@comcast.net wrote: On Mar 16, 2012, at 1:11 PM, Ashish Agarwal wrote: I want to import this CSV file into

Re: [R] Problem reading mixed CSV file

2012-03-16 Thread Peter Ehlers
On 2012-03-16 10:48, Ashish Agarwal wrote: Line 10 has City and State that too separated by comma. For line 10 how can I read differently as compared to the other lines? Edit the file and put quotes around the city-state combination: Raleigh, North Carol Also: always run count.fields() on

Re: [R] Problem reading mixed CSV file

2012-03-16 Thread Ashish Agarwal
I have a file that is 5000 records and to edit that file is not easy. Is there any way to line 10 differently to account for changes in the third field? On Fri, Mar 16, 2012 at 11:35 PM, Peter Ehlers ehl...@ucalgary.ca wrote: On 2012-03-16 10:48, Ashish Agarwal wrote: Line 10 has City and

Re: [R] Problem reading mixed CSV file

2012-03-16 Thread jim holtman
Here is a solution that looks for the line with 7 elements and inserts the quotes: fileName - '/temp/text.txt' input - readLines(fileName) # count the fields to find 7 nFields - count.fields(fileName, sep = ',') # now fix the data for (i in which(nFields == 7)){ + # split on comma +