[R] error in importing text files
Hello All: I am very new to R and am trying to import some (107) text files into R while simultaneously manipulating the data into a structure that I can use. Below is a description of what I am trying to ultimately accomplish. However, I find that I am unable to even read one file into R using the read.table function. I get the following error: test-read.table(test4.txt) Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, : line 16 did not have 8 elements So, I would like to resolve the error first, and then ultimately accomplish the goal outlined below. I understand that my ultimate goal may require multiple steps. I am open to that. First, a little background: Data were collected from 107 participants on palm pilots. Participants were randomly beeped 5 times throughout the day for 5 days resulting in a total of 25 measurement occasions. Participants responded to beeps only when they were able to (not in class, driving, etc). Thus, most participants completed far less than 25 measurement occasions. During each measurement occasion participants responded to 32 questions by moving a slider on a 100 point scale. The palms recorded the date, time, palm pilot ID number, response to the beep (non-response = -32767), question number, response latency in milliseconds, response to each question, and three addtiional data points of no interest. The data are arranged in an unstacked (long) text file such that each line contains all of the above information and there are 34 (32 responses plus 2 extra lines of meaningless data) lines per measurement occasion (upto 850 lines of data if all 34 lines are present ! for all 25 measurment occasions). Below is an example of how the data are arranged. 20080204131646 23256063 6 0 20080204131646 233152-1 7 0 20080204150043 2-32767 0 0 65535 20080204182117 2 1283-1 7 0 20080204182117 2 283834 6 0 20080204182117 2 326636 6 0 Year/Month/Day/Time Palm ID Response/Q#Latency Response 3 meangingless columnsThe dataset presented above begins with question 32 of one measurement occasion on Febraury 4, 2008 taken at 13:16:46. The next line (33) is in the datafile because participants had to click a button to exit the measurement occasion. You then see the beginning of another measurement occasion (20080204192117) in which the participant did not respond (-32767). The next measurement occasion begins on the next line which actually starts with response 2 because participants were required to read a screen and click through prior to answering any questions. Thus, anytime participants simply read an instruction page responses are coded as a -1. What I would like to do is write code to automatically import these 107 files into R and structure them appropriately while importing them. Furthermore, I would like for the code to use conditional statements so that whenever it encounters a -32767! it inserts 32 variables (columns) with missing data and whenever it encounters a -1 it deletes that column all together. I would also like the code to separate the combined year/month/day/time column into 4 separate columns (year, month, day, time). Finally, I would like the code to stack the 32 responses during each measurement occasion so that I have 32 columns of reponses plus columns for year, month, day, and latency, but leave each measurment occasion unstacked. Thanks! Eric S McKibben Industrial-Organizational Psychology Graduate Student Clemson University Clemson, SC [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] error in importing text files
Eric, The error suggests that the file has an inconsistent number of attributes. You should inspect the file to make sure that each line has the same number of expected attributes (columns). If one or more is missing, it suggests that the data collection process from the Palm Pilot is not what you expect it to be. Perhaps, participants, just didn't answer all of the questions or something like that. Steve Friedman Ph. D. Spatial Statistical Analyst Everglades and Dry Tortugas National Park 950 N Krome Ave (3rd Floor) Homestead, Florida 33034 steve_fried...@nps.gov Office (305) 224 - 4282 Fax (305) 224 - 4147 Eric McKibben emck...@clemson. edu To Sent by: r-help@r-project.org r-help-boun...@r- cc project.org Subject [R] error in importing text files 05/18/2009 10:42 AM AST Hello All: I am very new to R and am trying to import some (107) text files into R while simultaneously manipulating the data into a structure that I can use. Below is a description of what I am trying to ultimately accomplish. However, I find that I am unable to even read one file into R using the read.table function. I get the following error: test-read.table(test4.txt) Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, : line 16 did not have 8 elements So, I would like to resolve the error first, and then ultimately accomplish the goal outlined below. I understand that my ultimate goal may require multiple steps. I am open to that. First, a little background: Data were collected from 107 participants on palm pilots. Participants were randomly beeped 5 times throughout the day for 5 days resulting in a total of 25 measurement occasions. Participants responded to beeps only when they were able to (not in class, driving, etc). Thus, most participants completed far less than 25 measurement occasions. During each measurement occasion participants responded to 32 questions by moving a slider on a 100 point scale. The palms recorded the date, time, palm pilot ID number, response to the beep (non-response = -32767), question number, response latency in milliseconds, response to each question, and three addtiional data points of no interest. The data are arranged in an unstacked (long) text file such that each line contains all of the above information and there are 34 (32 responses plus 2 extra lines of meaningless data) lines per measurement occasion (upto 850 lines of data if all 34 lines are present ! for all 25 measurment occasions). Below is an example of how the data are arranged. 20080204131646 23256063 6 0 20080204131646 233152-1 7 0 20080204150043 2-32767 0 0 65535 20080204182117 2 1283-1 7 0 20080204182117 2 283834 6 0 20080204182117 2 326636 6 0 Year/Month/Day/Time Palm ID Response/Q#Latency Response 3 meangingless columnsThe dataset presented above begins with question 32 of one measurement occasion on Febraury 4, 2008 taken at 13:16:46. The next line (33) is in the datafile because participants had to click a button to exit the measurement occasion. You then see the beginning of another measurement occasion (20080204192117) in which the participant did not respond (-32767). The next measurement occasion begins on the next line which actually starts with response 2 because participants were required to read a screen and click through prior to answering any questions. Thus, anytime participants simply read an instruction page responses are coded as a -1. What I would like to do is write code to automatically import these 107 files into R and structure them appropriately while importing them. Furthermore, I would like for the code to use conditional statements so that whenever it encounters a -32767! it inserts 32 variables (columns
Re: [R] error in importing text files
Hi Steve, Thanks for the reply. My initial thought was just as you indicated, one of the rows must be missing an element. Interestingly, all of the rows in that text file had 8 columns. I went directly to the line in the text file indicated in the error and there was nothing unusual about that line as far as I could tell. On a related note, is it possible to import one set of characters at a time to create a series of vectors that I could then bind together into a data frame? For example, lets say I want the first 4 characters in a column that contains the following data 20080204131646. Could I assign these characters to a vector called year? Could I then assign characters five and six to another vector called day? If so, how do I do this? Thanks again, Eric S McKibben Industrial-Organizational Psychology Graduate Student Clemson University Clemson, SC - Original Message - From: steve_fried...@nps.gov To: Eric McKibben emck...@clemson.edu Cc: r-help@r-project.org; r-help-boun...@r-project.org Sent: Monday, May 18, 2009 11:01 AM Subject: Re: [R] error in importing text files Eric, The error suggests that the file has an inconsistent number of attributes. You should inspect the file to make sure that each line has the same number of expected attributes (columns). If one or more is missing, it suggests that the data collection process from the Palm Pilot is not what you expect it to be. Perhaps, participants, just didn't answer all of the questions or something like that. Steve Friedman Ph. D. Spatial Statistical Analyst Everglades and Dry Tortugas National Park 950 N Krome Ave (3rd Floor) Homestead, Florida 33034 steve_fried...@nps.gov Office (305) 224 - 4282 Fax (305) 224 - 4147 Eric McKibben emck...@clemson. edu To Sent by: r-help@r-project.org r-help-boun...@r- cc project.org Subject [R] error in importing text files 05/18/2009 10:42 AM AST Hello All: I am very new to R and am trying to import some (107) text files into R while simultaneously manipulating the data into a structure that I can use. Below is a description of what I am trying to ultimately accomplish. However, I find that I am unable to even read one file into R using the read.table function. I get the following error: test-read.table(test4.txt) Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, : line 16 did not have 8 elements So, I would like to resolve the error first, and then ultimately accomplish the goal outlined below. I understand that my ultimate goal may require multiple steps. I am open to that. First, a little background: Data were collected from 107 participants on palm pilots. Participants were randomly beeped 5 times throughout the day for 5 days resulting in a total of 25 measurement occasions. Participants responded to beeps only when they were able to (not in class, driving, etc). Thus, most participants completed far less than 25 measurement occasions. During each measurement occasion participants responded to 32 questions by moving a slider on a 100 point scale. The palms recorded the date, time, palm pilot ID number, response to the beep (non-response = -32767), question number, response latency in milliseconds, response to each question, and three addtiional data points of no interest. The data are arranged in an unstacked (long) text file such that each line contains all of the above information and there are 34 (32 responses plus 2 extra lines of meaningless data) lines per measurement occasion (upto 850 lines of data if all 34 lines are present ! for all 25 measurment occasions). Below is an example of how the data are arranged. 20080204131646 23256063 6 0 20080204131646 233152-1 7 0 20080204150043 2-32767 0 0 65535 20080204182117 2 1283-1 7 0 20080204182117 2 283834 6 0 20080204182117 2 326636 6 0 Year/Month/Day/Time Palm ID Response/Q#Latency Response 3 meangingless columnsThe dataset presented above begins with question 32 of one measurement occasion on Febraury 4, 2008 taken at 13:16:46. The next line (33) is in the datafile because participants had to click a button to exit the measurement occasion. You then see the beginning of another measurement occasion (20080204192117) in which the participant did not respond (-32767). The next measurement occasion begins on the next line which actually starts with response 2
Re: [R] error in importing text files
For the sample of data that you provided, I would read it fine:
x - read.table('clipboard')
x
V1 V2 V3 V4 V5 V6V7 V8
1 20080204131646 2 32 560 63 6 0 NA
2 20080204131646 2 33 152 -1 7 0 NA
3 20080204150043 2 -32767 0 0 65535 NA
4 20080204182117 2 1 283 -1 7 0 NA
5 20080204182117 2 2 838 34 6 0 NA
6 20080204182117 2 3 266 36 6 0 NA
My guess is that you have imbalanced quotes in your data. Try
read.table(..., quote='')
to turn off the interpretation of quotes. Also use 'count.fields' to see
where the variation is. If you post the actual data, that would help.
On Mon, May 18, 2009 at 3:59 PM, Eric McKibben emck...@clemson.edu wrote:
Hi Steve,
Thanks for the reply. My initial thought was just as you indicated, one of
the rows must be missing an element. Interestingly, all of the rows in that
text file had 8 columns. I went directly to the line in the text file
indicated in the error and there was nothing unusual about that line as far
as I could tell.
On a related note, is it possible to import one set of characters at a time
to create a series of vectors that I could then bind together into a data
frame? For example, lets say I want the first 4 characters in a column that
contains the following data 20080204131646. Could I assign these characters
to a vector called year? Could I then assign characters five and six to
another vector called day? If so, how do I do this?
Thanks again,
Eric S McKibben
Industrial-Organizational Psychology Graduate Student
Clemson University
Clemson, SC
- Original Message - From: steve_fried...@nps.gov
To: Eric McKibben emck...@clemson.edu
Cc: r-help@r-project.org; r-help-boun...@r-project.org
Sent: Monday, May 18, 2009 11:01 AM
Subject: Re: [R] error in importing text files
Eric,
The error suggests that the file has an inconsistent number of
attributes. You should inspect the file to make sure that each line has
the same number of expected attributes (columns). If one or more is
missing, it suggests that the data collection process from the Palm Pilot
is not what you expect it to be. Perhaps, participants, just didn't answer
all of the questions or something like that.
Steve Friedman Ph. D.
Spatial Statistical Analyst
Everglades and Dry Tortugas National Park
950 N Krome Ave (3rd Floor)
Homestead, Florida 33034
steve_fried...@nps.gov
Office (305) 224 - 4282
Fax (305) 224 - 4147
Eric McKibben
emck...@clemson.
edu To
Sent by: r-help@r-project.org
r-help-boun...@r- cc
project.org
Subject
[R] error in importing text files
05/18/2009 10:42
AM AST
Hello All:
I am very new to R and am trying to import some (107) text files into R
while simultaneously manipulating the data into a structure that I can
use.
Below is a description of what I am trying to ultimately accomplish.
However, I find that I am unable to even read one file into R using the
read.table function. I get the following error:
test-read.table(test4.txt)
Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings,
:
line 16 did not have 8 elements
So, I would like to resolve the error first, and then ultimately
accomplish
the goal outlined below. I understand that my ultimate goal may require
multiple steps. I am open to that.
First, a little background: Data were collected from 107 participants on
palm pilots. Participants were randomly beeped 5 times throughout the day
for 5 days resulting in a total of 25 measurement occasions. Participants
responded to beeps only when they were able to (not in class, driving,
etc). Thus, most participants completed far less than 25 measurement
occasions. During each measurement occasion participants responded to 32
questions by moving a slider on a 100 point scale. The palms recorded the
date, time, palm pilot ID number, response to the beep (non-response =
-32767), question number, response latency in milliseconds, response to
each question, and three addtiional data points of no interest. The data
are arranged in an unstacked (long) text file such that each line contains
all of the above information and there are 34 (32 responses plus 2 extra
lines of meaningless data) lines per measurement occasion (upto 850 lines
of data if all 34 lines are present !
for all 25 measurment occasions). Below is an example of how the data are
arranged.
20080204131646 23256063 6 0
20080204131646 233152-1 7 0
20080204150043 2-32767 0
