I am not a statistician and not a computer scientist by education. I consider myself an R novice and came to R - thanks to my boss - from an SPSS background. I work for a market research company and the most typical data files we deal with are not huge - up to several thousand rows and up to a thousand variables. I would say, on certain projects, most of what we do in R (if you look at the number of lines in R we devote to a given task) is data manipulation. The actual statistical method is frequently just a line - all the rest is getting the data shaped right and then spitting out the results of the analysis in a way that is usable (i.e., presentable). I find R to be excellent in data manipulations that we perform. First of all, it's great that you can always grab variables/cases you need and ignore all the rest. In SPSS you just keep staring at all those data and variables that you don't need - trying to find the one you need. Second - I find R to be incredibly fast (as opposed to SPSS or Excel) with the amounts of data we are dealing with. And third - nothing is "written in stone" and your original data is always untouched - you can always read it in again and again. For example, if I create a new variable and make a mistake, I can always fix the code, rerun that piece of the code and that gives me the corrected object that containes that new variable. I never touch the original data and hence - never "spoil" it.
Dimitri On Mon, May 11, 2009 at 11:20 AM, ronggui <ronggui.hu...@gmail.com> wrote: > 2009/5/6 Emmanuel Charpentier <charp...@bacbuc.dyndns.org>: >> Le mercredi 06 mai 2009 à 00:22 -0400, Farrel Buchinsky a écrit : >>> Is R an appropriate tool for data manipulation and data reshaping and data >>> organizing? >> [ Large Snip ! ... ] >> >> Depends on what you have to do. >> >> I've done what can be more or less termed "data management" with almost >> uncountable tools (from Excel (sigh...) to R with SQL, APL, Pascal, C, >> Basic (in 1982 !), Fortran and even Lisp in passing...). > >> >> SQL has strong points : join is, to my tastes, more easily expressed in >> SQL than in most languages, projection and aggregation are natural. >> >> However, in SQL, there is no "natural" ordering of row tables, which >> makes expressing algorithms using this order difficult. Try for example >> to express the differences of a time series ... (it can be done, but it >> is *not* a pretty sight). >> >> On the other hand, R has some unique expressive possibilities (reshape() >> comes to mind). >> >> So I tend to use a combination of tools : except for very small samples, >> I tend to manage my data in SQL and with associated tools (think data >> editing, for example ; a simple form in OpenOffice's Base is quite easy >> to create, can handle anything for which an ODBC driver exists, and >> won't crap out for more than a few hundreds line...). finer manipulation >> is usually done in R with native tools and sqldf. > >> But, at least in my trade, the ability to handle Excel files is a must >> (this is considered as a standard for data entry. Sigh ...). So the >> first task is usually a) import data in an SQL database, and b) prepare >> some routines to dump SQL tables / R dataframes in Excel tor returning >> back to the original data author... > > I don't think Excel is a standard tool for data entry. Epidata entry > is much more professional. > >> >> HTH >> >> Emmanuel Charpentier >> >> ______________________________________________ >> R-help@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> > > > > -- > HUANG Ronggui, Wincent > PhD Candidate > Dept of Public and Social Administration > City University of Hong Kong > Home page: http://asrr.r-forge.r-project.org/rghuang.html > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > -- Dimitri Liakhovitski MarketTools, Inc. dimitri.liakhovit...@markettools.com ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.