[R] History pruning
Hi, I find that a typical workflow for me looks something like this: 1) import some data from files 2) mess around with the data for a while 3) mess around with plotting for a while 4) get a plot or analysis that looks good 5) go back through my history to make a list of the shortest command sequence to recreate the plot or analysis 6) send out that sequence to colleagues, along with the generated plots or analysis output I wonder if there are any tools people have developed to help with step 5. Typically I do something like this: 5a) save my entire history to a text file 5b) open it up in Emacs 5c) prune any lines that don't have assignment operators 5d) prune any plotting commands that were superseded by later plots and then start on other more subtle stuff like pruning assignments that were later overwritten, unless the later assignments have variable overlap between the LHS and the RHS. Then I just start eyeballing it. Would any deeper introspection of the history expressions be feasible, e.g. detecting statements that have no side effects, dead ends, etc. The holy grail would be something like "show me all the statements that contributed to the current plot" or the like. Thanks. -- Ken Williams Research Scientist The Thomson Reuters Corporation Eagan, MN __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] history: recording
Hi, I'm using windows xp and R 2.8.0 - I wonder what is the command to put in a script that has the same effect as when in a plot you choose menu "History" and "Recording". Best wishes Troels -- Troels Ring - - Department of nephrology - - Aalborg Hospital 9100 Aalborg, Denmark - - +45 99326629 - - tr...@gvdnet.dk __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
on 07/30/2008 01:12 PM Ken Williams wrote: Hi, I find that a typical workflow for me looks something like this: 1) import some data from files 2) mess around with the data for a while 3) mess around with plotting for a while 4) get a plot or analysis that looks good 5) go back through my history to make a list of the shortest command sequence to recreate the plot or analysis 6) send out that sequence to colleagues, along with the generated plots or analysis output I wonder if there are any tools people have developed to help with step 5. Typically I do something like this: 5a) save my entire history to a text file 5b) open it up in Emacs 5c) prune any lines that don't have assignment operators 5d) prune any plotting commands that were superseded by later plots and then start on other more subtle stuff like pruning assignments that were later overwritten, unless the later assignments have variable overlap between the LHS and the RHS. Then I just start eyeballing it. Would any deeper introspection of the history expressions be feasible, e.g. detecting statements that have no side effects, dead ends, etc. The holy grail would be something like "show me all the statements that contributed to the current plot" or the like. Thanks. I (and many others) use ESS (Emacs Speaks Statistics), in which case, I have an R source buffer in the upper frame and an R session in the lower frame. In my particular case, I also happen to use ECB (Emacs Code Browser) which also has a left hand column spanning the full vertical length, to provide access to other things (file browser, R function and data objects, etc.). It also helps integrate Sweave/LaTeX functionality to further centralize things and increase productivity. I have also tied in Subversion functionality to enable me to engage in version control of my code and other key files. I do all of my editing in the upper frame and use the built-in ESS functions to submit the code to the R session. This also provides for code syntax highlighting, which makes it easier to visualize code as well as to check for things like matching parens/braces, etc. In this way, your working code (including comments) is kept functionally intact in the upper frame and you can edit and use it without having to scroll through a long history of commands (which is still there if you need it). More information here: http://ess.r-project.org/ HTH, Marc Schwartz __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
On 7/30/08 1:59 PM, "Marc Schwartz" <[EMAIL PROTECTED]> wrote: > I (and many others) use ESS (Emacs Speaks Statistics), in which case, I > have an R source buffer in the upper frame and an R session in the lower > frame. I also use ESS to edit my R code (inside Aquamacs Emacs), but I usually use the OS X port R.app for most of my interactive sessions. Together I think those give me roughly the same amount of IDE-like support as you've got in your setup. I think the ess-smart-underscore command alone is worth the price of admission. But none of that directly addresses the issue of automatically (or semi-automatically) taking a long sequence of commands and pruning it down to a smaller sequence that produces the same results. Theoretically the allowable prunings would be akin to those of a good optimizer. And the R language would seem fairly amenable to such things, with its pass-by-value functional semantics, etc. -- Ken Williams Research Scientist The Thomson Reuters Corporation Eagan, MN __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
on 07/31/2008 08:35 AM Ken Williams wrote: On 7/30/08 1:59 PM, "Marc Schwartz" <[EMAIL PROTECTED]> wrote: I (and many others) use ESS (Emacs Speaks Statistics), in which case, I have an R source buffer in the upper frame and an R session in the lower frame. I also use ESS to edit my R code (inside Aquamacs Emacs), but I usually use the OS X port R.app for most of my interactive sessions. Together I think those give me roughly the same amount of IDE-like support as you've got in your setup. I think the ess-smart-underscore command alone is worth the price of admission. But none of that directly addresses the issue of automatically (or semi-automatically) taking a long sequence of commands and pruning it down to a smaller sequence that produces the same results. Theoretically the allowable prunings would be akin to those of a good optimizer. And the R language would seem fairly amenable to such things, with its pass-by-value functional semantics, etc. On that point, I would need to defer to others as to any pre-existing tools. I am not aware of any at this point, but that does not mean that none exist. Sounds like the basis of an interesting research project. Marc P.S. I just noted the "Eagan, MN" in your sig. I live in Eden Prairie... __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
On 7/31/08 11:01 AM, "hadley wickham" <[EMAIL PROTECTED]> wrote: > I think that would be a very hard task - Well, at least medium-hard. But I think significant automatic steps could be made, and then a human can take over for the last few steps. That's why I was enquiring about "tools" rather than a complete solution. Does R provide facilities for introspection or interrogation of expression objects? I couldn't find anything useful on first look: > methods(class="expression") no methods were found > dput(expression(foo <- 5 * bar)) expression(foo <- 5 * bar) > str(expression(foo <- 5 * bar)) expression(foo <- 5 * bar) > it's equivalent to taking a > long rambling conversation and then automatically turning it into a > concise summary of what was said. I think you must have human > intervention. It's not really equivalent, natural language has ambiguities and subtleties that computer languages, especially functional languages, intentionally don't have. By their nature, computer languages can be turned into parse trees unambiguously and then those trees can be manipulated. But coincidentally I work in a Natural Language Processing group, and one of the things we do is create exactly the kind of concise summaries you describe. =) -- Ken Williams Research Scientist The Thomson Reuters Corporation Eagan, MN __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
On 7/31/2008 2:08 PM, Ken Williams wrote: On 7/31/08 11:01 AM, "hadley wickham" <[EMAIL PROTECTED]> wrote: I think that would be a very hard task - Well, at least medium-hard. But I think significant automatic steps could be made, and then a human can take over for the last few steps. That's why I was enquiring about "tools" rather than a complete solution. Does R provide facilities for introspection or interrogation of expression objects? I couldn't find anything useful on first look: You can index an expression as a list: > e <- expression(foo <- 5 * bar) > > e[[1]] foo <- 5 * bar > str(e[[1]]) language foo <- 5 * bar expression() returns a list of language objects, and we only asked for one. We can look inside it: > e[[1]][[1]] `<-` The as.list function is also useful: > as.list(e[[1]]) [[1]] `<-` [[2]] foo [[3]] 5 * bar and proceed recursively: > as.list(e[[1]][[3]]) [[1]] `*` [[2]] [1] 5 [[3]] bar Duncan Murdoch > >> methods(class="expression") > no methods were found >> dput(expression(foo <- 5 * bar)) > expression(foo <- 5 * bar) >> str(expression(foo <- 5 * bar)) > expression(foo <- 5 * bar) > > >> it's equivalent to taking a >> long rambling conversation and then automatically turning it into a >> concise summary of what was said. I think you must have human >> intervention. > > It's not really equivalent, natural language has ambiguities and subtleties > that computer languages, especially functional languages, intentionally > don't have. By their nature, computer languages can be turned into parse > trees unambiguously and then those trees can be manipulated. > > But coincidentally I work in a Natural Language Processing group, and one of > the things we do is create exactly the kind of concise summaries you > describe. =) > __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
On 7/31/08 2:12 PM, "Duncan Murdoch" <[EMAIL PROTECTED]> wrote: > > expression() returns a list of language objects, and we only asked for > one. We can look inside it: Hey, cool. Now let me see if I can do anything useful with that. Thanks. -Ken -- Ken Williams Research Scientist The Thomson Reuters Corporation Eagan, MN __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
> It's not really equivalent, natural language has ambiguities and subtleties > that computer languages, especially functional languages, intentionally > don't have. By their nature, computer languages can be turned into parse > trees unambiguously and then those trees can be manipulated. But in some ways that makes things easier - i.e. you don't expect to be able to summarise a conversation/paper/book in a way that completely recreates it - some ambiguity is unavoidable. > But coincidentally I work in a Natural Language Processing group, and one of > the things we do is create exactly the kind of concise summaries you > describe. =) Well good luck! And I'll be interested to see anything you come up with. Hadley -- http://had.co.nz/ __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
JGR's "Copy Commands" command works well for me (even if it is both fascinating and embarrassing how little is sometimes left over). It retains only commands that worked, so it is still not the minimum possible. Antony Unwin Professor of Computer-Oriented Statistics and Data Analysis, Mathematics Institute, University of Augsburg, 86135 Augsburg, Germany Tel: + 49 821 5982218 [EMAIL PROTECTED] http://stats.math.uni-augsburg.de/ [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
>5a) save my entire history to a text file >5b) open it up in Emacs >5c) prune any lines that don't have assignment operators > > >Ken Williams >Research Scientist >The Thomson Reuters Corporation >Eagan, MN No one has yet mentioned the obvious. ESS does your 5a 5b 5c with M-x ess-transcript-clean-buffer It works in either the *R* buffer or a *.rt or *.st buffer. It handles multiple-line commands correctly. Make sure the buffer is writable (C-x C-q on the *.rt buffer) M-x ess-transcript-clean-buffer Save the buffer as a *.r file. On automatic content analysis, that is tougher. I would be scared to do your >5d) prune any plotting commands that were superseded by later plots because I don't know what supersede means. I can imagine situations, for example, par(mfrow=c(1,2)) plot(y ~ x) x <- x + 1 plot(y ~ x) where I want to keep both plots. You also have to trust that there are no side effects, which I wouldn't want to do, because plot() changes the value of par() parameters. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
On 8/1/08 12:40 PM, "Richard M. Heiberger" <[EMAIL PROTECTED]> wrote: > >> 5a) save my entire history to a text file >> 5b) open it up in Emacs >> 5c) prune any lines that don't have assignment operators > > No one has yet mentioned the obvious. ESS does your 5a 5b 5c with >M-x ess-transcript-clean-buffer I think you mean just 5a & 5b, right? Lines with syntax errors are (I think) removed, but that's it. That part is relatively easy to perform as the first step of a tool, just by running commands through R's parse() and discarding anything that throws an exception. > On automatic content analysis, that is tougher. I would be scared to do your >> 5d) prune any plotting commands that were superseded by later plots True. There are lots of (perhaps relatively common) edge cases that would have to be taken into account. Perhaps a more interactive approach would be better, something like "get rid of this plot command and all subsequent modifications to its canvas". Not sure. My basic philosophy on stuff like this is, given the choice of me fumbling around using tools and me fumbling around without using tools, I tend to do better when I have tools. > You also have to trust that there are no side effects, which I wouldn't > want to do, because plot() changes the value of par() parameters. It does? I wasn't aware of that, could you give an example? -- Ken Williams Research Scientist The Thomson Reuters Corporation Eagan, MN __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
I meant 5a 5b 5c. Multiple-line commands are handled correctly. What is is doing is looking for "> " and " +" prompts. Anything else is removed. Here is a selection from the *R* buffer and the result after cleaning. It includes an example of par(). Rich *R* > options(chmhelp = FALSE) > options(STERM='iESS', editor='gnuclient.exe') > par()$usr [1] 0 1 0 1 > plot(1:10) > par()$usr [1] 0.64 10.36 0.64 10.36 > a <- + 3+4 > After cleaning options(chmhelp = FALSE) options(STERM='iESS', editor='gnuclient.exe') par()$usr plot(1:10) par()$usr a <- 3+4 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
On 8/1/08 1:13 PM, "Richard M. Heiberger" <[EMAIL PROTECTED]> wrote: > I meant 5a 5b 5c. Multiple-line commands are handled correctly. > What is is doing is looking for "> " and " +" prompts. Anything else > is removed. When I said "5c) prune any lines that don't have assignment operators" I meant to take a sequence like this (to pick a semi-random chunk from my history log): --- df <- data.frame(x=2:9, y=(1:8)^2) cor(df) ?cor mad(c(1:9)) ?reshape a <- matrix(1:12, nrow=3) b <- matrix(2:13, nrow=3) b <- matrix(4:15, nrow=3) b <- matrix(2:13, nrow=3) c <- matrix(4:15, nrow=3) a b c --- And turn it into this: --- df <- data.frame(x=2:9, y=(1:8)^2) a <- matrix(1:12, nrow=3) b <- matrix(2:13, nrow=3) b <- matrix(4:15, nrow=3) b <- matrix(2:13, nrow=3) c <- matrix(4:15, nrow=3) --- Obviously I wouldn't *always* want this performed, but selectively it would be quite nice. Further, if the dependency graph among variable definitions were computable, the sequence could be reduced to this: --- df <- data.frame(x=2:9, y=(1:8)^2) a <- matrix(1:12, nrow=3) b <- matrix(2:13, nrow=3) c <- matrix(4:15, nrow=3) --- Note that the starting point of all of this is a sequence of commands (the output of savehistory(), so separating commands from output isn't necessary. I've made a bit of progress on this, hopefully I can get clearance to show my work soon. It would be nice if this could be hooked into ESS for selective pruning or something. -Ken -- Ken Williams Research Scientist The Thomson Reuters Corporation Eagan, MN __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
Ken, Others have given hints on pruning the history, but are you committed to doing this way? An alternative would be something more like sink, where when you get to a place that you know you want to start saving the commands you run a function to start saving your commands, then at the end you run a command to stop saving the commands. One tool for doing this is in the TeachingDemos package, see the help on ?txtStart. The main goal of this set of functions was more to save a transcript of a session (including graphical output if you use the etxtStart interface and an external tool), but it has a possible side effect of saving the commands issued in a file that could be 'source'd to rerun the set of commands (which seems similar to what you want). Commands (actually expressions) that result in an error are not included and you can use the txtSkip function to run a command without saving the command in the file (for things like "?plot" that you don't want to rerun). This may give you what you want, or at least something that needs less editing to get at what you want. Another option would be to take the source code for the above utilities and add some checks that will decide whether to save the command or not (check if an assignment was made, check if any 'par'ameters were changed, etc.). Another option if you just want some code to recreate the current plot is to look at the plot2script function in the TeachingDemos package. It will create a script (put it on the clipboard by default) to recreate the current plot. It does NOT use the same set of commands that you used to create the plot, but rather lowlevel commands, but it creates a script that you can edit to recreate the plot with just your changes (the current version needs some edits (line wrapping, fixing the box command) before running the script, but it may be another place to start). Hope this helps, -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare [EMAIL PROTECTED] (801) 408-8111 > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Ken Williams > Sent: Wednesday, July 30, 2008 12:13 PM > To: [EMAIL PROTECTED] > Subject: [R] History pruning > > Hi, > > I find that a typical workflow for me looks something like this: > > 1) import some data from files > 2) mess around with the data for a while > 3) mess around with plotting for a while > 4) get a plot or analysis that looks good > 5) go back through my history to make a list of the shortest > command sequence to recreate the plot or analysis > 6) send out that sequence to colleagues, along with the > generated plots or analysis output > > I wonder if there are any tools people have developed to help > with step 5. Typically I do something like this: > > 5a) save my entire history to a text file > 5b) open it up in Emacs > 5c) prune any lines that don't have assignment operators > 5d) prune any plotting commands that were superseded by later plots > > and then start on other more subtle stuff like pruning > assignments that were later overwritten, unless the later > assignments have variable overlap between the LHS and the > RHS. Then I just start eyeballing it. > > Would any deeper introspection of the history expressions be > feasible, e.g. detecting statements that have no side > effects, dead ends, etc. > > The holy grail would be something like "show me all the > statements that contributed to the current plot" or the like. > > Thanks. > > -- > Ken Williams > Research Scientist > The Thomson Reuters Corporation > Eagan, MN > > __ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
On 8/8/08 1:04 PM, "Greg Snow" <[EMAIL PROTECTED]> wrote: > Ken, > > Others have given hints on pruning the history, but are you committed to doing > this way? Not necessarily. Only the starting point & ending point really matter; I'd like to be able to start with a transcript of a bunch of aimless commands (e.g. the output of sink() or of ess-transcript-clean-buffer or a history transcript), and end up with a nice focused handful of commands suitable for showing to other people. I'm sure the final goal can't typically be achieved fully automatically, but some kind of support from tools would be great. Thanks for mentioning plot2script() and the TeachingDemos package, those are indeed nice examples to look at. -- Ken Williams Research Scientist The Thomson Reuters Corporation Eagan, MN __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History pruning
We have different starting points. Please be sure that your modularity allows a cleaned region as well as a history log to be the input to your next step. The history log is incomplete; lines sent to the *R* buffer by C-c C-n are explicitly excluded from history. Lines picked up from a saved transcript file aren't in history. Will your program handle this correctly: aa <- bb + cc ? It is valid code. Suppressing the line with cc will make it not valid code. Rich __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] history: recording
?windows On Tue, Jan 6, 2009 at 10:44 AM, Troels Ring wrote: > Hi, I'm using windows xp and R 2.8.0 - I wonder what is the command to put > in a script that has the same effect as when in a plot you choose menu > "History" and "Recording". > Best wishes > Troels > > -- > > Troels Ring - - > Department of nephrology - - Aalborg Hospital 9100 Aalborg, Denmark - - > +45 99326629 - - > tr...@gvdnet.dk > > __ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > -- Jim Holtman Cincinnati, OH +1 513 646 9390 What is the problem that you are trying to solve? __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] history: recording
Thanks a lot - windows(record=TRUE) seems to do the job. Sorry not to have found it. Troels jim holtman skrev: ?windows On Tue, Jan 6, 2009 at 10:44 AM, Troels Ring wrote: Hi, I'm using windows xp and R 2.8.0 - I wonder what is the command to put in a script that has the same effect as when in a plot you choose menu "History" and "Recording". Best wishes Troels -- Troels Ring - - Department of nephrology - - Aalborg Hospital 9100 Aalborg, Denmark - - +45 99326629 - - tr...@gvdnet.dk __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Troels Ring - - Department of nephrology - - Aalborg Hospital 9100 Aalborg, Denmark - - +45 99326629 - - tr...@gvdnet.dk __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] history: recording
On Tue, 6 Jan 2009, Troels Ring wrote: Thanks a lot - windows(record=TRUE) seems to do the job. If you want to control an already open graphics device you will need the R-devel version, which has a function msgWindow() to change a lot of settings, including 'record'. Sorry not to have found it. Troels jim holtman skrev: ?windows On Tue, Jan 6, 2009 at 10:44 AM, Troels Ring wrote: Hi, I'm using windows xp and R 2.8.0 - I wonder what is the command to put in a script that has the same effect as when in a plot you choose menu "History" and "Recording". Best wishes Troels -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] History of R
Earlier today I sent a question to Frank Harrell as an R developer with whom I am most familiar. He suggested also that I put my questions to the list for additional responses. Next month I'll be giving a talk on R as an example of high quality open source software. I think there is much to learn from R as a high quality extensible product that (at least as far as I can tell) has never been "spun" or "hyped" like so many open source fads. The question that intrigues me the most is why is R as an open source project is so incredibly successful and other projects, say for example, Octave don't enjoy that level of success? I have some ideas of course, but I would really like to know your thoughts when you look at R from such a vantage point. Thanks. Kathy Gerber University of Virginia ITC - Research Computing Support __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History of R
On Fri, Feb 15, 2008 at 1:53 PM, Kathy Gerber <[EMAIL PROTECTED]> wrote: > Earlier today I sent a question to Frank Harrell as an R developer with > whom I am most familiar. He suggested also that I put my questions to > the list for additional responses. Next month I'll be giving a talk on > R as an example of high quality open source software. I think there is > much to learn from R as a high quality extensible product that (at least > as far as I can tell) has never been "spun" or "hyped" like so many open > source fads. > The question that intrigues me the most is why is R as an open source > project is so incredibly successful and other projects, say for example, > Octave don't enjoy that level of success? First and foremost there is the incredible generosity of Ross Ihaka and Robert Gentleman who, after spending an enormous amount of time and effort in development of the initial implementation, did not demand exclusive ownership of their work but allowed others to make changes. I believe Martin Maechler was the first non-Auckland person to get write access to the source code repository and I'm sure that the good experience of working at a distance with Martin persuaded R & R to open it up to others. Martin is polite, considerate, meticulous and precise (he is a German-speaking Swiss so meticulous and precise kind of comes with the territory) and you couldn't ask for a first experience in sharing something that is very valuable to you with someone whom you may never have met in person. Not everyone has been that pleasant to work with. One of the first things that I did when I joined R-core was to blow up at Kurt and Fritz about something - on Christmas Eve! I surprised the group didn't boot me out after that start. When a project is gaining momentum the personalities of the initial developers have a big influence on its success. The R project has been fortunate in that regard. > I have some ideas of course, but I would really like to know your > thoughts when you look at R from such a vantage point. > Thanks. > Kathy Gerber > University of Virginia > ITC - Research Computing Support > > __ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History of R
On Fri, Feb 15, 2008 at 3:23 PM, Douglas Bates <[EMAIL PROTECTED]> wrote: > On Fri, Feb 15, 2008 at 1:53 PM, Kathy Gerber <[EMAIL PROTECTED]> wrote: > > Earlier today I sent a question to Frank Harrell as an R developer with > > whom I am most familiar. He suggested also that I put my questions to > > the list for additional responses. Next month I'll be giving a talk on > > R as an example of high quality open source software. I think there is > > much to learn from R as a high quality extensible product that (at least > > as far as I can tell) has never been "spun" or "hyped" like so many open > > source fads. > > > The question that intrigues me the most is why is R as an open source > > project is so incredibly successful and other projects, say for example, > > Octave don't enjoy that level of success? > > First and foremost there is the incredible generosity of Ross Ihaka > and Robert Gentleman who, after spending an enormous amount of time > and effort in development of the initial implementation, did not > demand exclusive ownership of their work but allowed others to make > changes. I believe Martin Maechler was the first non-Auckland person > to get write access to the source code repository and I'm sure that > the good experience of working at a distance with Martin persuaded R & > R to open it up to others. Martin is polite, considerate, meticulous > and precise (he is a German-speaking Swiss so meticulous and precise > kind of comes with the territory) and you couldn't ask for a first I meant to write "for a better first experience" > experience in sharing something that is very valuable to you with > someone whom you may never have met in person. > > Not everyone has been that pleasant to work with. One of the first > things that I did when I joined R-core was to blow up at Kurt and > Fritz about something - on Christmas Eve! I surprised the group > didn't boot me out after that start. > > When a project is gaining momentum the personalities of the initial > developers have a big influence on its success. The R project has > been fortunate in that regard. > > > > > I have some ideas of course, but I would really like to know your > > thoughts when you look at R from such a vantage point. > > > Thanks. > > Kathy Gerber > > University of Virginia > > ITC - Research Computing Support > > > > __ > > R-help@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > > > __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History of R
Kathy.
A suggestion. As you gather your information about the history, I suggest you
put fingers to keyboard and write down the history. You could start with the
material Douglas just sent to you. Perhaps we can convince the R folks to place
the history on the CRAN website - perhaps in WIKI format so our founding
fathers, and mothers, can add to the history. I certainly would be nice to be
able to give credit to the may people who have selflessly contributed their
time, effort, and expertise to the R project!.
John
John Sorkin M.D., Ph.D.
Chief, Biostatistics and Informatics
University of Maryland School of Medicine Division of Gerontology
Baltimore VA Medical Center
10 North Greene Street
GRECC (BT/18/GR)
Baltimore, MD 21201-1524
(Phone) 410-605-7119
(Fax) 410-605-7913 (Please call phone number above prior to faxing)
>>> "Douglas Bates" <[EMAIL PROTECTED]> 2/15/2008 4:23 PM >>>
On Fri, Feb 15, 2008 at 1:53 PM, Kathy Gerber <[EMAIL PROTECTED]> wrote:
> Earlier today I sent a question to Frank Harrell as an R developer with
> whom I am most familiar. He suggested also that I put my questions to
> the list for additional responses. Next month I'll be giving a talk on
> R as an example of high quality open source software. I think there is
> much to learn from R as a high quality extensible product that (at least
> as far as I can tell) has never been "spun" or "hyped" like so many open
> source fads.
> The question that intrigues me the most is why is R as an open source
> project is so incredibly successful and other projects, say for example,
> Octave don't enjoy that level of success?
First and foremost there is the incredible generosity of Ross Ihaka
and Robert Gentleman who, after spending an enormous amount of time
and effort in development of the initial implementation, did not
demand exclusive ownership of their work but allowed others to make
changes. I believe Martin Maechler was the first non-Auckland person
to get write access to the source code repository and I'm sure that
the good experience of working at a distance with Martin persuaded R &
R to open it up to others. Martin is polite, considerate, meticulous
and precise (he is a German-speaking Swiss so meticulous and precise
kind of comes with the territory) and you couldn't ask for a first
experience in sharing something that is very valuable to you with
someone whom you may never have met in person.
Not everyone has been that pleasant to work with. One of the first
things that I did when I joined R-core was to blow up at Kurt and
Fritz about something - on Christmas Eve! I surprised the group
didn't boot me out after that start.
When a project is gaining momentum the personalities of the initial
developers have a big influence on its success. The R project has
been fortunate in that regard.
> I have some ideas of course, but I would really like to know your
> thoughts when you look at R from such a vantage point.
> Thanks.
> Kathy Gerber
> University of Virginia
> ITC - Research Computing Support
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Confidentiality Statement:
This email message, including any attachments, is for th...{{dropped:6}}
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] History of R
For those of you who want to learn more about the history of the R
project: There will be an invited lecture by John Fox and Kurt Hornik at
this year's useR! conference in Dortmund in August (...unfortunately a bit
too late for Kathy) about "The Past, Present, and Future of the R Project"
see
http://www.R-project.org/useR-2008/
The talk will be a double feature with John focusing on "Social
Organization of the R Project" and Kurt on "Development in the R Project".
The core ingredients of the two parts will be
- interviews that John has conducted with the R-core members (and a few
other R developers),
- development of CRAN, DSC, R News, R Foundation, useR!, R-Forge, ...
BTW: Registration for useR! is possible online at the URL mentioned above.
Sorry for the shameless plug ;-)
Z
On Fri, 15 Feb 2008, John Sorkin wrote:
> Kathy.
> A suggestion. As you gather your information about the history, I
> suggest you put fingers to keyboard and write down the history. You
> could start with the material Douglas just sent to you. Perhaps we can
> convince the R folks to place the history on the CRAN website - perhaps
> in WIKI format so our founding fathers, and mothers, can add to the
> history. I certainly would be nice to be able to give credit to the may
> people who have selflessly contributed their time, effort, and expertise
> to the R project!.
> John
>
> John Sorkin M.D., Ph.D.
> Chief, Biostatistics and Informatics
> University of Maryland School of Medicine Division of Gerontology
> Baltimore VA Medical Center
> 10 North Greene Street
> GRECC (BT/18/GR)
> Baltimore, MD 21201-1524
> (Phone) 410-605-7119
> (Fax) 410-605-7913 (Please call phone number above prior to faxing)
>
> >>> "Douglas Bates" <[EMAIL PROTECTED]> 2/15/2008 4:23 PM >>>
> On Fri, Feb 15, 2008 at 1:53 PM, Kathy Gerber <[EMAIL PROTECTED]> wrote:
> > Earlier today I sent a question to Frank Harrell as an R developer with
> > whom I am most familiar. He suggested also that I put my questions to
> > the list for additional responses. Next month I'll be giving a talk on
> > R as an example of high quality open source software. I think there is
> > much to learn from R as a high quality extensible product that (at least
> > as far as I can tell) has never been "spun" or "hyped" like so many open
> > source fads.
>
> > The question that intrigues me the most is why is R as an open source
> > project is so incredibly successful and other projects, say for example,
> > Octave don't enjoy that level of success?
>
> First and foremost there is the incredible generosity of Ross Ihaka
> and Robert Gentleman who, after spending an enormous amount of time
> and effort in development of the initial implementation, did not
> demand exclusive ownership of their work but allowed others to make
> changes. I believe Martin Maechler was the first non-Auckland person
> to get write access to the source code repository and I'm sure that
> the good experience of working at a distance with Martin persuaded R &
> R to open it up to others. Martin is polite, considerate, meticulous
> and precise (he is a German-speaking Swiss so meticulous and precise
> kind of comes with the territory) and you couldn't ask for a first
> experience in sharing something that is very valuable to you with
> someone whom you may never have met in person.
>
> Not everyone has been that pleasant to work with. One of the first
> things that I did when I joined R-core was to blow up at Kurt and
> Fritz about something - on Christmas Eve! I surprised the group
> didn't boot me out after that start.
>
> When a project is gaining momentum the personalities of the initial
> developers have a big influence on its success. The R project has
> been fortunate in that regard.
>
> > I have some ideas of course, but I would really like to know your
> > thoughts when you look at R from such a vantage point.
>
> > Thanks.
> > Kathy Gerber
> > University of Virginia
> > ITC - Research Computing Support
> >
> > __
> > R-help@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> > http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
> Confidentiality Statement:
> This email message, including any attachments, is for th...{{dropped:6}}
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
_
Re: [R] History of R
Dear Kathy, As Achim has mentioned, I've been doing interviews with members of the R Core team and with some other people central to the R Project. Although I haven't entirely organized and finished reflecting on this material, the following factors come immediately to mind: (1) Doug has already mentioned the personal and technical talents of the original developers, and their generosity in opening up development to a Core group and in making R open source. To that I would add the collective talents of the Core group as a whole. (2) R implements the S language, which already was in wide use, and which has many attractive features (each of use, etc.). (3) The R package system and the establishment of CRAN allowed literally hundreds of developers to contribute to the broader R Project. More generally, the Core group worked to integrate users into the R Project, e.g., through R News, the r-help list (though naive users aren't always treated gently there), and the useR conferences. Regards, John John Fox, Professor Department of Sociology McMaster University Hamilton, Ontario, Canada L8S 4M4 905-525-9140x23604 http://socserv.mcmaster.ca/jfox > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > project.org] On Behalf Of Kathy Gerber > Sent: February-15-08 2:53 PM > To: r-help@r-project.org > Subject: [R] History of R > > Earlier today I sent a question to Frank Harrell as an R developer with > whom I am most familiar. He suggested also that I put my questions to > the list for additional responses. Next month I'll be giving a talk on > R as an example of high quality open source software. I think there is > much to learn from R as a high quality extensible product that (at > least > as far as I can tell) has never been "spun" or "hyped" like so many > open > source fads. > > The question that intrigues me the most is why is R as an open source > project is so incredibly successful and other projects, say for > example, > Octave don't enjoy that level of success? > > I have some ideas of course, but I would really like to know your > thoughts when you look at R from such a vantage point. > > Thanks. > Kathy Gerber > University of Virginia > ITC - Research Computing Support > > __ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting- > guide.html > and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History of R
Thanks to all who responded so thoughtfully. I would like to summarize briefly the observations and opinions so far with some of my own interpretations and thoughts. John Fox is working on a much deeper history scheduled for August, and his three factors are a good starting point. John Fox wrote: > Dear Kathy, > > As Achim has mentioned, I've been doing interviews with members of the R > Core team and with some other people central to the R Project. Although I > haven't entirely organized and finished reflecting on this material, the > following factors come immediately to mind: > > (1) Doug has already mentioned the personal and technical talents of the > original developers, and their generosity in opening up development to a > Core group and in making R open source. To that I would add the collective > talents of the Core group as a whole. > There are three attributes here: a) Personal talent: I take this to mean communication and teaching ability along with leadership. These are the talents and skills that provide groundwork for a mature type of collaboration, more along the lines found in tightly focused academic areas. I would think that these attributes are big factors in why R has not devolved into forks and holy wars. b) Technical talent: Both the technical talent and domain knowledge of the original developers and the R Core group are better than consistently solid. The leaders are not rock stars or cult figures. c) Generosity: The responses themselves sincerely gave credit to others. While this may appear to be consistent with Eric Raymond's notions of open source as built upon a "gift culture," I haven't really seen this going on elsewhere at such a level. > (2) R implements the S language, which already was in wide use, and which > has many attractive features (each of use, etc.). > > One person who emailed privately pointed out that many open source projects are "knock-offs," e.g., linux itself is a unix knock-off. I believe the point is that R is not a totally new approach or invention, rather it is based upon advancing some product or collection of ideas that are already in place. > (3) The R package system and the establishment of CRAN allowed literally > hundreds of developers to contribute to the broader R Project. More > generally, the Core group worked to integrate users into the R Project, > e.g., through R News, the r-help list (though naive users aren't always > treated gently there), and the useR conferences. > > Again, this is another distinctive feature, perhaps not in concept but in degree and level of actual success thanks to good planning. Like so many other points, this goes back to the leadership. Another point made was the need or demand for such an application. Yet another was the planning that goes into avoiding breakage of packages. What no one mentioned though was the idea of standards. Finally, in comparing with Octave, it was mentioned that Octave may be stuck in a position of playing catch-up to Matlab. What I have here is far from complete, but I did want to give some feedback tonight. Again, thanks to you all for such articulate responses, and I will point to my slides, and later on write up a summary. Kathy Gerber __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History of R
Hi, Kathy, John, et al.: Has there been an answer to the question of why R has been much more successful than Octave? In this regard, can anyone provide a price comparison of student versions for S-Plus and Matlab during R's gestation period, 10-15 years ago? I had the impression, perhaps incorrect, that several college profs (including Ross and Robert) felt their student's could not afford S-Plus, and that was a large part of the motivation, not just for R & R, but for many other early contributors to R. Insightful has been incredibly generous to the R community recently, and I hope that continues. However, I wonder if R would have emerged when it did and been as successful if academic prices for S-Plus prior to, say, 1992 or 1997 had been substantially lower, especially outside the US. Of course, it may have been easier for the Matlab to offer deep academic discounts than S-Plus, because Matlab may have a larger industrial base market. Beyond that, the "contributed packages" system has helped immensely in R's growth; if Octave has such a system, it's not as visible as CRAN. I recall hearing from Doug Bates (last August at useR! 2007 in Ames, IA) that a major turning point in the development of R came when Martin Maechler convinced Ross & Robert to accept contributed packages. However, this is my memory, and it would be wise if feasible to clarify this with Ross, Robert, Martin, Doug, and others. That's just my US$0.02 (which is only worth roughly half what it was on the international market at this time in 2001). Hope this helps, Spencer Kathy Gerber wrote: > Thanks to all who responded so thoughtfully. I would like to summarize > briefly the observations and opinions so far with some of my own > interpretations and thoughts. John Fox is working on a much deeper > history scheduled for August, and his three factors are a good starting > point. > > John Fox wrote: > >> Dear Kathy, >> >> As Achim has mentioned, I've been doing interviews with members of the R >> Core team and with some other people central to the R Project. Although I >> haven't entirely organized and finished reflecting on this material, the >> following factors come immediately to mind: >> >> (1) Doug has already mentioned the personal and technical talents of the >> original developers, and their generosity in opening up development to a >> Core group and in making R open source. To that I would add the collective >> talents of the Core group as a whole. >> >> > There are three attributes here: > a) Personal talent: I take this to mean communication and teaching > ability along with leadership. These are the talents and skills that > provide groundwork for a mature type of collaboration, more along the > lines found in tightly focused academic areas. I would think that these > attributes are big factors in why R has not devolved into forks and > holy wars. > b) Technical talent: Both the technical talent and domain knowledge of > the original developers and the R Core group are better than > consistently solid. The leaders are not rock stars or cult figures. > c) Generosity: The responses themselves sincerely gave credit to > others. While this may appear to be consistent with Eric Raymond's > notions of open source as built upon a "gift culture," I haven't really > seen this going on elsewhere at such a level. > >> (2) R implements the S language, which already was in wide use, and which >> has many attractive features (each of use, etc.). >> >> >> > One person who emailed privately pointed out that many open source > projects are "knock-offs," e.g., linux itself is a unix knock-off. I > believe the point is that R is not a totally new approach or invention, > rather it is based upon advancing some product or collection of ideas > that are already in place. > >> (3) The R package system and the establishment of CRAN allowed literally >> hundreds of developers to contribute to the broader R Project. More >> generally, the Core group worked to integrate users into the R Project, >> e.g., through R News, the r-help list (though naive users aren't always >> treated gently there), and the useR conferences. >> >> >> > Again, this is another distinctive feature, perhaps not in concept but > in degree and level of actual success thanks to good planning. Like so > many other points, this goes back to the leadership. > > Another point made was the need or demand for such an application. Yet > another was the planning that goes into avoiding breakage of packages. > What no one mentioned though was the idea of standards. > > Finally, in comparing with Octave, it was mentioned that Octave may be > stuck in a position of playing catch-up to Matlab. > > What I have here is far from complete, but I did want to give some > feedback tonight. Again, thanks to you all for such articulate > responses, and
Re: [R] History of R
Spencer, I believe this is the first mention of pricing that I've seen. The accommodation and consideration of contributed packages has been addressed to some degree. Several additional points have been made about the comparison of R to Octave, some off list. -- Matlab did not alienate developers all that much, so people were not driven as much to Octave. With S-PLUS package developers often had their packages break with the new release. So this frustration was a big driver. R met this kind of demand; Octave not so much. As an aside, I would note that rather than update their libraries and scripts or develop a real Maple alternative, many mathematicians stuck with Maple V for several years beyond its "life cycle." Though there were some efforts and I don't want to dismiss their current though somewhat limited success, I don't believe there was a critical mass of folks able or interested in developing an alternative from the ground up in the past. But also those libraries had a tiny user base, and just maybe in mathematics a mindset of "Q.E.D. - I've moved on" enters into play. -- Somewhat related, one observer noted that statisticians have a need for "usable, reliable, accurate, comprehensive software." Along with this R is something of a category killer along the lines of LaTeX. -- Matlab users often switch to R rather than Octave. -- Finally, there is overlap from early on and a high level of collaboration between Insightful and the R community. Kathy Gerber Spencer Graves wrote: > Hi, Kathy, John, et al.: > Has there been an answer to the question of why R has been much > more successful than Octave? > In this regard, can anyone provide a price comparison of student > versions for S-Plus and Matlab during R's gestation period, 10-15 > years ago? I had the impression, perhaps incorrect, that several > college profs (including Ross and Robert) felt their student's could > not afford S-Plus, and that was a large part of the motivation, not > just for R & R, but for many other early contributors to R. > Insightful has been incredibly generous to the R community recently, > and I hope that continues. However, I wonder if R would have emerged > when it did and been as successful if academic prices for S-Plus prior > to, say, 1992 or 1997 had been substantially lower, especially outside > the US. Of course, it may have been easier for the Matlab to offer > deep academic discounts than S-Plus, because Matlab may have a larger > industrial base market. > Beyond that, the "contributed packages" system has helped > immensely in R's growth; if Octave has such a system, it's not as > visible as CRAN. I recall hearing from Doug Bates (last August at > useR! 2007 in Ames, IA) that a major turning point in the development > of R came when Martin Maechler convinced Ross & Robert to accept > contributed packages. However, this is my memory, and it would be > wise if feasible to clarify this with Ross, Robert, Martin, Doug, and > others. > > That's just my US$0.02 (which is only worth roughly half what it > was on the international market at this time in 2001). > Hope this helps, > Spencer > Kathy Gerber wrote: >> Thanks to all who responded so thoughtfully. I would like to >> summarize briefly the observations and opinions so far with some of >> my own interpretations and thoughts. John Fox is working on a much >> deeper history scheduled for August, and his three factors are a good >> starting point. >> >> John Fox wrote: >> >>> Dear Kathy, >>> >>> As Achim has mentioned, I've been doing interviews with members of >>> the R >>> Core team and with some other people central to the R Project. >>> Although I >>> haven't entirely organized and finished reflecting on this material, >>> the >>> following factors come immediately to mind: >>> >>> (1) Doug has already mentioned the personal and technical talents of >>> the >>> original developers, and their generosity in opening up development >>> to a >>> Core group and in making R open source. To that I would add the >>> collective >>> talents of the Core group as a whole. >>> >> There are three attributes here: >> a) Personal talent: I take this to mean communication and teaching >> ability along with leadership. These are the talents and skills that >> provide groundwork for a mature type of collaboration, more along the >> lines found in tightly focused academic areas. I would think that >> these attributes are big factors in why R has not devolved into >> forks and holy wars. >> b) Technical talent: Both the technical talent and domain knowledge >> of the original developers and the R Core group are better than >> consistently solid. The leaders are not rock stars or cult figures. >> c) Generosity: The responses themselves sincerely gave credit to >> others. While this may appear to be consistent with Eric Raymond's >> notions of open so
Re: [R] History of R
The windows port of R has been very good for a long time. I know some people who even think that the current windows port is better than the Linux version. Thanks to those who have made the windows port available and who continue to maintain it. I now use both MS Windows and Linux (Fedora) and would not like to lose either.. The windows port of Octave before the recent version 3 was not good. As far as I know one was restricted to a very old version or using cygwin. This would not suit most users of windows. Thus Octave was not available to the majority of MS windows users. Compared to R which had the latest version available to Windows users is it any wonder that Octave is not as popular. The new version 3 is a vast improvement and should be looked at by anyone familiar with Matlab. The new front end, using the SciTE editor is a vast improvement on what was previously available. Best Regards John On 16/02/2008, Kathy Gerber <[EMAIL PROTECTED]> wrote: > Thanks to all who responded so thoughtfully. I would like to summarize > briefly the observations and opinions so far with some of my own > interpretations and thoughts. John Fox is working on a much deeper > history scheduled for August, and his three factors are a good starting > point. > > John Fox wrote: > > Dear Kathy, > > > > As Achim has mentioned, I've been doing interviews with members of the R > > Core team and with some other people central to the R Project. Although I > > haven't entirely organized and finished reflecting on this material, the > > following factors come immediately to mind: > > > > (1) Doug has already mentioned the personal and technical talents of the > > original developers, and their generosity in opening up development to a > > Core group and in making R open source. To that I would add the collective > > talents of the Core group as a whole. > > > There are three attributes here: > a) Personal talent: I take this to mean communication and teaching > ability along with leadership. These are the talents and skills that > provide groundwork for a mature type of collaboration, more along the > lines found in tightly focused academic areas. I would think that these > attributes are big factors in why R has not devolved into forks and > holy wars. > b) Technical talent: Both the technical talent and domain knowledge of > the original developers and the R Core group are better than > consistently solid. The leaders are not rock stars or cult figures. > c) Generosity: The responses themselves sincerely gave credit to > others. While this may appear to be consistent with Eric Raymond's > notions of open source as built upon a "gift culture," I haven't really > seen this going on elsewhere at such a level. > > (2) R implements the S language, which already was in wide use, and which > > has many attractive features (each of use, etc.). > > > > > One person who emailed privately pointed out that many open source > projects are "knock-offs," e.g., linux itself is a unix knock-off. I > believe the point is that R is not a totally new approach or invention, > rather it is based upon advancing some product or collection of ideas > that are already in place. > > (3) The R package system and the establishment of CRAN allowed literally > > hundreds of developers to contribute to the broader R Project. More > > generally, the Core group worked to integrate users into the R Project, > > e.g., through R News, the r-help list (though naive users aren't always > > treated gently there), and the useR conferences. > > > > > Again, this is another distinctive feature, perhaps not in concept but > in degree and level of actual success thanks to good planning. Like so > many other points, this goes back to the leadership. > > Another point made was the need or demand for such an application. Yet > another was the planning that goes into avoiding breakage of packages. > What no one mentioned though was the idea of standards. > > Finally, in comparing with Octave, it was mentioned that Octave may be > stuck in a position of playing catch-up to Matlab. > > What I have here is far from complete, but I did want to give some > feedback tonight. Again, thanks to you all for such articulate > responses, and I will point to my slides, and later on write up a summary. > > Kathy Gerber > > __ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > -- John C Frain Trinity College Dublin Dublin 2 Ireland www.tcd.ie/Economics/staff/frainj/home.html mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented,
Re: [R] History of R
Hi Kathy, maybe this article could be also of use for you? Ihaka, R., and Gentleman, R. (1996)," R: A Language for Data Analysis and Graphics," The Journal of Computational and Graphical Statistics, 5, 299-314 Best, Roland Kathy Gerber wrote: > Earlier today I sent a question to Frank Harrell as an R developer with > whom I am most familiar. He suggested also that I put my questions to > the list for additional responses. Next month I'll be giving a talk on > R as an example of high quality open source software. I think there is > much to learn from R as a high quality extensible product that (at least > as far as I can tell) has never been "spun" or "hyped" like so many open > source fads. > > The question that intrigues me the most is why is R as an open source > project is so incredibly successful and other projects, say for example, > Octave don't enjoy that level of success? > > I have some ideas of course, but I would really like to know your > thoughts when you look at R from such a vantage point. > > Thanks. > Kathy Gerber > University of Virginia > ITC - Research Computing Support > > __ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History of R
"Kathy Gerber" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > Spencer, > > I believe this is the first mention of pricing that I've seen. > Several additional points have been made about the comparison of R to > Octave, some off list. > -- Matlab did not alienate developers all that much, so people were not > driven as much to Octave. > -- Matlab users often switch to R rather than Octave. One big reason we use R now was what we considered to be unreasonable MatLab licensing terms from the Mathworks. We have mostly biologists here who usually have an intermittent need for analysis tools using MatLab -- they just don't need a dedicated MatLab license per person for analysis, yet that is what the Mathworks expected us to buy. As a new and growing research institute, for several years it didn't make financial sense to have a single shared network MatLab license that cost the same as four named-user licenses when most of our users were biologists. In our opinion, the Mathworks wanted us to pay too much for too little use of their product due to their license limitations. Even though we are a 501(c)3 non-profit research institute, the Mathworks refused to give us academic pricing (and still does). About four years ago MatLab refused to allow one of our postdocs and me to share a single license for casual use. I asked "what is your pricing model"? I asked why the Mathworks cared if a post doc used MatLab for two hours a month and I used MatLab for two hours a month using the same license. So, frustrated by the licensing inflexibility of the Mathworks at that time (four years ago), I abandoned MatLab and re-wrote the MatLab project I was working on in R, and do most analysis now in R. I avoid using MatLab as much as possible and explain licensing terms to new students and post-docs as we start new projects when MatLab is proposed as a solution. Over the years, the Mathworks has been inconsistent on whether a single license on a PC can be shared. About a year ago, we were told it was OK to share such a license on a walk-up workstation, but who wants to walk to another floor or building to use MatLab? The networking option is quite expensive, especially to support a number of MatLab toolboxes. Late last year we finally did have enough MatLab use to warrant the purchase of a network license, but only for a small number of toolboxes. MatLab is rarely my tool of choice when R is such a good alternative. Non-academic pricing from the Mathworks for a non-profit research environment, combined with the large number of R and Bioconductor packages that solved problems of interest (mostly microarray analysis) resulted in much more use of R here than MatLab. Nearly six years ago, SAS also refused to give us academic pricing because we were not a degree granting institution. About a year ago, SAS finally granted us academic pricing, but most of the analysis momentum was already for the use of R/Bioconductor. We casually looked at Octave a few times, but there was no strong attraction to use it. Some early tests showed no problems with computations using Octave, but showed some annoying issues with graphics that we didn't want to deal with. efg Earl F. Glynn Scientific Programmer Stowers Institute for Medical Research __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History of R
Kathy Gerber wrote: > > Earlier today I sent a question to Frank Harrell as an R developer with > whom I am most familiar. He suggested also that I put my questions to > the list for additional responses. Next month I'll be giving a talk on > R as an example of high quality open source software. I think there is > much to learn from R as a high quality extensible product that (at least > as far as I can tell) has never been "spun" or "hyped" like so many open > source fads. > > The question that intrigues me the most is why is R as an open source > project is so incredibly successful and other projects, say for example, > Octave don't enjoy that level of success? > > I have some ideas of course, but I would really like to know your > thoughts when you look at R from such a vantage point. > > Thanks. > Kathy Gerber > University of Virginia > ITC - Research Computing Support > > __ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > > Kathy, If you don't mind, I'll also throw in my perspective as a 60+ year-old new-comer to R. For me there are at least three hugely appealing aspects to R. First, since it is totally free, R is accessible to those poor in material wealth but rich in intellectual curiosity wherever they live. I personally think that is extremely important. Second, contributors to R are selflessly and continuously doing quite a lot to improve approaches to the analysis of data (and there is such a rich history of that growing daily). I have to say that I am in awe of what I see already developed in R and know from the frequency of updates that the entire enterprise is alive, well and growing. Third, R is just flat out wonderful - I know it rekindles my energy making me feel like a "kid in a candy store" again who wants to see what's new, learn more, and contribute. I truthfully can't think of another element in my professional life that makes me feel so strongly this way. I've never seen anything before like R and I'm just grateful to have lived long enough to experience it. I know that I owe a debt of gratitude to R-developers from top to bottom - and I'm certain I'm not alone in this. In summary, let me just say WOW! You can bet that I've incorporated R into all the graduate classes I teach. It so challenges and opens the imagination. Andy Bush ps This is not a solicited remark. It is simply what I personally think and feel. -- View this message in context: http://www.nabble.com/History-of-R-tp15508906p15578677.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History of R
Earl F. Glynn wrote: > Nearly six years ago, SAS also refused to give us academic pricing because > we were not a degree granting institution. About a year ago, SAS finally > granted us academic pricing, but most of the analysis momentum was already > for the use of R/Bioconductor. I recently read the small print on the academic license our site has for SAS. You have to: 1 inform SAS of any taught courses that use SAS 2 inform SAS of any research projects using SAS 3 allow SAS to refer to your institution as a SAS user 4 allow SAS to review your taught courses 5 ensure your courses are taught using qualified personnel. 6 give SAS your first-born male offspring I spoke to our site's licensing supremos and they say they've never heard of anyone complying with 1 or 2. Point 4 sounds like petty fiddling in our educational business, and point 5 left 'qualified' undefined. Point 6 doesn't bother me since I don't have kids. Luckily other parts of our institution have made deals with SAS to use it for consulting and training, so perhaps some of these points don't apply to my department. I've not seen the small print on that contract though, but I expect it to be written in blood on freshly slaughtered deerskin... Barry __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History of R
Many of the doctors that I work with also have teaching appointments at the local university and use the bookstore to buy academic versions of SAS, SPSS, Minitab, and others. When they have asked me to use those (sometimes offered to buy me a student copy) I have refered them to the licence agreements. In some cases you are supposed to upgrade to a full version to be able to use it for anything published, the acedemic price is to be used only in class for learning how to use it. I have SAS, SPSS, S-PLUS, and others installed on my work computer, but I have R installed at work, home, my laptop, and even the pc at my church (submitting monthly statistics as Chernoff faces got me promoted to working in the nursary once, need to try that again). I tend to default to using R mostly these days, partly because I know that I don't have to be chained to my work computer to use it. I even submitted a package to CRAN yesterday while traveling 60 mph on the bus ride home. My first born male child is still 3 and cute, so I had better check my SAS licence and maybe not use it for a while. Maybe in 10 years when he is a teenager I can get the licence that includes #6 (that is a joke, don't tell my wife). -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare [EMAIL PROTECTED] (801) 408-8111 > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Barry Rowlingson > Sent: Wednesday, February 20, 2008 5:52 AM > To: Earl F. Glynn > Cc: [EMAIL PROTECTED] > Subject: Re: [R] History of R > > Earl F. Glynn wrote: > > > Nearly six years ago, SAS also refused to give us academic pricing > > because we were not a degree granting institution. About a > year ago, > > SAS finally granted us academic pricing, but most of the analysis > > momentum was already for the use of R/Bioconductor. > > I recently read the small print on the academic license our > site has for SAS. You have to: > > 1 inform SAS of any taught courses that use SAS > 2 inform SAS of any research projects using SAS > 3 allow SAS to refer to your institution as a SAS user > 4 allow SAS to review your taught courses > 5 ensure your courses are taught using qualified personnel. > 6 give SAS your first-born male offspring > > I spoke to our site's licensing supremos and they say > they've never heard of anyone complying with 1 or 2. Point 4 > sounds like petty fiddling in our educational business, and > point 5 left 'qualified' > undefined. Point 6 doesn't bother me since I don't have kids. > > Luckily other parts of our institution have made deals with > SAS to use it for consulting and training, so perhaps some of > these points don't apply to my department. I've not seen the > small print on that contract though, but I expect it to be > written in blood on freshly slaughtered deerskin... > > Barry > > __ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History of R
I agree with what others have said, the R core team is a great and unique group. There are a couple of ideas that I would like to add that may have played a part in the level of growth that R has had. I think timing has played a part. The field of statistics has matured along with the computer. Math and the other sciences were already mature and established before computers came along, statistics is a much younger science and we were better able to develop our use of computers as computers developed. Also when you look at the trends of comercial packages in the 90's you see that a big focus in the comercial stats packages was on developing easier to use graphical user interfaces, the money at the time was in expanding to new users who were not as technical, and comercial companies need to go where the money is. This meant that the power users who wanted more flexibility and did not care as much about ease of use would natually migrate to R which was not interested in following the money. Another thing to take into account is that R is a package used by statisticians and statisticians are naturally a collaborative group. -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare [EMAIL PROTECTED] (801) 408-8111 > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Kathy Gerber > Sent: Friday, February 15, 2008 12:53 PM > To: r-help@r-project.org > Subject: [R] History of R > > Earlier today I sent a question to Frank Harrell as an R > developer with whom I am most familiar. He suggested also > that I put my questions to the list for additional responses. > Next month I'll be giving a talk on R as an example of high > quality open source software. I think there is much to learn > from R as a high quality extensible product that (at least as > far as I can tell) has never been "spun" or "hyped" like so > many open source fads. > > The question that intrigues me the most is why is R as an > open source project is so incredibly successful and other > projects, say for example, Octave don't enjoy that level of success? > > I have some ideas of course, but I would really like to know > your thoughts when you look at R from such a vantage point. > > Thanks. > Kathy Gerber > University of Virginia > ITC - Research Computing Support > > __ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History of R
> "Greg" == Greg Snow <[EMAIL PROTECTED]> writes: > There are a couple of ideas that I would like to add that > may have played a part in the level of growth that R has > had. Something that I haven't seen mentioned yet that played a role in my adopting R is the ability to create excellent graphics. Every few years I used to do a search to find better tools for analysis and graphics - I haven't felt the need to repeat that since I found R. Mike -- Michael A. Miller [EMAIL PROTECTED] Imaging Sciences, Department of Radiology, IU School of Medicine __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] History of R
Kathy The dedication of the developers and several other important things have already been mentioned. Here are a few points I have not seen. - I believe S was originally open source (before the term existed and before GPL, and license issues were probably clouded with respect to changing the code). This meant parts of the community of S user had this tradition. Some, no doubt, were a bit upset about the Splus move to closed source. - This community had also significantly contributed to Statlib, so there were some "packages" that could be leveraged in the beginning. This may have been not so important for what the packages did, but for the fact that they gave an extensive test suite, so one could have considerable confidence in the results. - Purchase cost is typically not so important for corporate and institutional users, since it is usually dominated by support costs. However, young users may often feel they would prefer to have their personal investment in something they can easily take with them if they move. Some of us at the other end like the idea that we don't need a corporate account to continue research we might be interested in doing when we retire. - All risk averse users should like the idea that programs and acquired skills are not tied to the operating system and hardware flavor of the month. (R has excelled in this respect.) - Help on the R lists has always been exceptionally good (sometimes even if you don't read the documentation first - but expect to be chastised). If you look at the S help list over the past 15 years, you will find many of the most difficult questions were answered by people involved with R. - I ran my own code interchangeably in Splus and R for many years (starting with R-0.16). For a long time Splus was "production" and R was so I would have a backup. For me, the defining factor in moving to R for "production" was the introduction of the "package" system. This is really special in the way that it develops the synergy of the community. By packaging your code you get to leverage all the code checking and documentation checking of the system, and you get to add your own tests that run automatically when you build your package. Not only that, but if you make your package availabe on CRAN you get not only the useful feedback from users, but also the automatic information about what is going to break in your code in the next release of R (from the daily checks on multiple platforms). This is not only useful to package developers, but provides R itself with what I would guess is the largest automatic test bed in the industry. The system is also interesting in the way that it has resolved one of the big problems of Statlib: there is an automatic mechanism for removing broken and unmaintained packages. Paul Gilbert Kathy Gerber wrote: > Earlier today I sent a question to Frank Harrell as an R developer with > whom I am most familiar. He suggested also that I put my questions to > the list for additional responses. Next month I'll be giving a talk on > R as an example of high quality open source software. I think there is > much to learn from R as a high quality extensible product that (at least > as far as I can tell) has never been "spun" or "hyped" like so many open > source fads. > > The question that intrigues me the most is why is R as an open source > project is so incredibly successful and other projects, say for example, > Octave don't enjoy that level of success? > > I have some ideas of course, but I would really like to know your > thoughts when you look at R from such a vantage point. > > Thanks. > Kathy Gerber > University of Virginia > ITC - Research Computing Support > > __ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. La version française suit le texte anglais. This email may contain privileged and/or confidential information, and the Bank of Canada does not waive any related rights. Any distribution, use, or copying of this email or the information it contains by other than the intended recipient is unauthorized. If you received this email in error please delete it immediately from your system and notify the sender promptly by email that you have done so. Le présent courriel peut contenir de l'information privilégiée ou confidentielle. La Banque du Canada ne renonce pas aux droits qui s'y rapportent. Toute diffusion, utilisation ou copie de ce courriel ou des renseignements qu'il contient par une personne autre
Re: [R] History of R
Kathy, You might find some relevant reading in volume 13 of the Journal of Statistical Software: http://www.jstatsoft.org/v13 Some of the papers have a bit of discussion on why R has become more widely used than lisp-stat. K Wright On Fri, Feb 15, 2008 at 1:53 PM, Kathy Gerber <[EMAIL PROTECTED]> wrote: > Earlier today I sent a question to Frank Harrell as an R developer with > whom I am most familiar. He suggested also that I put my questions to > the list for additional responses. Next month I'll be giving a talk on > R as an example of high quality open source software. I think there is > much to learn from R as a high quality extensible product that (at least > as far as I can tell) has never been "spun" or "hyped" like so many open > source fads. > > The question that intrigues me the most is why is R as an open source > project is so incredibly successful and other projects, say for example, > Octave don't enjoy that level of success? > > I have some ideas of course, but I would really like to know your > thoughts when you look at R from such a vantage point. > > Thanks. > Kathy Gerber > University of Virginia > ITC - Research Computing Support > > __ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
