Hi John, Have you tried wrapping your R read in system.time()? If you are right about :results silent eating up lots of time, then this should fix the problem. system.time yields just a bit of output, so shouldn't slow things down if writing out the data is indeed the problem as you suspect.
hth, Tom John Hendy <jw.he...@gmail.com> writes: > On Wed, Oct 31, 2012 at 3:12 PM, <cbe...@tajo.ucsd.edu> wrote: > >> John Hendy <jw.he...@gmail.com> writes: >> >> > On Wed, Oct 31, 2012 at 11:41 AM, <span dir="ltr"><mailto: >> cbe...@tajo.ucsd.edu></span> wrote: >> > John Hendy <mailto:jw.he...@gmail.com> writes: >> > >> >> I edited the subject to be more concise/clear.I let orgmode chug away >> >> on reading in some ~10-30mb csv files for nearly 30min. >> > >> > [rest deleted] >> > >> > You need an ECM.I did my best to provide one, other than the file, which >> I offered to provide if others requested that I upload it somewhere. Since >> you have done so, so have I: >> > - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM >> >> > Let me know if that doesn't work. I put it on Google docs and >> sometimes have issues with the sharing settings... >> >> Not an ECM in my book, but ... >> >> > What else would you like? I provided: > - the config > - the data > - how to [attempt to] reproduce > - the org-mode text > > > >> On my 4 year old MacBook: >> >> ,---- >> | >> | #+PROPERTY: session *R* >> | >> | #+name: bigcsv >> | #+begin_src R >> | bigcsv <- Sys.glob("~/Downloads/*.csv") >> | #+end_src >> | >> | #+RESULTS: bigcsv >> | : /Users/cberry/Downloads/test-file.csv >> | >> | #+name: readbig >> | #+begin_src R :results output >> | system.time( >> | tmp <- read.csv(bigcsv) >> | ) >> | >> | #+end_src >> | >> | #+RESULTS: readbig >> | : user system elapsed >> | : 5.679 0.306 6.002 >> | >> `---- >> >> About the same as running from ESS. >> >> > Not sure what to say. Looking for ways to troubleshoot or confirm. Since > you can't confirm, any suggestions on where I should look for my issue? I > can't explain it! All I know is that org chugs and chugs and the direct > execution in ESS session is lightning fast. > > > > FYI, >> >> > dim(tmp) >> [1] 964805 9 >> >> >> > Was that just to show the size, or is something wrong with my file? > > >> Chuck >> >> > > Thanks for attempting to replicate! > John > > >> [ rest deleted ] >> >> >> >> > On Wed, Oct 31, 2012 at 3:12 PM, <cbe...@tajo.ucsd.edu> wrote: > > John Hendy <jw.he...@gmail.com> writes: > > > On Wed, Oct 31, 2012 at 11:41 AM, <span > dir="ltr"><mailto:cbe...@tajo.ucsd.edu></span> wrote: > > John Hendy <mailto:jw.he...@gmail.com> writes: > > > >> I edited the subject to be more concise/clear.I let orgmode > chug away > >> on reading in some ~10-30mb csv files for nearly 30min. > > > > [rest deleted] > > > > You need an ECM.I did my best to provide one, other than the > file, which I offered to provide if others requested that I upload > it somewhere. Since you have done so, so have I: > > - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM > > > Let me know if that doesn't work. I put it on Google docs > and sometimes have issues with the sharing settings... > > Not an ECM in my book, but ... > > > > What else would you like? I provided: > - the config > - the data > - how to [attempt to] reproduce > - the org-mode text > > > > On my 4 year old MacBook: > > ,---- > | > | #+PROPERTY: session *R* > | > | #+name: bigcsv > | #+begin_src R > | bigcsv <- Sys.glob("~/Downloads/*.csv") > | #+end_src > | > | #+RESULTS: bigcsv > | : /Users/cberry/Downloads/test-file.csv > | > | #+name: readbig > | #+begin_src R :results output > | system.time( > | tmp <- read.csv(bigcsv) > | ) > | > | #+end_src > | > | #+RESULTS: readbig > | : user system elapsed > | : 5.679 0.306 6.002 > | > `---- > > About the same as running from ESS. > > > > Not sure what to say. Looking for ways to troubleshoot or confirm. > Since you can't confirm, any suggestions on where I should look for my > issue? I can't explain it! All I know is that org chugs and chugs and > the direct execution in ESS session is lightning fast. > > > > FYI, > > > dim(tmp) > [1] 964805 9 > > > > > Was that just to show the size, or is something wrong with my file? > > > Chuck > > > > > Thanks for attempting to replicate! > John > > > [ rest deleted ] > > > > > -- Thomas S. Dye http://www.tsdye.com
