Re: [R] Parallel Scan of Large File

2010-12-09 Thread Luedde, Mirko
Hi Ryan, the Getting Started with doMC and foreach manual tells me that you might have forgotten a registerDoMC() in the example you provided. Best, Mirko __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE

Re: [R] Parallel Scan of Large File

2010-12-08 Thread Mike Marchywka
Date: Tue, 7 Dec 2010 17:22:57 -0800 From: ryan.steven.gar...@gmail.com To: r-help@r-project.org Subject: [R] Parallel Scan of Large File Is it possible to parallel scan a large file into a character vector in 1M chunks using scan

Re: [R] Parallel Scan of Large File

2010-12-08 Thread jim holtman
: ryan.steven.gar...@gmail.com To: r-help@r-project.org Subject: [R] Parallel Scan of Large File Is it possible to parallel scan a large file into a character vector in 1M chunks using scan() with the doMC package? Furthermore, can I specify the tasks for each child? i.e. I'm working on a Linux box with 8

Re: [R] Parallel Scan of Large File

2010-12-08 Thread Ryan Garner
Hi Jim, Thanks for your insight. I used Linux split to split my large file into smaller partitions. On the server I work on, multipath I/O access is enabled and we use RAID for storage; thus, I don't think I can put each partition on a spindle. I'm able to open multiple files at a time into

[R] Parallel Scan of Large File

2010-12-07 Thread Ryan Garner
Is it possible to parallel scan a large file into a character vector in 1M chunks using scan() with the doMC package? Furthermore, can I specify the tasks for each child? i.e. I'm working on a Linux box with 8 cores and would like to scan in 8M records at time (all 8 cores scan 1M records at a