Uhm... maybe this is the issue.
The issue seems to specially occur when I am building the vignette,
which performs some parallel computations on a reduced example,
therefore faster than in a normal usage of the package.
The two processes (on my dual core) output some logging information
using cat, which are normally sent to the console, but I guess that in
the case of a vignette these are written to tex file. It is very few
data though (a loop counter), so writing should be also very quick, even
in a file.
Could it be possible that a I/O deadlock happens because of this output?
I actually use mutexes, at the end of each loop to perform bigger
writing operation on a common file, but I hadn't think these would be
required for the logging output, assuming that stdout and stderr were
thread safe. Maybe I should use mutexes at this level too.
Does multicore or doMC provide optional separate logging as doMPI does?
(I guess this might be better posted to R-hpc)
Thank you.
Renaud
On 17/08/2011 14:56, Brian G. Peterson wrote:
On Wed, 2011-08-17 at 04:50 -0700, Tim Triche, Jr. wrote:
I'll see if I can put together a self-contained example. Primarily,
the times that I use multicore (and attempted to use doSMP, mostly
because one of my users refuses to ditch Windows) are when I am
reading a ton of binary files, none of which depend on the others.
This is a blindingly obvious use-case for e.g. doMC and doSMP, yet
what typically happens is that the entire operation wedges. I'm told
that doSMP really only works well with Revolution R, but per above, I
will try to put together a working self-contained example to show
how.
Remember that physical I/O can bind up the processes too. Having a
bunch of processes all trying to read from local disk at the same time
(especially when they are all trying to read the same file(s), a problem
it seems you may not have) is a recipe for I/O locks that can seize up
your processes.
So, if the problem only occurs with physical I/O, the first thing I
would try is to move that storage to a storage device on another machine
that is tuned for that level of disk I/O.
Regards,
- Brian
###
UNIVERSITY OF CAPE TOWN
This e-mail is subject to the UCT ICT policies and e-mai...{{dropped:5}}
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
