Windows uses separate processes that do not share memory (SnowParam()), whereas
linux / mac by default use forked processes that share the original memory
(MulticoreParam()). So
> y = 1
> param = MulticoreParam()
> res = bplapply(1:2, function(x) y, BPPARAM=param)
works because the function can 'see' y, whereas
> param = SnowParam()
> res = bplapply(1:2, function(x) y, BPPARAM=param)
Error: BiocParallel errors
element index: 1, 2
first error: object 'y' not found
fails because the new processes cannot see y. The robust way to implement this
makes the FUN in bplapply truly functional, depending only on variables
received as arguments
> res = bplapply(1:2, function(x, y) y, y = 1, BPPARAM=param)
>
The situation is more complicated in package code in particular and in general,
because R sends not just FUN to the workers, but also the environment (up to
but not including the global environment) in which the function applies. So
doit <- function() {
y = 1
bplapply(1:2, function(x) y, BPPARAM = SnowParam(2))
}
also works
> res = doit()
>
Note that
fun = function(x) y
doit = function() {
y = 1
bplapply(1:2, fun, BPPARAM = SnowParam(2))
}
fails, because y is not defined in the environment in which fun is defined
(it's defined in the calling environment, which is different).
The most robust advice is to develop you code with the most conservative
assumptions (e.g., register(SnowParam())) and to write functional functions
where all variables are passed as arguments.
Martin
On 3/17/20, 6:00 AM, "Bioc-devel on behalf of Dario Strbenac"
<bioc-devel-boun...@r-project.org on behalf of dstr7...@uni.sydney.edu.au>
wrote:
Good day,
I have a loop in a function of my R package which by default uses bpparam()
to set the framework used for parallelisation. On Windows, I see the error
Error: BiocParallel errors
element index: 1, 2, 3, 4, 5, 6, ...
first error: object 'selParams' not found
This error does not happen on the Linux or MacOS operating systems. It
happens using both R 3.6 and the upcoming version 4. The error can be
reproduced running the examples of runTests function in ClassifyR.
--------------------------------------
Dario Strbenac
University of Sydney
Camperdown NSW 2050
Australia
_______________________________________________
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
_______________________________________________
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
