Re: [Bioc-devel] BiocParallel load balancing and runtime

2023-08-08 Thread Jiefei Wang
Hello Anna, The speed of parallel computing depends on many factors. To avoid any potential confounders, Please try to use this code for timing (assuming you still have all the variables you used in your example) ``` parallel_param <- SnowParam(workers = ncores, type = "SOCK", tasks =

Re: [Bioc-devel] BiocParallel load balancing and runtime

2023-08-08 Thread Anna Plaxienko
My motivation for using distributed memory was that my package is also accessible on Windows. Is it better to use shared memory as default but check the user's system and then switch to socket only if necessary? Regarding the real data. I have 68 samples (rows) of methylation EPIC array data (850K

Re: [Bioc-devel] BiocParallel load balancing and runtime

2023-08-08 Thread Waldir Leoncio Netto
Dear Anna, According to the documentation of "BiocParallelParam", SnowParam() is a subclass suitable for distributed memory (e.g. cluster) computing. If you're running your code on a simpler machine with shared memory (e.g. your PC), you're probably better off using MulticoreParam() instead. He

[Bioc-devel] BiocParallel load balancing and runtime

2023-08-08 Thread Anna Plaxienko
Hi all! I'm switching from the base R *parallel* package to *BiocParallel* for my Bioconductor submission and I have two questions. First, I wanted advice on whether I've implemented load balancing correctly. Second, I've noticed that the running time is about 15% longer with BiocParallel. Any ide