Re: [Bioc-devel] BiocParallel: BatchJobs backend (Was: Re: BiocParallel)

2013-06-06 Thread Henrik Bengtsson
Hi, I'd like to pick up the discussion on a BatchJobs backend for BiocParallel where it was left back in Dec 2012 (Bioc-devel thread 'BiocParallel' [https://stat.ethz.ch/pipermail/bioc-devel/2012-December/003918.html]). Florian, would you mind sharing your BatchJobs backend code? Is it independe

Re: [Bioc-devel] BiocParallel: BatchJobs backend (Was: Re: BiocParallel)

2013-06-06 Thread Dan Tenenbaum
On Thu, Jun 6, 2013 at 1:56 PM, Henrik Bengtsson wrote: > Hi, I'd like to pick up the discussion on a BatchJobs backend for > BiocParallel where it was left back in Dec 2012 (Bioc-devel thread > 'BiocParallel' > [https://stat.ethz.ch/pipermail/bioc-devel/2012-December/003918.html]). > > Florian,

Re: [Bioc-devel] BiocParallel: BatchJobs backend (Was: Re: BiocParallel)

2013-06-06 Thread Michael Lawrence
And here is the on-going development of the backend: https://github.com/mllg/BiocParallel/tree/batchjobs Not sure how well it's been tested. Kudos to Michel Lang for making so much progress so quickly. Michael On Thu, Jun 6, 2013 at 1:59 PM, Dan Tenenbaum wrote: > On Thu, Jun 6, 2013 at 1:56

Re: [Bioc-devel] BiocParallel: BatchJobs backend (Was: Re: BiocParallel)

2013-06-06 Thread Henrik Bengtsson
Great - this looks promising already. What's your test system(s), beyond standard SSH and multicore clusters? I'm on a Torque/PBS system. I'm happy to test, give feedback etc. I don't see an 'Issues' tab on the GitHub page. Michel, how do you prefer to get feedback? /Henrik On Thu, Jun 6, 2

Re: [Bioc-devel] BiocParallel: BatchJobs backend (Was: Re: BiocParallel)

2013-06-06 Thread Michael Lawrence
We're regularly running BatchJobs itself on an LSF cluster. Works great. On Thu, Jun 6, 2013 at 5:48 PM, Henrik Bengtsson wrote: > Great - this looks promising already. > > What's your test system(s), beyond standard SSH and multicore > clusters? I'm on a Torque/PBS system. > > I'm happy to tes

Re: [Bioc-devel] BiocParallel: BatchJobs backend (Was: Re: BiocParallel)

2013-06-25 Thread Michel Lang
Hi Henrik, Sorry for the late response. Suggestions and feedback are always welcome. I just forgot to enable the issue tracker (now enabled). For prototyping I usually use Interactive/Multicore, but I'll regularly test on our local clusters which use Torque or Slurm, respectively. Michel 2013/6

Re: [Bioc-devel] BiocParallel: BatchJobs backend (Was: Re: BiocParallel)

2013-06-26 Thread Hahne, Florian
This slipped under my radar, sorry. Guess there already is some considerable work going on right now to bring queued clusters closer into the Bioconductor world. If my first attempt on this is still of any help I am happy to share it, of course. Just let me know. Michael: I tried to install your Bi

Re: [Bioc-devel] BiocParallel: BatchJobs backend (Was: Re: BiocParallel)

2013-06-27 Thread Michel Lang
Hi Florian, Yes you're absolutely right. The fork currently depends on some functions which are not yet included in the CRAN build. For now you can get the latest development version on http://batchjobs.googlecode.com. We'll upload a new version of BatchJobs soon. I've documented this as an issue

Re: [Bioc-devel] BiocParallel: BatchJobs backend (Was: Re: BiocParallel)

2013-07-03 Thread Hahne, Florian
Thanks for the info, the package now installs. Looks pretty cool. Attached is my first hack at this from a couple of months ago. In a way it is very similar to what you guys are doing, only that It predates the times of a formal registry for parallel backends. I just tried to make it look like a cl