Han Si, asks a good question. See below.
I usually use a structure like this:
snakemake .... -c "qsub -l {params.cluster}"
Then in the rules:
rule ABC:
input: ....
output: ....
params: cluster="-l nodes=1:gpfs"
This way, you can specify the nodes per rule. If you use this
approach, remember to set the "params: cluster" for ALL the rules
since snakemake will need to substitute it into the -c argument.
I use a submit script that looks like this:
#!/bin/bash
#
cd /path/to/your/project
snakemake --unlock
snakemake -d `pwd` -s `pwd`/Snakefile --stats snakemake.stats
--rerun-incomplete -j 100 --cluster 'qsub -l {params.stuff}' &>
snakemake.log
Then, I qsub this script and IT begins the snakemake pipeline. This
keeps snakemake from running on the head node. The -j 100 will have
snakemake submit as many as 100 jobs at a time; when one completes, it
will submit another to keep the number of submitted jobs at 100.
Hope that helps.
Sean
On Fri, May 10, 2013 at 9:59 AM, Si, Han (NIH/NCI) [C] <[email protected]>
wrote:
> Sean,
> I am curious how to specify the nodes on biowulf using snakemake? Can we
> just use any biowulf qsub options following the -c option of snakemake?
> Thanks
>
> Han
>
> Han Si, Ph.D. [Contractor]
> Bioinformatics Analyst
> CCR Bioinformatics Core
>
> Advanced Biomedical Computing Center (ABCC)
> Information Systems Program
> SAIC-Frederick, Inc.
> Frederick National Laboratory for Cancer Research (FNLCR)
> P.O. Box B
> Frederick, MD 21702
>
> Email: [email protected]
> Office: Building 41- B620, NIH, Bethesda, MD
> Phone: 301.451.6087
> Fax (Bethesda) : 301.480.0391
>
>
>
>
>
> On 5/10/13 7:59 AM, "Davis, Sean (NIH/NCI) [E]" <[email protected]>
> wrote:
>
>>I wrote a quick blog post about snakemake, a python-based make-like
>>workflow framework.
>>
>>http://watson.nci.nih.gov/~sdavis/blog/flexible_bioinformatics_pipelines_w
>>ith_snakemake/
>>
>>For those local to NIH, snakemake can seemlessly submit entire
>>projects to biowulf and track changes to inputs, outputs, software
>>versions, and code.
>>
>>Sean
>>
>>########################################################################
>>
>>To unsubscribe from the CCRIFX_CORE_GROUP list, click the following link:
>>http://list.nih.gov/cgi-bin/wa.exe?SUBED1=CCRIFX_CORE_GROUP&A=1
>
