Re: [galaxy-dev] Improved DRMAAJobRunner (Peter van Heusden)
gt; >>> Good morning, > >>> > >>> > >>> I have a user attempting to run ustacks. They are receiving the > >>> following error > >>> > >>> > >>> Fatal error: Exit code 126 (Error in Stacks execution) > >>> > >>> > >>> > /home/big_galaxy/galaxy/database/jobs_directory/079/79363/tool_script.sh: > >>> > >>> > >>> > /home/big_galaxy/galaxy/tool_dependencies/_conda/envs/mulled-v1-8e6d35eac718a13db6458b9361ca87c93be9feed656f9938ebcc435642f3a0a3/bin/stacks_summary.py: > >>> > >>> > >>> > /home/big_galaxy/galaxy/tool_dependencies/_conda/envs/mulled-v1-8e6d35eac718a1: > >>> > >>> bad interpreter: No such file or directory > >>> > >>> > >>> The tool produces an additional output on the bug report – > >>> > >>> > >>> ustacks parameters selected: > >>> > >>> Sample ID: 1 > >>> > >>>Min depth of coverage to create a stack: 3 > >>> > >>>Max distance allowed between stacks: 2 > >>> > >>>Max distance allowed to align secondary reads: 4 > >>> > >>>Max number of stacks allowed per de novo locus: 3 > >>> > >>>Deleveraging algorithm: disabled > >>> > >>>Removal algorithm: enabled > >>> > >>>Model type: SNP > >>> > >>>Alpha significance level for model: 0.05 > >>> > >>>Gapped alignments: disabled > >>> > >>> Parsing stacks_inputs/ZU-6.fq > >>> > >>> Loading RAD-Tags...done > >>> > >>> Loaded 933120 RAD-Tags. > >>> > >>>Inserted 256463 elements into the RAD-Tags hash map. > >>> > >>>0 reads contained uncalled nucleotides that were modified. > >>> > >>> 65509 initial stacks were populated; 190954 stacks were set > >>> aside as > >>> secondary reads. > >>> > >>> Initial coverage mean: 10.9276; Std Dev: 25.3864; Max: 4650 > >>> > >>> Deleveraging trigger: 36; Removal trigger: 62 > >>> > >>> Calculating distance for removing repetitive stacks. > >>> > >>>Distance allowed between stacks: 1; searching with a k-mer > length > >>> of 35 (36 k-mers per read); 1 k-mer hits required. > >>> > >>> Removing repetitive stacks. > >>> > >>>Removed 993 stacks. > >>> > >>>64677 stacks remain for merging. > >>> > >>> Post-Repeat Removal, coverage depth Mean: 10.2655; Std Dev: > >>> 9.03472; > >>> Max: 62 > >>> > >>> Calculating distance between stacks... > >>> > >>>Distance allowed between stacks: 2; searching with a k-mer > length > >>> of 23 (48 k-mers per read); 2 k-mer hits required. > >>> > >>> Merging stacks, maximum allowed distance: 2 nucleotide(s) > >>> > >>>64677 stacks merged into 57327 loci; deleveraged 0 loci; > >>> blacklisted 189 loci. > >>> > >>> After merging, coverage depth Mean: 11.0747; Std Dev: 9.82823; > >>> Max: 91 > >>> > >>> Merging remainder radtags > >>> > >>>217265 remainder sequences left to merge. > >>> > >>>Distance allowed between stacks: 4; searching with a k-mer > length > >>> of 13 (58 k-mers per read); 6 k-mer hits required. > >>> > >>>Matched 26211 remainder reads; unable to match 191054 > >>> remainder reads. > >>> > >>> After remainders merged, coverage depth Mean: 11.5347; Std Dev: > >>> 9.95581; Max: 93 > >>> > >>> Calling final consensus sequences, invoking SNP-calling model... > >>> > >>> Number of utilized reads: 742066 > >>> > >>> Writing loci, SNPs, and alleles to 'stacks_outputs/'... > >>> > >>>Refetching sequencing IDs from stacks_inputs/ZU-6.fq... read > >>> 933120 sequence IDs. > >>> > >
Re: [galaxy-dev] Improved DRMAAJobRunner (Peter van Heusden)
e6d35eac718a1: bad interpreter: No such file or directory The tool produces an additional output on the bug report – ustacks parameters selected: Sample ID: 1 Min depth of coverage to create a stack: 3 Max distance allowed between stacks: 2 Max distance allowed to align secondary reads: 4 Max number of stacks allowed per de novo locus: 3 Deleveraging algorithm: disabled Removal algorithm: enabled Model type: SNP Alpha significance level for model: 0.05 Gapped alignments: disabled Parsing stacks_inputs/ZU-6.fq Loading RAD-Tags...done Loaded 933120 RAD-Tags. Inserted 256463 elements into the RAD-Tags hash map. 0 reads contained uncalled nucleotides that were modified. 65509 initial stacks were populated; 190954 stacks were set aside as secondary reads. Initial coverage mean: 10.9276; Std Dev: 25.3864; Max: 4650 Deleveraging trigger: 36; Removal trigger: 62 Calculating distance for removing repetitive stacks. Distance allowed between stacks: 1; searching with a k-mer length of 35 (36 k-mers per read); 1 k-mer hits required. Removing repetitive stacks. Removed 993 stacks. 64677 stacks remain for merging. Post-Repeat Removal, coverage depth Mean: 10.2655; Std Dev: 9.03472; Max: 62 Calculating distance between stacks... Distance allowed between stacks: 2; searching with a k-mer length of 23 (48 k-mers per read); 2 k-mer hits required. Merging stacks, maximum allowed distance: 2 nucleotide(s) 64677 stacks merged into 57327 loci; deleveraged 0 loci; blacklisted 189 loci. After merging, coverage depth Mean: 11.0747; Std Dev: 9.82823; Max: 91 Merging remainder radtags 217265 remainder sequences left to merge. Distance allowed between stacks: 4; searching with a k-mer length of 13 (58 k-mers per read); 6 k-mer hits required. Matched 26211 remainder reads; unable to match 191054 remainder reads. After remainders merged, coverage depth Mean: 11.5347; Std Dev: 9.95581; Max: 93 Calling final consensus sequences, invoking SNP-calling model... Number of utilized reads: 742066 Writing loci, SNPs, and alleles to 'stacks_outputs/'... Refetching sequencing IDs from stacks_inputs/ZU-6.fq... read 933120 sequence IDs. done. ustacks is done. I am not sure what is causing the bad interpreter: No such file or directory. The tools are correctly installed along with its requisite conda packages. Any help would be greatly appreciated. Best Regards, David ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ -- Message: 3 Date: Sat, 08 Jul 2017 11:56:50 +0200 From: Matthias Bernt <m.be...@ufz.de> To: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Improved DRMAAJobRunner (Peter van Heusden) Message-ID: <fb33e9595421.5960c...@ufz.de> Content-Type: text/plain; charset="utf-8" Hi Peter, Code looks interesting. This is for Univa Grid Engine? In its current state its tailored for the Univa Grid Engine. But the drmaa library code that is based on job_info() and wait() should run on all grid engines. For the command line based code changes might be necessary for the different grid engines. (Currently there is only a small bug, because wait() is currently called twice for finished jobs: in the repeated polling and then in the final check. The second call won't work which causes a fallback to qacct. I will fix this soon.) I guess that the output of qstat and qacct will be different. But I think this can be configured one just needs a way to get the info which grid engine is running. In terms of the upload jobs, are those not designed to be run as 'local' jobs and not with the 'real user' setting? Sounds reasonable, but this is not what is happening on my installation of galaxy. Any idea where I could start to find the problem. Best, Matthias -- next part -- An HTML attachment was scrubbed... URL: <https://lists.galaxyproject.org/piperma
Re: [galaxy-dev] Improved DRMAAJobRunner (Peter van Heusden)
Max distance allowed to align secondary reads: 4 Max number of stacks allowed per de novo locus: 3 Deleveraging algorithm: disabled Removal algorithm: enabled Model type: SNP Alpha significance level for model: 0.05 Gapped alignments: disabled Parsing stacks_inputs/ZU-6.fq Loading RAD-Tags...done Loaded 933120 RAD-Tags. Inserted 256463 elements into the RAD-Tags hash map. 0 reads contained uncalled nucleotides that were modified. 65509 initial stacks were populated; 190954 stacks were set aside as secondary reads. Initial coverage mean: 10.9276; Std Dev: 25.3864; Max: 4650 Deleveraging trigger: 36; Removal trigger: 62 Calculating distance for removing repetitive stacks. Distance allowed between stacks: 1; searching with a k-mer length of 35 (36 k-mers per read); 1 k-mer hits required. Removing repetitive stacks. Removed 993 stacks. 64677 stacks remain for merging. Post-Repeat Removal, coverage depth Mean: 10.2655; Std Dev: 9.03472; Max: 62 Calculating distance between stacks... Distance allowed between stacks: 2; searching with a k-mer length of 23 (48 k-mers per read); 2 k-mer hits required. Merging stacks, maximum allowed distance: 2 nucleotide(s) 64677 stacks merged into 57327 loci; deleveraged 0 loci; blacklisted 189 loci. After merging, coverage depth Mean: 11.0747; Std Dev: 9.82823; Max: 91 Merging remainder radtags 217265 remainder sequences left to merge. Distance allowed between stacks: 4; searching with a k-mer length of 13 (58 k-mers per read); 6 k-mer hits required. Matched 26211 remainder reads; unable to match 191054 remainder reads. After remainders merged, coverage depth Mean: 11.5347; Std Dev: 9.95581; Max: 93 Calling final consensus sequences, invoking SNP-calling model... Number of utilized reads: 742066 Writing loci, SNPs, and alleles to 'stacks_outputs/'... Refetching sequencing IDs from stacks_inputs/ZU-6.fq... read 933120 sequence IDs. done. ustacks is done. I am not sure what is causing the bad interpreter: No such file or directory. The tools are correctly installed along with its requisite conda packages. Any help would be greatly appreciated. Best Regards, David ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ -- Message: 3 Date: Sat, 08 Jul 2017 11:56:50 +0200 From: Matthias Bernt <m.be...@ufz.de> To: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Improved DRMAAJobRunner (Peter van Heusden) Message-ID: <fb33e9595421.5960c...@ufz.de> Content-Type: text/plain; charset="utf-8" Hi Peter, Code looks interesting. This is for Univa Grid Engine? In its current state its tailored for the Univa Grid Engine. But the drmaa library code that is based on job_info() and wait() should run on all grid engines. For the command line based code changes might be necessary for the different grid engines. (Currently there is only a small bug, because wait() is currently called twice for finished jobs: in the repeated polling and then in the final check. The second call won't work which causes a fallback to qacct. I will fix this soon.) I guess that the output of qstat and qacct will be different. But I think this can be configured one just needs a way to get the info which grid engine is running. In terms of the upload jobs, are those not designed to be run as 'local' jobs and not with the 'real user' setting? Sounds reasonable, but this is not what is happening on my installation of galaxy. Any idea where I could start to find the problem. Best, Matthias -- next part -- An HTML attachment was scrubbed... URL: <https://lists.galaxyproject.org/pipermail/galaxy-dev/attachments/20170708/a72251d4/attachment-0001.html> -- Message: 4 Date: Sat, 08 Jul 2017 13:52:39 + From: Peter van Heusden <p...@sanbi.ac.za> To: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Impro
Re: [galaxy-dev] Improved DRMAAJobRunner (Peter van Heusden)
On Sat, 8 Jul 2017 at 11:56 Matthias Berntwrote: > Hi Peter, > > > Code looks interesting. This is for Univa Grid Engine? > > In its current state its tailored for the Univa Grid Engine. But the drmaa > library code that is based on job_info() and wait() should run on all grid > engines. > For the command line based code changes might be necessary for the > different grid engines. (Currently there is only a small bug, because > wait() is currently > called twice for finished jobs: in the repeated polling and then in the > final check. The second call won't work which causes a fallback to qacct. I > will fix this > soon.) > I guess that the output of qstat and qacct will be different. But I think > this can be configured one just needs a way to get the info which grid > engine is running. > > > In terms of the upload jobs, are those not designed to be run as 'local' > jobs and not with the 'real user' setting? > > Sounds reasonable, but this is not what is happening on my installation of > galaxy. Any idea where I could start to find the problem. > > I'm not an expert on this, but what does your job_conf.xml look like? Peter > Best, > Matthias ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Re: [galaxy-dev] Improved DRMAAJobRunner (Peter van Heusden)
Hi Peter, > Code looks interesting. This is for Univa Grid Engine? In its current state its tailored for the Univa Grid Engine. But the drmaa library code that is based on job_info() and wait() should run on all grid engines. For the command line based code changes might be necessary for the different grid engines. (Currently there is only a small bug, because wait() is currently called twice for finished jobs: in the repeated polling and then in the final check. The second call won't work which causes a fallback to qacct. I will fix this soon.) I guess that the output of qstat and qacct will be different. But I think this can be configured one just needs a way to get the info which grid engine is running. > In terms of the upload jobs, are those not designed to be run as 'local' jobs > and not with the 'real user' setting? Sounds reasonable, but this is not what is happening on my installation of galaxy. Any idea where I could start to find the problem. Best, Matthias ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/