Re: [galaxy-dev] [SPAM] RE: GalaxyAdmins Online Meetup this Thursday: Peter Briggs on Galactic Engineering at Manchester
We’re going to try this one now: https://connect.johnshopkins.edu/r283zwzv0yi On Jun 18, 2015, at 11:09 AM, Hans-Rudolf Hotz hansrudolf.h...@fmi.ch wrote: sorry my (copy paste) mistake https://connect.johnshopkins.edu/galaxyadmins201506/ On Jun 18, 2015, at 5:08 PM, Christian Brenninkmeijer christian.brenninkmei...@manchester.ac.uk wrote: Does anyone have a link to the meeting? Page below only has a general link not to this meeting. From: galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf of Hans-Rudolf Hotz [hansrudolf.h...@fmi.ch] Sent: Thursday, June 18, 2015 4:03 PM To: Galaxy Dev List Subject: Re: [galaxy-dev] GalaxyAdmins Online Meetup this Thursday: Peter Briggs on Galactic Engineering at Manchester Hi everyone, the online meetup will (hopefully) start in a few minutes at: https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2015_06_18 Regards, Hans-Rudolf On Jun 15, 2015, at 3:47 PM, Dave Clements cleme...@galaxyproject.org wrote: Hello all, The June 2015 online GalaxyAdmins meetup will be this Thursday, 18 June, at 5pm Central European Time (see your local time). Peter Briggs of the University of Manchester and author of the Galactic Engineer blog will talk about his Galaxy work deploying Galaxy and developing tools within the Bioinformatics Core Facility at the University of Manchester. John Chilton will also give an update on recent and upcoming work on Planemo, a set of command-line utilities to assist in building and publishing Galaxy tools. We'll also discuss the upcoming GalaxyAdmins meetup, and the hackathons at GCC2015. We'll post a link on the meetup page on Thursday. Hope to see/hear you on Thursday, Dave C -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] BAM datatype - bam bai
The response speed is awesome! On github Eric answered: https://github.com/galaxyproject/tools-iuc/blob/0887009a23d176b21536c9fd8a18c4fecc417d4f/tools/bedtools/multiCov.xml see L12 of that file for where the bai file is stored. That explains a lot. Very precise example. Thank you! What I'm still wondering is: When is it being generated, what code is used for that? More specific: When I use Picard SortSam in galaxy, the output is a bam file with an accompanying bai file. But when I use Picard on command line, the output is just a bam file. In the Picard SortSam galaxy tool, it is not being done. (If I didn't miss something) What leads me to When I do multiple actions with my bam file - is the index generated every time? (overhead) GATK processes BAM files and at the same time the bai files. It would be a waste to just use the bam files and re-generate the bai files, that already exist. How could that be done? Thanks, Alexander 2015-06-18 18:44 GMT-05:00 Alexander Vowinkel vowinkel.alexan...@gmail.com : Hi, when I 'produce' a bam file in my output, I can download the bam file and a corresponding bai file. I am now wondering where does the bai file come from? When is it being generated, what code is used for that? When I do multiple actions with my bam file - is the index generated every time? (overhead) Where is the bai saved on disk (if it is)? I found that in the IUC gatk, a new bai is always generated when a gatk tool is started. This creates quite an overhead. Isn't there a better solution? @ref https://github.com/galaxyproject/tools-iuc/issues/194 Best, Alexander ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Fwd: Adding packages for bedtools and ucsc-utils for Trackster?
(We discussed this over IRC, but responding to the list posting for posterity/searchability) The JSON.pm file isn't found in your PERL5LIB path because the original version of my JBrowse tool failed to specify the full dependency list and assumed you had some perl modules manually installed. The latest version of my JBrowse tool now includes the full set of dependencies. Since chatting on IRC, this version has been uploaded to the MTS and is available here: https://toolshed.g2.bx.psu.edu/view/iuc/jbrowse/497c6bb3b717 (For the rest of the mailing list reading this, an unofficial release announcement) JBrowse 1.11.6 is available as a tool which can be run and can display: - bed/gff/gff3 - BAM - Blast XML data (automatically converted to gapped GFF3 and displayed with evalue-based intensities) - BigWig data - VCF data putting it closer to being on-par with trackster. It's an excellent end-of-workflow step to summarize all of your generated genomic analyses. If any of the community have feature requests, send them my way! NB: BAM/BigWigs do NOT play nicely with galaxy if you aren't using X-SendFile. 2015-06-18 5:42 GMT-05:00 Peter van Heusden p...@sanbi.ac.za: Thanks Bjoern My initial install didn't pick up the Perl dependencies correctly, but I re-installed and now the install seems to have the required modules but is missing something in the environment variables. Specifically I get an error import JSON.pm, see below. Note that the jbrowse/1.11.6/iuc/package_jbrowse_1_11_6/d5ab749c9e97/lib/perl5 directory isn't included in @INC and in fact the paths that *are* included seem to imply that the wrong perl interpreter is being picked up. Fatal error: Exit code 1 () Can't locate JSON.pm in @INC (you may need to install the JSON module) (@INC contains: /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/database/job_working_directory/000/29/dataset_29_files/JBrowse-1.11.6/bin/../src/perl5 /etc/perl /usr/local/lib/perl/5.18.2 /usr/local/share/perl/5.18.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.18 /usr/share/perl/5.18 /usr/local/lib/site_perl .) at /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/database/job_working_directory/000/29/dataset_29_files/JBrowse-1.11.6/bin/../src/perl5/Bio/JBrowse/JSON.pm line 10. BEGIN failed--compilation aborted at /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/database/job_working_directory/000/29/dataset_29_files/JBrowse-1.11.6/bin/../src/perl5/Bio/JBrowse/JSON.pm line 10. Compilation failed in require at /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/database/job_working_directory/000/29/dataset_29_files/JBrowse-1.11.6/bin/../src/perl5/Bio/JBrowse/Cmd/FormatSequences.pm line 21. BEGIN failed--compilation aborted at /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/database/job_working_directory/000/29/dataset_29_files/JBrowse-1.11.6/bin/../src/perl5/Bio/JBrowse/Cmd/FormatSequences.pm line 21. Compilation failed in require at bin/prepare-refseqs.pl line 7. BEGIN failed--compilation aborted at bin/prepare-refseqs.pl line 7. Traceback (most recent call last): File /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/jbrowse/2c9e5136b416/jbrowse/jbrowse.py, line 44, in module process_genome(jbrowse_dir, os.path.realpath(args.genome.name)) File /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/jbrowse/2c9e5136b416/jbrowse/jbrowse.py, line 17, in process_genome subprocess.check_output(['perl', 'bin/prepare-refseqs.pl', '--fasta', genome], cwd=jbrowse_dir) File /usr/lib/python2.7/subprocess.py, line 573, in check_output raise CalledProcessError(retcode, cmd, output=output) subprocess.CalledProcessError: Command '['perl', 'bin/prepare-refseqs.pl', '--fasta', '/net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/database/files/000/dataset_3.dat']' returned non-zero exit status 2 On 18 June 2015 at 10:01, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Peter, I'm in a hurry ... have a look at https://github.com/galaxyproject/tools-iuc/tree/master/tools/jbrowse Developed by IUC member Eric Rasche. Cheers, Bjoern On 18.06.2015 10:00, Peter van Heusden wrote: I've tried that - I've added the Environment Modules for the relevant tools to the local_env.sh file, so they're there in the environment when Galaxy runs. These tools are scheduled via our cluster, however, so I don't think the environment passes across to them. They depend on these Galaxy packages, however, and those (bedtools and ucsc-utils) are now installed, so I'm not sure why it can't pick them up from the shed_tools. On 18 June 2015 at 09:10, Hans-Rudolf Hotz h...@fmi.ch wrote: On 06/18/2015 09:06 AM, Peter van Heusden wrote: Hi there I'm trying to visualise some of our genome
[galaxy-dev] package_bowtie_2_2_4 Executable Permissions - can't run bowtie as non-galaxy user
Installed bowtie2 from the toolshed, which brings in package_bowtie_2_2_4, which downloads bowtie binaries from depot.galaxyproject.org http://depot.galaxyproject.org/package/linux/x86_64/bowtie2/bowtie2-2.2.4-linux-x86_64.tgz Jobs submitted on our setup, running as a 'real user' (not galaxy user) via pulsar + SLURM fail with a permissions error. This is due to the permissions on some of the bowtie executables in the .tar.gz from depot being owner/group executable only. Other users cannot run bowtie2: -rwxrwx---. 1 galaxy galaxy 18K Oct 22 2014 bowtie2 If I download the tar.gz from depot manually and extract then I get: -rwxrwx--- 1 dtrudgian biohpc_admin 18K Oct 22 2014 bowtie2 Could the .tar.gz be fixed, so that the executables will work in a run as real user configuration? Is there any sensible way Galaxy could enforce permissions to guard against this issue? Cheers, -- David Trudgian Ph.D. Computational Scientist, BioHPC UT Southwestern Medical Center Dallas, TX 75390-9039 Tel: (214) 648-4833 Please contact biohpc-help@utsouthwestern with general BioHPC inquries. UT Southwestern Medical Center The future of medicine, today. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Adding packages for bedtools and ucsc-utils for Trackster?
On 06/18/2015 09:06 AM, Peter van Heusden wrote: Hi there I'm trying to visualise some of our genome data using Trackster. Initial input is 790,000 lines of GFF3. When I try and create the visualisation, I get errors from two tools: Convert GFF to Interval Index on data 3 -- an out of memory error - I'll try and fix this by upping the memory limit for our Galaxy jobs Convert BED, GFF, or VCF to BigWig on data 3 -- bedtools and bedGraphToBigWig not found errors. I tried to resolve these by installing the ucsc-tools and bedtools packages - and they did install correctly - but I still get these file not found errors. Hi Peter make sure 'galaxy' (ie the user galaxy is running as) has the newly installed tools in its path. You might need to add them to run.sh and restart the server. Hans-Rudolf Any idea how to fix this second problem? Thanks! Peter P.S. as an alternative to Trackster, does anyone have Galaxy interfaced to JBrowse? ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Adding packages for bedtools and ucsc-utils for Trackster?
Peter, I'm in a hurry ... have a look at https://github.com/galaxyproject/tools-iuc/tree/master/tools/jbrowse Developed by IUC member Eric Rasche. Cheers, Bjoern On 18.06.2015 10:00, Peter van Heusden wrote: I've tried that - I've added the Environment Modules for the relevant tools to the local_env.sh file, so they're there in the environment when Galaxy runs. These tools are scheduled via our cluster, however, so I don't think the environment passes across to them. They depend on these Galaxy packages, however, and those (bedtools and ucsc-utils) are now installed, so I'm not sure why it can't pick them up from the shed_tools. On 18 June 2015 at 09:10, Hans-Rudolf Hotz h...@fmi.ch mailto:h...@fmi.ch wrote: On 06/18/2015 09:06 AM, Peter van Heusden wrote: Hi there I'm trying to visualise some of our genome data using Trackster. Initial input is 790,000 lines of GFF3. When I try and create the visualisation, I get errors from two tools: Convert GFF to Interval Index on data 3 -- an out of memory error - I'll try and fix this by upping the memory limit for our Galaxy jobs Convert BED, GFF, or VCF to BigWig on data 3 -- bedtools and bedGraphToBigWig not found errors. I tried to resolve these by installing the ucsc-tools and bedtools packages - and they did install correctly - but I still get these file not found errors. Hi Peter make sure 'galaxy' (ie the user galaxy is running as) has the newly installed tools in its path. You might need to add them to run.sh and restart the server. Hans-Rudolf Any idea how to fix this second problem? Thanks! Peter P.S. as an alternative to Trackster, does anyone have Galaxy interfaced to JBrowse? ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Adding packages for bedtools and ucsc-utils for Trackster?
Thanks Bjoern My initial install didn't pick up the Perl dependencies correctly, but I re-installed and now the install seems to have the required modules but is missing something in the environment variables. Specifically I get an error import JSON.pm, see below. Note that the jbrowse/1.11.6/iuc/package_jbrowse_1_11_6/d5ab749c9e97/lib/perl5 directory isn't included in @INC and in fact the paths that *are* included seem to imply that the wrong perl interpreter is being picked up. Fatal error: Exit code 1 () Can't locate JSON.pm in @INC (you may need to install the JSON module) (@INC contains: /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/database/job_working_directory/000/29/dataset_29_files/JBrowse-1.11.6/bin/../src/perl5 /etc/perl /usr/local/lib/perl/5.18.2 /usr/local/share/perl/5.18.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.18 /usr/share/perl/5.18 /usr/local/lib/site_perl .) at /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/database/job_working_directory/000/29/dataset_29_files/JBrowse-1.11.6/bin/../src/perl5/Bio/JBrowse/JSON.pm line 10. BEGIN failed--compilation aborted at /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/database/job_working_directory/000/29/dataset_29_files/JBrowse-1.11.6/bin/../src/perl5/Bio/JBrowse/JSON.pm line 10. Compilation failed in require at /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/database/job_working_directory/000/29/dataset_29_files/JBrowse-1.11.6/bin/../src/perl5/Bio/JBrowse/Cmd/FormatSequences.pm line 21. BEGIN failed--compilation aborted at /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/database/job_working_directory/000/29/dataset_29_files/JBrowse-1.11.6/bin/../src/perl5/Bio/JBrowse/Cmd/FormatSequences.pm line 21. Compilation failed in require at bin/prepare-refseqs.pl line 7. BEGIN failed--compilation aborted at bin/prepare-refseqs.pl line 7. Traceback (most recent call last): File /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/jbrowse/2c9e5136b416/jbrowse/jbrowse.py, line 44, in module process_genome(jbrowse_dir, os.path.realpath(args.genome.name)) File /net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/jbrowse/2c9e5136b416/jbrowse/jbrowse.py, line 17, in process_genome subprocess.check_output(['perl', 'bin/prepare-refseqs.pl', '--fasta', genome], cwd=jbrowse_dir) File /usr/lib/python2.7/subprocess.py, line 573, in check_output raise CalledProcessError(retcode, cmd, output=output) subprocess.CalledProcessError: Command '['perl', 'bin/prepare-refseqs.pl', '--fasta', '/net/datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/database/files/000/dataset_3.dat']' returned non-zero exit status 2 On 18 June 2015 at 10:01, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Peter, I'm in a hurry ... have a look at https://github.com/galaxyproject/tools-iuc/tree/master/tools/jbrowse Developed by IUC member Eric Rasche. Cheers, Bjoern On 18.06.2015 10:00, Peter van Heusden wrote: I've tried that - I've added the Environment Modules for the relevant tools to the local_env.sh file, so they're there in the environment when Galaxy runs. These tools are scheduled via our cluster, however, so I don't think the environment passes across to them. They depend on these Galaxy packages, however, and those (bedtools and ucsc-utils) are now installed, so I'm not sure why it can't pick them up from the shed_tools. On 18 June 2015 at 09:10, Hans-Rudolf Hotz h...@fmi.ch wrote: On 06/18/2015 09:06 AM, Peter van Heusden wrote: Hi there I'm trying to visualise some of our genome data using Trackster. Initial input is 790,000 lines of GFF3. When I try and create the visualisation, I get errors from two tools: Convert GFF to Interval Index on data 3 -- an out of memory error - I'll try and fix this by upping the memory limit for our Galaxy jobs Convert BED, GFF, or VCF to BigWig on data 3 -- bedtools and bedGraphToBigWig not found errors. I tried to resolve these by installing the ucsc-tools and bedtools packages - and they did install correctly - but I still get these file not found errors. Hi Peter make sure 'galaxy' (ie the user galaxy is running as) has the newly installed tools in its path. You might need to add them to run.sh and restart the server. Hans-Rudolf Any idea how to fix this second problem? Thanks! Peter P.S. as an alternative to Trackster, does anyone have Galaxy interfaced to JBrowse? ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
