Re: [galaxy-dev] let galaxy use more hardware resouses
If you did what the links posted previously suggested (inc. parallelization of tools, mutliple server processes), there is nothing more you can do. Most of the tools galaxy just calls, so its the tool that you have to change. Often genomic tools are also IO bound, so a faster storage system could lead to better CPU utilization - if this is the bottleneck. HTH, ido On Dec 10, 2012, at 9:51 AM, 泽 蔡 wrote: Hi all, I run a local instance of galaxy. Today, I take a look at the system monitor, I found the galaxy just use 4% of a CPU and a little bit of the memory of the PC. But the PC has twelve CPUs and 126G memory. So, how to configure the galaxy to use more hardware resouces to calculate quickly? ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Toolshed confusion
Hi guys, I have my fresh galaxy install on our cluster and now I'm trying to figure out which tools are missing and how to install them. I just came across the toolshed so now I'm really confused. Question 1: Can I install EVERY tool I need from the tool shed or are there some things I need to install manually first like R, Samtools, etc? How do I know what to install from where? Question 2: Is it preferable to use the toolshed whenever possible, or it is not fully developed yet? Thanks, Greg ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Toolshed confusion
Hi Greg, In case you haven't found it, the tool shed wiki provides a lot of information. It's available at http://wiki.galaxyproject.org/Tool%20Shed On Dec 10, 2012, at 10:59 AM, greg wrote: Hi guys, I have my fresh galaxy install on our cluster and now I'm trying to figure out which tools are missing and how to install them. I just came across the toolshed so now I'm really confused. Question 1: Can I install EVERY tool I need from the tool shed or are there some things I need to install manually first like R, Samtools, etc? How do I know what to install from where? The intent of the tool shed is to share Galaxy utilities ( tools, exported workflows, proprietary datatypes, etc) developed by many contributors in the Galaxy community. The tool shed provides a very rich set of features that enhance those provided by Galaxy itself. In this way it complements galaxy. For example, the tool shed provides features that allow tool developers to include information within their tool shed repository to automatically install ( and possibly compile ) tool dependencies for tools within the repository. However, it is up to the repository owner (the community contributor ) to use tool shed features like this. So some repositories include tools that will install dependencies, while others require manual dependency installation. The core Galaxy development team is in the process of migrating tools from the Galaxy distribution to the main Galaxy tool shed. Tools that have been migrated in this way are all contained in repositories with the owner devteam. If these repositories include tools, then they will have dependencies defined for them, so the dependencies will be installed along with the repositories. Question 2: Is it preferable to use the toolshed whenever possible, or it is not fully developed yet? It is preferable to use the tool shed whenever possible, but additional features will continue to be added. This does not imply that you have to wait for additional features before using the tool shed. Thanks, Greg ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ Greg Von Kuster ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] How to modify save icon
Hello, The tool SAM to BAM generates a dataset from which it is possible to download two files (instead of one) from the icon save. As my script generates an output result and another txt output containing information about linux command, I will wish not to generate two datasets in history but only one dataset with two files downloaded from the icon save. Do you know how to encode the corresponding xml? Moreover, I would like to add file formats of file which can be opened in the central window central of Galaxy (thanks to eye icon). Do you know, please, how to add more files formats ? Thanks in advance, Sarah Mom ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] postgres connection
That worked perfectly! Thanks Nate! Chris Galaxy dev: I'm installing a local galaxy instance (Dec 4, 2012 vintage) and it is running well under mySQL, but I'm having trouble getting it to work with a postgres database. I've done the following to create the db and provide permissions: $ whoami galaxy $ createdb galaxy_prod4 $ psql galaxy_prod4 psql (8.4.13) Type help for help. galaxy_prod4=# CREATE ROLE galaxy_prod_role4 WITH PASSWORD 'milkweed'; CREATE ROLE galaxy_prod4=# GRANT ALL PRIVILEGES ON DATABASE galaxy_prod4 to galaxy_prod_role4; GRANT galaxy_prod4=# ALTER ROLE galaxy_prod_role4 LOGIN; ALTER ROLE galaxy_prod4=# \du List of roles Role name | Attributes | Member of ---+--+--- galaxy| Superuser| {} : Create role : Create DB galaxy_prod_role | Cannot login | {} galaxy_prod_role4 | | {} mydb_role | Cannot login | {} mydb_user | | {} postgres | Superuser| {} : Create role : Create DB galaxy_prod4=# \l List of databases Name | Owner | Encoding | Collation |Ctype| Access privileges --+--+--+-+-+-- galaxy_prod3 | galaxy | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =Tc/galaxy : galaxy=CTc/galaxy : galaxy_prod_role=CTc/galaxy galaxy_prod4 | galaxy | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =Tc/galaxy : galaxy=CTc/galaxy : galaxy_prod_role4=CTc/galaxy postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | template0| postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres : postgres=CTc/postgres template1| postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres : postgres=CTc/postgres (5 rows) Looks promising, right? My universe_wsgi.ini has the following entry: database_connection = postgres:///galaxy_prod_role4:milkweed@127.0.0.1:5432/galaxy_prod4 Hi Chris, You're a field off in your URL, try this: postgres://galaxy_prod_role4:milkweed@127.0.0.1:5432/galaxy_prod4 --nate and started the application, but it failed with: OperationalError: (OperationalError) FATAL: database galaxy_prod_role4:milkweed@127.0.0.1:5432/galaxy_prod4 does not exist commenting the database_connection line out allows galaxy to start, using mySQL I presume. Any ideas? This is my first time configuring something like this. Thank you! Chris ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Fix for SelectToolParameter rerun, workflow when multiple=true
Fix for SelectToolParameter rerun,workflow when multiple=true The following change allows multiple select values to be set on rerun and in workflows. I checked that DrillDownSelectToolParameter and ColumnListParameter stiil worked. $ hg diff lib/galaxy/tools/parameters/basic.py diff -r 0042b30216fc lib/galaxy/tools/parameters/basic.py --- a/lib/galaxy/tools/parameters/basic.py Tue Nov 06 11:14:22 2012 -0800 +++ b/lib/galaxy/tools/parameters/basic.py Mon Dec 10 11:03:10 2012 -0600 @@ -715,6 +715,12 @@ if value not in legal_values: raise ValueError( An invalid option was selected, please verify ) return value +def to_html_value( self, value, app ): +if isinstance( value, list ): +return value +else: +return str( value ) + def to_param_dict_string( self, value, other_values={} ): if value is None: return None Need to override method to_html_value() from ToolParameter otherwise a list value will be returned as a string: ['a','b'] instead of a list: ['a','b'] and options will not be selected in method get_html_field() : def get_html_field( self, trans=None, value=None, context={} ): # Dynamic options are not yet supported in workflow, allow # specifying the value as text for now. if self.need_late_validation( trans, context ): if value is not None: assert isinstance( value, UnvalidatedValue ), Late validation needed for '%s', but provided value (%s) is not of type UnvalidatedValue (%s). % ( self.name, value, type( value ) ) value = value.value if self.multiple: if value is None: value = else: value = \n.join( value ) return form_builder.TextArea( self.name, value=value ) else: return form_builder.TextField( self.name, value=(value or ) ) if value is not None: if not isinstance( value, list ): value = [ value ] ### ['a','b'] is not a list so value set to [['a','b']] field = form_builder.SelectField( self.name, self.multiple, self.display, self.refresh_on_change, refresh_on_change_values = self.refresh_on_change_values ) options = self.get_options( trans, context ) for text, optval, selected in options: if isinstance( optval, UnvalidatedValue ): optval = optval.value text = %s (unvalidated) % text if value: selected = ( optval in value )### 'a' will not be found in: [['a','b']] field.add_option( text, optval, selected ) return field diff -r 0042b30216fc lib/galaxy/tools/parameters/basic.py --- a/lib/galaxy/tools/parameters/basic.py Tue Nov 06 11:14:22 2012 -0800 +++ b/lib/galaxy/tools/parameters/basic.py Mon Dec 10 11:26:41 2012 -0600 @@ -715,6 +715,12 @@ if value not in legal_values: raise ValueError( An invalid option was selected, please verify ) return value +def to_html_value( self, value, app ): +if isinstance( value, list ): +return value +else: +return str( value ) + def to_param_dict_string( self, value, other_values={} ): if value is None: return None ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Toolshed confusion
Thanks, but I think now I'm even more confused :-( It seems like a whole lot of the tools that are listed on the left panel after installing galaxy aren't actually installed. For example FastQC. Then I do see FastQC in the tool shed, but it seems like when I go to install it, it will create its own entry on the left panel and not fix the FastQC tool that's already there. So if I install everything I can from the tool shed I'll still be left with a whole lot of broken tools in the left panel? (I guess the tool shed is just for new tools, and not to fix the ones already listed in Galaxy. Would it make sense to stop including tool links on the left panel that aren't bundled with Galaxy, and then let the toolshed add them in?) -Greg On Mon, Dec 10, 2012 at 11:16 AM, Greg Von Kuster g...@bx.psu.edu wrote: Hi Greg, In case you haven't found it, the tool shed wiki provides a lot of information. It's available at http://wiki.galaxyproject.org/Tool%20Shed On Dec 10, 2012, at 10:59 AM, greg wrote: Hi guys, I have my fresh galaxy install on our cluster and now I'm trying to figure out which tools are missing and how to install them. I just came across the toolshed so now I'm really confused. Question 1: Can I install EVERY tool I need from the tool shed or are there some things I need to install manually first like R, Samtools, etc? How do I know what to install from where? The intent of the tool shed is to share Galaxy utilities ( tools, exported workflows, proprietary datatypes, etc) developed by many contributors in the Galaxy community. The tool shed provides a very rich set of features that enhance those provided by Galaxy itself. In this way it complements galaxy. For example, the tool shed provides features that allow tool developers to include information within their tool shed repository to automatically install ( and possibly compile ) tool dependencies for tools within the repository. However, it is up to the repository owner (the community contributor ) to use tool shed features like this. So some repositories include tools that will install dependencies, while others require manual dependency installation. The core Galaxy development team is in the process of migrating tools from the Galaxy distribution to the main Galaxy tool shed. Tools that have been migrated in this way are all contained in repositories with the owner devteam. If these repositories include tools, then they will have dependencies defined for them, so the dependencies will be installed along with the repositories. Question 2: Is it preferable to use the toolshed whenever possible, or it is not fully developed yet? It is preferable to use the tool shed whenever possible, but additional features will continue to be added. This does not imply that you have to wait for additional features before using the tool shed. Thanks, Greg ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ Greg Von Kuster ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Updating help text for a tool shed repository
I recently updated the htseq-count tool in the Galaxy toolshed to version 0.3. This version has some functional changes and thus the tool version was incremented. I installed this new version of the tool in my local Galaxy instance. However, I realized later that I missed updating one part of the help text. Today I fixed that text (but left the tool version the same), tarred up the file, and uploaded it to the toolshed. I then attempted to Get Updates from my local Galaxy instances for that tool got the following error: URL: http://galaxy-dev.princeton.edu/admin_toolshed/update_to_changeset_revision?tool_shed_url=http://toolshed.g2.bx.psu.edu/name=htseq_countowner=lparsonschangeset_revision=5d969cb56112latest_changeset_revision=f320093f1e8elatest_ctx_rev=11 File '/data/galaxy-dev/galaxy-dev/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/data/galaxy-dev/galaxy-dev/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/data/galaxy-dev/galaxy-dev/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/data/galaxy-dev/galaxy-dev/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/data/galaxy-dev/galaxy-dev/lib/galaxy/web/framework/middleware/remoteuser.py', line 91 in __call__ return self.app( environ, start_response ) File '/data/galaxy-dev/galaxy-dev/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/data/galaxy-dev/galaxy-dev/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/data/galaxy-dev/galaxy-dev/lib/galaxy/web/framework/__init__.py', line 206 in decorator return func( self, trans, *args, **kwargs ) File '/data/galaxy-dev/galaxy-dev/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 1677 in update_to_changeset_revision update_repository( repo, latest_ctx_rev ) File '/data/galaxy-dev/galaxy-dev/lib/galaxy/util/shed_util_common.py', line 1428 in update_repository rev=ctx_rev ) File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/commands.py', line 5661 in update rev = scmutil.revsingle(repo, rev, rev).rev() File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/scmutil.py', line 515 in revsingle l = revrange(repo, [revspec]) File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/scmutil.py', line 588 in revrange for r in m(repo, range(len(repo))): File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/revset.py', line 1414 in mfunc return getset(repo, subset, tree) File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/revset.py', line 156 in getset return methods[x[0]](repo, subset, *x[1:]) File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/revset.py', line 171 in symbolset return stringset(repo, subset, x) File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/revset.py', line 161 in stringset x = repo[x].rev() File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/localrepo.py', line 235 in __getitem__ return context.changectx(self, changeid) File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/context.py', line 103 in __init__ _(unknown revision '%s') % changeid) RepoLookupError: unknown revision '11' I believe this is because the revision I have installed (10) is not longer considered a valid revision by the toolshed (I presume since it is an older revision of with the same tool version). Any help would be appreciated. Thanks. -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Toolshed confusion
On Dec 10, 2012, at 1:14 PM, greg wrote: Thanks, but I think now I'm even more confused :-( Hmm...sorry. I highly advise you to read the tool shed wiki as it will clear up a lot of your confusion. It seems like a whole lot of the tools that are listed on the left panel after installing galaxy aren't actually installed. I'm not sure to what tools you are referring here, but some Galaxy tools consist of 2 components ( a UI based on Cheetah, and a script that has no requirements ) while others have a 3rd component ( a 3rd-party dependency ). When you clone the Galaxy distribution, those tools that require dependencies must have those dependencies manually installed and defined on a path in such a way that the Galaxy user ( the account used to start the Galaxy server ) can find them. This has always been the case, and has nothing to do with the tool shed. However, the tool shed improves this process for those repositories that you install into your Galaxy instance that include tools and defined dependencies for them. For example FastQC. Then I do see FastQC in the tool shed, but it seems like when I go to install it, it will create its own entry on the left panel and not fix the FastQC tool that's already there. These are 2 different tools, although they are named similarly. You'll notice, though, that they have different IDs. So installing a tool from the tool shed will not fix or replace an existing tool with the same name in a Galaxy instance into which it was installed. So if I install everything I can from the tool shed I'll still be left with a whole lot of broken tools in the left panel? Possibly - again, it is the responsibility of the repository owner to ensure the tools are functionally correct when installed into Galaxy. A commission is in place to review and approve the contents of repositories on the main Galaxy tool shed, but it has not yet undertaken this endeavor. (I guess the tool shed is just for new tools, and not to fix the ones already listed in Galaxy. Any tools included in the Galaxy distribution does not need fixing. At most they need a dependency to be manually installed. See the tool dependencies page in the Galaxy wiki for details. It's available here: http://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies Would it make sense to stop including tool links on the left panel that aren't bundled with Galaxy, and then let the toolshed add them in?) Not sure what you mean here. -Greg On Mon, Dec 10, 2012 at 11:16 AM, Greg Von Kuster g...@bx.psu.edu wrote: Hi Greg, In case you haven't found it, the tool shed wiki provides a lot of information. It's available at http://wiki.galaxyproject.org/Tool%20Shed On Dec 10, 2012, at 10:59 AM, greg wrote: Hi guys, I have my fresh galaxy install on our cluster and now I'm trying to figure out which tools are missing and how to install them. I just came across the toolshed so now I'm really confused. Question 1: Can I install EVERY tool I need from the tool shed or are there some things I need to install manually first like R, Samtools, etc? How do I know what to install from where? The intent of the tool shed is to share Galaxy utilities ( tools, exported workflows, proprietary datatypes, etc) developed by many contributors in the Galaxy community. The tool shed provides a very rich set of features that enhance those provided by Galaxy itself. In this way it complements galaxy. For example, the tool shed provides features that allow tool developers to include information within their tool shed repository to automatically install ( and possibly compile ) tool dependencies for tools within the repository. However, it is up to the repository owner (the community contributor ) to use tool shed features like this. So some repositories include tools that will install dependencies, while others require manual dependency installation. The core Galaxy development team is in the process of migrating tools from the Galaxy distribution to the main Galaxy tool shed. Tools that have been migrated in this way are all contained in repositories with the owner devteam. If these repositories include tools, then they will have dependencies defined for them, so the dependencies will be installed along with the repositories. Question 2: Is it preferable to use the toolshed whenever possible, or it is not fully developed yet? It is preferable to use the tool shed whenever possible, but additional features will continue to be added. This does not imply that you have to wait for additional features before using the tool shed. Thanks, Greg ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Re: [galaxy-dev] Trying to Run MegaBlast
Please see the Megablast section on this page for instructions about getting and adding target databases to Galaxy: http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup Best, J. On Dec 10, 2012, at 2:59 PM, greg wrote: I have a fresh install on our cluster. I'm trying to test out the Megablast tool. The problem is that nothing is listed under against target database. Where do I get the target databases and how do I tell galaxy about them? Thanks, Greg Here's what the tool looks like: Megablast (version 1.2.0) Compare these sequences: against target database: using word size: Size of best perfect match (-word_size) report hits above this identity (-perc_identity): no cutoff if 0 set expectation value cutoff (-evalue): Filter out low complexity regions? (-dust): ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] add new slurm-drmaa runner to galaxy-central
On Sat, Jul 21, 2012 at 6:12 AM, Di Pe dip...@gmail.com wrote: may be some other people would find this a useful enhancement: https://bitbucket.org/galaxy/galaxy-central/issue/778/add-new-slurm-drmaa-runner-to-galaxy Hi Di, I've been testing several grid scheduling software trying to find the one best fits my group needs. I'm testing slurm now but quickly ran into this issue: galaxy.jobs.runners.slurm ERROR 2012-12-10 15:30:12,006 Uncaught exception queueing job Traceback (most recent call last): File /local/opt/galaxy/galaxy-dist.slurm/lib/galaxy/jobs/runners/slurm.py, line 165, in run_next self.queue_job( obj ) File /local/opt/galaxy/galaxy-dist.slurm/lib/galaxy/jobs/runners/slurm.py, line 286, in queue_job job_id = self.ds.runJob(jt) File /local/opt/galaxy/env.python2.7.precise/lib/python2.7/site-packages/drmaa/__init__.py, line 331, in runJob _h.c(_w.drmaa_run_job, jid, _ct.sizeof(jid), jobTemplate) File /local/opt/galaxy/env.python2.7.precise/lib/python2.7/site-packages/drmaa/helpers.py, line 213, in c return f(*(args + (error_buffer, sizeof(error_buffer File /local/opt/galaxy/env.python2.7.precise/lib/python2.7/site-packages/drmaa/errors.py, line 90, in error_check raise _ERRORS[code-1](code %s: %s % (code, error_buffer.value)) InvalidAttributeFormatException: code 13: invalid date/time format: As you can see I made sure I was using the latest version of python drmaa module by installing in a virtualenv. Not only in this error but by modifying slurm.py so it looks in the right place, I can see the virtualenv drmaa code is been used instead of Galaxy provided egg: galaxy.jobs.runners.slurm DEBUG 2012-12-10 15:31:12,410 using local python-drmaa from path /local/opt/galaxy/env.python2.7.precise/lib/python2.7/site-packages ** galaxy.jobs.runners.slurm DEBUG 2012-12-10 15:31:12,420 * DRMAA IMPORTED galaxy.jobs.runners.slurm DEBUG 2012-12-10 15:31:12,425 Supported DRM systems: SLURM 2.5.0 galaxy.jobs.runners.slurm DEBUG 2012-12-10 15:31:12,425 Supported DRMAA implementations: PSNC DRMAA for SLURM 1.0.6 galaxy.jobs.runners.slurm DEBUG 2012-12-10 15:31:12,425 Version 1.0 galaxy.jobs.runners.slurm DEBUG 2012-12-10 15:31:12,425 * DRMAA INITIALIZED galaxy.jobs.runners.slurm DEBUG 2012-12-10 15:31:12,426 3 workers ready galaxy.jobs.handler DEBUG 2012-12-10 15:31:12,426 Loaded job runner: galaxy.jobs.runners.slurm:DRMAAJobRunner galaxy.jobs.handler DEBUG 2012-12-10 15:31:12,426 Job runners: local:lwr:slurm Could you please help me figure out this issue? I'm using latest galaxy-dist and the most updated version from https://github.com/FredHutch/IT/tree/master/galaxy/slurm Thanks, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Trying to Run MegaBlast
So I just want to install one target database so that I can simply test whether blast is installed and working correctly. Are there any included with Galaxy, or do I need to download a database? Could you guys point me to a good file to test with? Following the directions you sent I got to the point of downloading a file. I tried download ftp://ftp.ncbi.nlm.nih.gov/blast/db/nr.00.tar.gz but I don't see a *.chunk file nor *.nhr, nin, nsq etc that are referenced in the included blastdb.loc Thanks, Greg On Mon, Dec 10, 2012 at 3:35 PM, Jeremy Goecks jeremy.goe...@emory.edu wrote: Please see the Megablast section on this page for instructions about getting and adding target databases to Galaxy: http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup Best, J. On Dec 10, 2012, at 2:59 PM, greg wrote: I have a fresh install on our cluster. I'm trying to test out the Megablast tool. The problem is that nothing is listed under against target database. Where do I get the target databases and how do I tell galaxy about them? Thanks, Greg Here's what the tool looks like: Megablast (version 1.2.0) Compare these sequences: against target database: using word size: Size of best perfect match (-word_size) report hits above this identity (-perc_identity): no cutoff if 0 set expectation value cutoff (-evalue): Filter out low complexity regions? (-dust): ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Exporting Trackster visualizations.
Hi, Is there a way to export visualizations from Trackster and Sweepster in .pdf or .jpg format? I have been using these fantastic tools to fine tune my parameters and would like to show how I did so in figure format (partially to locally get the word out about these powerful tools). Screenshots often do not suffice (and don't do the tools justice). Any solutions and/or fixes are appreciated! Cheers, Mo Heydarian PhD candidate The Johns Hopkins School of Medicine Department of Biological Chemistry 725 Wolfe Street 414 Hunterian Baltimore, MD 21205 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] About the output
I made a galaxy tool which is just create a txt file. When I use this tool , I wish to let it show the detail of the txt. In the xml file I use : output data name=test format=txtoutput/ But it does not work, when I click the eye button in galaxy, a download dialog appear, but it is just a empty file. Could somebody tell me what should I do , I want show the txt detail in the galaxy. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Trying to Run MegaBlast
Are there any included with Galaxy, or do I need to download a database? You need to download them. Could you guys point me to a good file to test with? Following the directions you sent I got to the point of downloading a file. I tried download ftp://ftp.ncbi.nlm.nih.gov/blast/db/nr.00.tar.gz but I don't see a *.chunk file nor *.nhr, nin, nsq etc that are referenced in the included blastdb.loc Decompress and unpack the file and you'll find the relevant files (though everything now has extensions with p* rather than n*); add an entry to blastdb pointing to the directory with these files and you should be good to go. J. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Exporting Trackster visualizations.
Unfortunately, no solutions right now; currently, screenshots are best that can be done. One trick we often use for publications/posters is zooming, taking multiple screenshots, and then composing them together; ugly, but pretty effective for showing details. We have some ideas about how to improve things on this front but haven't had time to try them out. I've created a Trello card for this request so you can follow it: https://trello.com/c/4NPfYF6F Best, J. On Dec 10, 2012, at 3:57 PM, Mohammad Heydarian wrote: Hi, Is there a way to export visualizations from Trackster and Sweepster in .pdf or .jpg format? I have been using these fantastic tools to fine tune my parameters and would like to show how I did so in figure format (partially to locally get the word out about these powerful tools). Screenshots often do not suffice (and don't do the tools justice). Any solutions and/or fixes are appreciated! Cheers, Mo Heydarian PhD candidate The Johns Hopkins School of Medicine Department of Biological Chemistry 725 Wolfe Street 414 Hunterian Baltimore, MD 21205 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Installation issue on EC2
On Dec 10, 2012, at 6:17 PM, Fabiano Lucchese fabiano.lucch...@hds.com wrote: I appreciate your effort to help me, but it looks like my AWS account has some serious hidden issues going on. I completely wiped out CloudMan/Galaxy instances from my EC2 environment as well as their volumes, and waited a couple of hours for the instances to disappear from the instances list. After that, I repeated the whole process twice, trying to create a Galaxy cluster with 10 and then 15 Gb of storage space, but the result was equally frustrating with some minor differences. PS: Dannon, one thing that intrigues me is how the web form manages to find out the names of the previous clusters that I tried to instantiate. Where does it get this information from if all the respective instances have been terminated and wiped out? Did you reuse the same cluster name for either of these? This would explain conflicting settings - there's more to a cluster than just the running instances and persistent volumes. That form retrieves those listings from the S3 buckets in your account. Each cluster has its own S3 bucket -- you can identify them with the yourCluster.clusterName files in the listing. These contain lots of information about your galaxy clusters (references to volumes, universe settings, etc.), and if you're attempting to eliminate a cluster completely (you never want to restart it and don't want *anything* preserved), you should delete the buckets referring to them as well. When you ask Cloudman to terminate and remove a cluster permanently, it removes all of this for you, and I'd recommend always using the interface to do this and not doing it manually. Definitely let me know what else I can do to help. If you have a running instance you'd like for me to look at directly I'd be happy to do so -- maybe this is indeed some weird issue that we can work around better. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Upload files from filesystem paths...
Hi, I have a local galaxy installation. I've created a data library, selected Upload files from filesystem paths, pasted a path in the path to upload window, and I've selected to preserve the directory structure. And the files get imported. How do I now access these files from my application? I don't want to import them into the history as then they lose the directory structure. I can't see where they are physically under the galaxy-dist structure Thanks for any help Neil ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Installation issue on EC2
Guys, I appreciate your effort to help me, but it looks like my AWS account has some serious hidden issues going on. I completely wiped out CloudMan/Galaxy instances from my EC2 environment as well as their volumes, and waited a couple of hours for the instances to disappear from the instances list. After that, I repeated the whole process twice, trying to create a Galaxy cluster with 10 and then 15 Gb of storage space, but the result was equally frustrating with some minor differences. In the less unsuccessful scenario, the Galaxy service was the only one down, apparently for the following reason: Traceback (most recent call last): File /mnt/galaxyTools/galaxy-central/lib/galaxy/webapps/galaxy/buildapp.py, line 36, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File /mnt/galaxyTools/galaxy-central/lib/galaxy/app.py, line 85, in __init__ from_shed_config=True ) File /mnt/galaxyTools/galaxy-central/lib/galaxy/tools/data/__init__.py, line 41, in load_from_config_file tree = util.parse_xml( config_filename ) File /mnt/galaxyTools/galaxy-central/lib/galaxy/util/__init__.py, line 143, in parse_xml tree = ElementTree.parse(fname) File /mnt/galaxyTools/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py, line 859, in parse tree.parse(source, parser) File /mnt/galaxyTools/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py, line 576, in parse source = open(source, rb) IOError: [Errno 2] No such file or directory: './shed_tool_data_table_conf.xml' Removing PID file paster.pid I restarted it a few times, rebooted the machine and even tried to update it, but nothing could magically fix the problem. I'm giving up the high-level approach and starting from a fresh installation of Galaxy in one of my instances. It's going to be less productive, but at least I have some control over what's going on and can try to diagnose problems as they occur. Cheers, F. PS: Dannon, one thing that intrigues me is how the web form manages to find out the names of the previous clusters that I tried to instantiate. Where does it get this information from if all the respective instances have been terminated and wiped out? ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Installation issue on EC2
Did you delete the S3/shapshots buckets? That is where everything is stored and I think EBS volumes related to everything dynamic are created. On 12/10/12 6:17 PM, Fabiano Lucchese fabiano.lucch...@hds.com wrote: Guys, I appreciate your effort to help me, but it looks like my AWS account has some serious hidden issues going on. I completely wiped out CloudMan/Galaxy instances from my EC2 environment as well as their volumes, and waited a couple of hours for the instances to disappear from the instances list. After that, I repeated the whole process twice, trying to create a Galaxy cluster with 10 and then 15 Gb of storage space, but the result was equally frustrating with some minor differences. In the less unsuccessful scenario, the Galaxy service was the only one down, apparently for the following reason: Traceback (most recent call last): File /mnt/galaxyTools/galaxy-central/lib/galaxy/webapps/galaxy/buildapp.py, line 36, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File /mnt/galaxyTools/galaxy-central/lib/galaxy/app.py, line 85, in __init__ from_shed_config=True ) File /mnt/galaxyTools/galaxy-central/lib/galaxy/tools/data/__init__.py, line 41, in load_from_config_file tree = util.parse_xml( config_filename ) File /mnt/galaxyTools/galaxy-central/lib/galaxy/util/__init__.py, line 143, in parse_xml tree = ElementTree.parse(fname) File /mnt/galaxyTools/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg /elementtree/ElementTree.py, line 859, in parse tree.parse(source, parser) File /mnt/galaxyTools/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg /elementtree/ElementTree.py, line 576, in parse source = open(source, rb) IOError: [Errno 2] No such file or directory: './shed_tool_data_table_conf.xml' Removing PID file paster.pid I restarted it a few times, rebooted the machine and even tried to update it, but nothing could magically fix the problem. I'm giving up the high-level approach and starting from a fresh installation of Galaxy in one of my instances. It's going to be less productive, but at least I have some control over what's going on and can try to diagnose problems as they occur. Cheers, F. PS: Dannon, one thing that intrigues me is how the web form manages to find out the names of the previous clusters that I tried to instantiate. Where does it get this information from if all the respective instances have been terminated and wiped out? ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] About the output
I made a galaxy tool which is just create a txt file. When I use this tool , I wish to let it show the detail of the txt. In the xml file I use : output data name=test format=txtoutput/ But it does not work, when I click the eye button in galaxy, a download dialog appear, but it is just a empty file. Could somebody tell me what should I do , I want show the txt detail in the galaxy. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Galaxy doesn't detect the file when we add dataset to a library under Manage data libraries
Hello Everyone, We have found that, in Galaxy, when we add dataset to a library under Manage data libraries, Galaxy sometimes detects the type of the file and sometimes does not. What I have noticed is that it detects the type of the file when we select Copy file into Galaxy under Copy data into Galaxy? option and does not detect it when we select Link to files ... Is this a known issue? If it is, is there any workaround to solve the issue? Thanks Regards, Sachit ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Re-installing error persists
Hi Greg, list When installing a tool dependency, and the Installation Status shows : Error In what log would the actual error be specified?? Thanks, Franco On 5/12/12 9:10 PM, Greg Von Kuster g...@bx.psu.edu wrote: If you have not yet updated your Galaxy instance to the revision from the galaxy-dist repository in bitbucket, then that is most likely the cause of the behavior you are seeing. Greg Von Kuster On Dec 5, 2012, at 12:52 AM, Franco Caramia wrote: Re-installing error persists Hi Greg, I have now modified the tool_dependencies.xml thank you for that.. I still have problems re-installing the tool... (need to reinstall to update the tool dependency and test it correctly) Is the reinstalling failure related to my galaxy version?? Here is the error: Server Error URL: http://bioinf-galaxy/admin_toolshed/browse_repositories?operation=activate+or +reinstallid=88ee2d7021e4dc1a Module paste.exceptions.errormiddleware:143 in __call__ http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# app_iter = self.application(environ, start_response) Module paste.debug.prints:98 in __call__ http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# environ, self.app) Module paste.wsgilib:539 in intercept_output http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# app_iter = application(environ, replacement_start_response) Module paste.recursive:80 in __call__ http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# return self.application(environ, start_response) Module paste.httpexceptions:632 in __call__ http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# return self.application(environ, start_response) Module galaxy.web.framework.base:160 in __call__ http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# body = method( trans, **kwargs ) Module galaxy.web.framework:184 in decorator http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# return func( self, trans, *args, **kwargs ) Module galaxy.web.controllers.admin_toolshed:350 in browse_repositories http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# return self.reselect_tool_panel_section( trans, **kwd ) Module galaxy.web.framework:184 in decorator http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# return func( self, trans, *args, **kwargs ) Module galaxy.web.controllers.admin_toolshed:1352 in reselect_tool_panel_section http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# status=status ) Module galaxy.web.framework:746 in fill_template http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# return self.fill_template_mako( filename, **kwargs ) Module galaxy.web.framework:757 in fill_template_mako http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# return template.render( **data ) Module mako.template:296 in render http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# return runtime._render(self, self.callable_, args, data) Module mako.runtime:660 in _render http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# **_kwargs_for_callable(callable_, data)) Module mako.runtime:692 in _render_context http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) Module mako.runtime:718 in _exec_template http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# callable_(context, *args, **kwargs) Module _base_mako:42 in render_body http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# __M_writer(unicode(next.body())) Module _admin_tool_shed_repository_reselect_tool_panel_section_mako:76 in render_body http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=activ ate+or+reinstallid=88ee2d7021e4dc1a# __M_writer(unicode(render_tool_dependency_section( install_tool_dependencies_check_box, [ repo_info_dict ] ))) Module _admin_tool_shed_repository_common_mako:204 in render_render_tool_dependency_section
Re: [galaxy-dev] Re-installing error persists
Never mind, the INSTALLATION .log is in the repository browser... Sorry, Franco On 11/12/12 4:17 PM, Franco Caramia franco.cara...@petermac.org wrote: Hi Greg, list When installing a tool dependency, and the Installation Status shows : Error In what log would the actual error be specified?? Thanks, Franco On 5/12/12 9:10 PM, Greg Von Kuster g...@bx.psu.edu wrote: If you have not yet updated your Galaxy instance to the revision from the galaxy-dist repository in bitbucket, then that is most likely the cause of the behavior you are seeing. Greg Von Kuster On Dec 5, 2012, at 12:52 AM, Franco Caramia wrote: Re-installing error persists Hi Greg, I have now modified the tool_dependencies.xml thank you for that.. I still have problems re-installing the tool... (need to reinstall to update the tool dependency and test it correctly) Is the reinstalling failure related to my galaxy version?? Here is the error: Server Error URL: http://bioinf-galaxy/admin_toolshed/browse_repositories?operation=activate+o r+reinstallid=88ee2d7021e4dc1a Module paste.exceptions.errormiddleware:143 in __call__ http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# app_iter = self.application(environ, start_response) Module paste.debug.prints:98 in __call__ http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# environ, self.app) Module paste.wsgilib:539 in intercept_output http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# app_iter = application(environ, replacement_start_response) Module paste.recursive:80 in __call__ http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# return self.application(environ, start_response) Module paste.httpexceptions:632 in __call__ http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# return self.application(environ, start_response) Module galaxy.web.framework.base:160 in __call__ http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# body = method( trans, **kwargs ) Module galaxy.web.framework:184 in decorator http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# return func( self, trans, *args, **kwargs ) Module galaxy.web.controllers.admin_toolshed:350 in browse_repositories http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# return self.reselect_tool_panel_section( trans, **kwd ) Module galaxy.web.framework:184 in decorator http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# return func( self, trans, *args, **kwargs ) Module galaxy.web.controllers.admin_toolshed:1352 in reselect_tool_panel_section http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# status=status ) Module galaxy.web.framework:746 in fill_template http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# return self.fill_template_mako( filename, **kwargs ) Module galaxy.web.framework:757 in fill_template_mako http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# return template.render( **data ) Module mako.template:296 in render http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# return runtime._render(self, self.callable_, args, data) Module mako.runtime:660 in _render http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# **_kwargs_for_callable(callable_, data)) Module mako.runtime:692 in _render_context http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) Module mako.runtime:718 in _exec_template http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# callable_(context, *args, **kwargs) Module _base_mako:42 in render_body http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# __M_writer(unicode(next.body())) Module _admin_tool_shed_repository_reselect_tool_panel_section_mako:76 in render_body http://bioinf-galaxy:83/admin_toolshed/browse_repositories?operation=acti vate+or+reinstallid=88ee2d7021e4dc1a# __M_writer(unicode(render_tool_dependency_section( install_tool_dependencies_check_box,