Re: [galaxy-dev] Broken tools ids from ./run_functional_tests.sh -list
Peter, Thanks for reporting this issue, it looks like there might be a bug in tool_list.py, which I will be looking into. In the meantime, you should still be able to run functional tests for your tool by executing sh run_functional_tests.sh -id tool_id Where tool_id is the ID you have defined in the tool XML. The functional_tests.py script uses Galaxy's internal tool XML parser to generate the list of functional tests to run, so it should be able to find your tool and its tests. --Dave B. On 4/11/13 12:36:42.000, Peter Cock wrote: Hi all, Does anyone else see this problem with broken tool ids in the function test script? I found this because one of my own tools wasn't being recognised because for some reason the id was treated as _name= [sic]. This happens on the default branch: $ ./run_functional_tests.sh -list | grep _name= _name= RNA/DNA _name= Convert _name= Compute quality statistics _name= Draw quality score boxplot _name= DAVID $ hg branch default hg log -b default | head changeset: 9349:199b62339f26 user:jeremy goecks jeremy.goe...@emory.edu date:Thu Apr 11 08:44:42 2013 -0400 summary: Whitespace fixes for Vcf datatype. changeset: 9348:dbfc964167ae user:guerler date:Wed Apr 10 17:30:27 2013 -0400 summary: Enhance button for trackster visualization in data display viewer Have I found a recent regression, or is my system messed up somehow? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] (no subject)
David, While I look into the cause of this error, could you tell me which revision of Galaxy you're running? --Dave B. On 4/11/13 12:51:03.000, David Bonsall wrote: Dear Galaxy team, I'm trying to install NCBI + tools on a local instance of galaxy (rep: 8.1546099212f), but I get the following error, which I do not get when trying to install other tools. Thank you for all your hard work. It's all much appreciated David Error Traceback: View as: Interactive | Text | XML (full) ⇝ HTTPError: HTTP Error 500: Internal Server Error URL: http://127.0.0.1:8080/admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.g2.bx.psu.edu/repository_ids=1d92ebdf7e8d466cchangeset_revisions=1f546099212f Module weberror.evalexception.middleware:364 in respond view app_iter = self.application(environ, detect_start_response) Module paste.recursive:84 in __call__ view return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ view return self.application(environ, start_response) Module galaxy.web.framework.base:125 in __call__ view return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:182 in handle_request view body = method( trans, **kwargs ) Module galaxy.web.framework:228 in decorator view return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:894 in prepare_for_install view common_install_util.get_dependencies_for_repository( trans, tool_shed_url, repo_info_dict, includes_tool_dependencies ) Module tool_shed.util.common_install_util:90 in get_dependencies_for_repository view required_repo_info_dicts = get_required_repo_info_dicts( tool_shed_url, util.listify( repo_info_dict ) ) Module tool_shed.util.common_install_util:283 in get_required_repo_info_dicts view response = urllib2.urlopen( url ) Module urllib2:126 in urlopen view return _opener.open(url, data, timeout) Module urllib2:400 in open view response = meth(req, response) Module urllib2:513 in http_response view 'http', request, response, code, msg, hdrs) Module urllib2:438 in error view return self._call_chain(*args) Module urllib2:372 in _call_chain view result = func(*args) Module urllib2:521 in http_error_default view raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 500: Internal Server Error ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] TestToolShed out of date? (not running unit tests)
Peter, The automated testing framework is also running against the test tool shed. Could you send me a link to the repository that is behaving oddly? --Dave B. On 4/12/13 06:42:04.000, Peter Cock wrote: Hi all, I'd like to be able to experiment with the new automated testing recently added to the main Tool Shed http://toolshed.g2.bx.psu.edu/ by first trying my updates on the Test Tool Shed (to make sure my tests will work before making the tool update public). It appears that the testing features are not available on the Test Tool Shed, http://testtoolshed.g2.bx.psu.edu/ Is this correct? Is this intentional? Thanks, Peter P.S. This would be less urgent if I could solve this issue which is blocking my local testing: http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-April/014123.html ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] drmaa and JSV
Hi,
Our galaxy instance runs jobs in a SGE cluster using 2 job-handlers. The
SGE cluster uses a Job Submission Verifier (JSV) that rejects any job
submission that specify core
binding strategies.
When Galaxy starts, the first jobs we submit works perfectly:
First job submission:
galaxy.jobs.manager DEBUG 2013-04-15 14:29:59,285 (194) Job assigned to
handler 'handler0' galaxy.jobs DEBUG 2013-04-15 14:29:59,934 (194)
Working directory for job is:
/scratch/nfs/galaxy.crg.es/job_working_directory/000/194
galaxy.jobs.handler DEBUG 2013-04-15 14:29:59,942 dispatching job 194 to
drmaa runner
galaxy.jobs.handler INFO 2013-04-15 14:30:00,166 (194) Job dispatched
galaxy.jobs.runners.drmaa DEBUG 2013-04-15 14:30:00,468 (194) submitting
file /scratch/nfs/galaxy.crg.es/ogs/galaxy_194.sh
galaxy.jobs.runners.drmaa DEBUG 2013-04-15 14:30:00,468 (194) command
is: python
/data/www-bi/apache/galaxy.crg.es/htdocs/galaxy-dist/tools/fastq/fastq_stats.py
'/data/www-bi/galaxy.crg.es/files/000/dataset_4.dat'
'/data/www-bi/galaxy.crg.es/files/000/dataset_238.dat' 'sanger'
galaxy.jobs.runners.drmaa INFO 2013-04-15 14:30:01,538 (194) queued as
458816
galaxy.jobs.runners.drmaa DEBUG 2013-04-15 14:30:02,115 (194/458816)
state change: job is queued and active
# qstat -cb -j 458816
==
job_number: 458816
exec_file: job_scripts/458816
submission_time:Mon Apr 15 14:30:01 2013
owner: www-bi
uid:66401
group: www-bi
gid:501
sge_o_home: /data/www-bi
sge_o_log_name: www-bi
sge_o_path:
/data/galaxy/apache/galaxy.crg.es/htdocs/scripts/galaxy-env/bin:/software/galaxy/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/data/www-bi/bin
sge_o_shell:/bin/bash
sge_o_workdir:
/data/www-bi/apache/galaxy.crg.es/htdocs/galaxy-dist
sge_o_host: galaxy
account:sge
stderr_path_list:
NONE:galaxy:/scratch/nfs/galaxy.crg.es/job_working_directory/000/194/194.drmerr
reserve:y
hard resource_list: virtual_free=12G,h_rt=21600
mail_list: www...@galaxy.crg.es
notify: FALSE
job_name: g194_fastq_stats_jtaly_crg_es
stdout_path_list:
NONE:galaxy:/scratch/nfs/galaxy.crg.es/job_working_directory/000/194/194.drmout
jobshare: 0
hard_queue_list:www-el6
env_list:
script_file:/scratch/nfs/galaxy.crg.es/ogs/galaxy_194.sh
parallel environment: smp range: 2
verify_suitable_queues: 2
binding:set linear:2:0,0
scheduling info:queue instance pr-...@fenn.linux.crg.es
dropped because it is overloaded: np_load_avg=1.70 (= 1.70 +
0.50 * 0.00 with nproc=12) = 1.7
queue instance
sh...@node-ib0209bi.linux.crg.es dropped because it is overloaded:
np_load_avg=2.837500 (= 2.837500 + 0.50 * 0.00 with nproc=8) = 1.3
queue instance
l...@node-ib0209bi.linux.crg.es dropped because it is overloaded:
np_load_avg=2.837500 (= 2.837500 + 0.50 * 0.00 with nproc=8) = 1.3
The core binding has been added by our jsv script. This is correct.
But our second submission fails:
galaxy.jobs.runners.drmaa ERROR 2013-04-15 14:30:56,263 Uncaught
exception queueing job
Traceback (most recent call last):
File
/data/www-bi/apache/galaxy.crg.es/htdocs/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py,
line 144, in run_next
self.queue_job( obj )
File
/data/www-bi/apache/galaxy.crg.es/htdocs/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py,
line 232, in queue_job
job_id = self.ds.runJob(jt)
File
/data/www-bi/apache/galaxy.crg.es/htdocs/galaxy-dist/eggs/drmaa-0.4b3-py2.6.egg/drmaa/__init__.py,
line 331, in runJob
_h.c(_w.drmaa_run_job, jid, _ct.sizeof(jid), jobTemplate)
File
/data/www-bi/apache/galaxy.crg.es/htdocs/galaxy-dist/eggs/drmaa-0.4b3-py2.6.egg/drmaa/helpers.py,
line 213, in c
return f(*(args + (error_buffer, sizeof(error_buffer
File
/data/www-bi/apache/galaxy.crg.es/htdocs/galaxy-dist/eggs/drmaa-0.4b3-py2.6.egg/drmaa/errors.py,
line 90, in error_check
raise _ERRORS[code-1](code %s: %s % (code, error_buffer.value))
DeniedByDrmException: code 17: contact us: x...@xxx.es
if we look at the submited params:
# cat /tmp/qsub_err.txt
$VAR1 = {
'w' = 'e',
'N' = 'g195_fastq_stats_jtaly_crg_es',
'binding_amount' = '2',
'CMDNAME' = '/scratch/nfs/galaxy.crg.es/ogs/galaxy_195.sh',
'binding_type' = 'set',
'M' = {
'www...@galaxy.crg.es' = undef
},
'binding_strategy' = 'linear',
'l_hard' = {
'virtual_free' = '12G',
Re: [galaxy-dev] TestToolShed out of date? (not running unit tests)
On Mon, Apr 15, 2013 at 2:20 PM, Dave Bouvier d...@bx.psu.edu wrote: Peter, The automated testing framework is also running against the test tool shed. Could you send me a link to the repository that is behaving oddly? e.g. http://toolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp On the main tool shed, when I am logged in, on this page under the top right menu Repository actions the second entry is View tool functional test results. However, on the test tool shed, that menu item is missing: http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp Although the history is different, I believe the same tarball for v0.2.1 of the suite was uploaded to both the main and test tool sheds. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Broken tools ids from ./run_functional_tests.sh -list
On Mon, Apr 15, 2013 at 2:15 PM, Dave Bouvier d...@bx.psu.edu wrote:
Peter,
Thanks for reporting this issue, it looks like there might be a bug in
tool_list.py, which I will be looking into. In the meantime, you should
still be able to run functional tests for your tool by executing
sh run_functional_tests.sh -id tool_id
Where tool_id is the ID you have defined in the tool XML. The
functional_tests.py script uses Galaxy's internal tool XML parser to
generate the list of functional tests to run, so it should be able to find
your tool and its tests.
--Dave B.
Thanks for the clue Dave - you're right it is a bug in tool_list.py
which uses its own quick and dirty XML parser. The bug was
that is was confused by the use of the magic words 'id', 'name'
or 'description' in the values.
The patch below works for me, but assumes that there will
not be a space between the key and the equals sign. The
script remains a fragile hack - for instance if a tool's XML file
split the tool tag over multiple lines, only the first line
is used. Using a real XML parser instead would be better.
The patch also addresses a potential overflow bug where it
tests n = len(keys) - 1 as a possible key, and would then
go out of bounds to get the value from list element n+1.
The patch also replaces the use of -1 from find, using in is
far clearer.
The patch does not fix the non-standard 3 space indentation
(PEP8 recommends 4 space indentation).
Could you test this and commit it?
Peter
P.S. Yes, you're right for tool identifiers the ID is easy to get from
the tool's XML file (that doesn't work for sections though).
--
diff -r d8d3e4fe7c45 tool_list.py
--- a/tool_list.py Fri Apr 12 11:27:17 2013 +0100
+++ b/tool_list.py Mon Apr 15 14:51:34 2013 +0100
@@ -35,14 +35,15 @@
for line in open(tools/+tool) :
if line.find(tool ) != -1 and line.find(id) != -1 :
keys = line.strip().split('\')
-n = 0
tool_info = dict()
tool_info[desc] = ''
-while n len(keys) :
- if keys[n].find(id) != -1 : tool_info[id] =
keys[n+1].replace(' ', '_')
- if keys[n].find(name) != -1 : tool_info[name] = keys[n+1]
- if keys[n].find(description) != -1 :
tool_info[desc] = keys[n+1]
- n = n + 1
+for n in range(len(keys)-1):
+ if id= in keys[n]:
+ tool_info[id] = keys[n+1].replace(' ', '_')
+ if name= in keys[n]:
+ tool_info[name] = keys[n+1]
+ if description= in keys[n]:
+ tool_info[desc] = keys[n+1]
tool_infos.append(tool_info)
break
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Broken tools ids from ./run_functional_tests.sh -list
Peter, Thanks for the contribution, I've verified that it works, and it has been committed in 9395:388e86c30bad. --Dave B. On 4/15/13 09:59:46.000, Peter Cock wrote: On Mon, Apr 15, 2013 at 2:15 PM, Dave Bouvier d...@bx.psu.edu wrote: Peter, Thanks for reporting this issue, it looks like there might be a bug in tool_list.py, which I will be looking into. In the meantime, you should still be able to run functional tests for your tool by executing sh run_functional_tests.sh -id tool_id Where tool_id is the ID you have defined in the tool XML. The functional_tests.py script uses Galaxy's internal tool XML parser to generate the list of functional tests to run, so it should be able to find your tool and its tests. --Dave B. Thanks for the clue Dave - you're right it is a bug in tool_list.py which uses its own quick and dirty XML parser. The bug was that is was confused by the use of the magic words 'id', 'name' or 'description' in the values. The patch below works for me, but assumes that there will not be a space between the key and the equals sign. The script remains a fragile hack - for instance if a tool's XML file split the tool tag over multiple lines, only the first line is used. Using a real XML parser instead would be better. The patch also addresses a potential overflow bug where it tests n = len(keys) - 1 as a possible key, and would then go out of bounds to get the value from list element n+1. The patch also replaces the use of -1 from find, using in is far clearer. The patch does not fix the non-standard 3 space indentation (PEP8 recommends 4 space indentation). Could you test this and commit it? Peter P.S. Yes, you're right for tool identifiers the ID is easy to get from the tool's XML file (that doesn't work for sections though). ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Broken tools ids from ./run_functional_tests.sh -list
On Mon, Apr 15, 2013 at 3:12 PM, Dave Bouvier d...@bx.psu.edu wrote: Peter, Thanks for the contribution, I've verified that it works, and it has been committed in 9395:388e86c30bad. --Dave B. Great, thank you :) Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] TestToolShed out of date? (not running unit tests)
Peter, I'm unable to duplicate that behavior, the View tool functional test results option shows up on the test tool shed both when I'm logged in and logged out. My suggestion would be to clear your browser's cookies and cache, and see if that makes a difference. --Dave B. On 4/15/13 09:35:40.000, Peter Cock wrote: On Mon, Apr 15, 2013 at 2:20 PM, Dave Bouvier d...@bx.psu.edu wrote: Peter, The automated testing framework is also running against the test tool shed. Could you send me a link to the repository that is behaving oddly? e.g. http://toolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp On the main tool shed, when I am logged in, on this page under the top right menu Repository actions the second entry is View tool functional test results. However, on the test tool shed, that menu item is missing: http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp Although the history is different, I believe the same tarball for v0.2.1 of the suite was uploaded to both the main and test tool sheds. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] TestToolShed out of date? (not running unit tests)
On Mon, Apr 15, 2013 at 3:56 PM, Dave Bouvier d...@bx.psu.edu wrote: Peter, I'm unable to duplicate that behavior, the View tool functional test results option shows up on the test tool shed both when I'm logged in and logged out. My suggestion would be to clear your browser's cookies and cache, and see if that makes a difference. --Dave B. That did it - thanks :) I also have a feature request now though ;) The View tool functional test results option clearly lists failed tests, but it is not obvious if there were any successful tests. Could that be indicated somehow (e.g. five tests passed for tool xxx). Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] tool_dependencies inside tool_dependencies
Hi, is there a general rule to handle dependencies inside of tool_dependencies.xml? Lets assume I write a matplotlib orphan tool_dependencies.xml file. matplotlib depends on numpy. Numpy has already a orphan definition. Is there a way to include numpy as dependency inside the matplotlib-definition, so that I did not need to fetch and compile numpy inside of matplotlib? I tried to specify it beforehand but that did not work. Thanks! Bjoern ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tool_dependencies inside tool_dependencies
Hi Greg, If numpy is not required for compiling matplotlib components (i.e., matplotlib components just use numpy after installation), then you should be able to make this work using a complex repository dependency for numpy in your tool_dependencies.xml definition for matplotlib. The discussion for doing this is at http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_repository_dependencies:_tool_dependency_definitions_that_contain_repository_dependency_definitions Thanks! But it is required at compile time. By the way, I noticed that revision 2:c5fbe4aa5a74 of your package_numpy_1_7 repository on the test tool shed includes the following contents. Is this the repository you are working with? Strangely, the repository dependency should be invalid because it should not be possible for a repository to define a dependency upon any revision of itself. You may have uncovered a way to do this using a tool dependency definition with a complex repository dependency. I'll look into this and make sure to provide a fix for the scenario you used. Oh, ok. in revision 3 of both packages you should see, what I was trying. Instead of the above approach, your approach here should be to include only the tool_dependencies.cml definition file for installing only numpy version 1.7.1 in a r epository named package_numpy_1_7_1 (use the full version in naming the repository). You should create a separate repository named package_matplotlib_1_2_1 that similarly contains a single tool_dependencies.xml file that (in addition to defining how to install and compile mtplotlib) defines a complex repository dependency on the package_numpy_1_7_1 repository as described in the wiki at the link above. This approach creates 2 separate orphan tool dependencies, the second of which (matplotlib) has a complex repository dependency on the first (numpy). When you install the package_matplotlib_1_2_1 repository and check the box for handling tool dependencies during the installation, it will install the package_numpy_1_7_1 repository and create a pointer to the numpy binary in the env.sh file within the package_matplotlib_1_2_1 repository environment. This enables matplotlib to locate the required version of numpy. I know this is a bit tricky, so please let me know if it still does not make sense. Lets see if I got it right. repository_dependencies.xml will be pared first. The defined repo's and the included and populated system variables will be available in tool_dependencies.xml, which is parsed afterwards. Is that correct? I will try that. Thanks! Bjoern Thanks very much, Greg Von Kuster On Apr 15, 2013, at 3:29 PM, Björn Grüning wrote: Hi, is there a general rule to handle dependencies inside of tool_dependencies.xml? Lets assume I write a matplotlib orphan tool_dependencies.xml file. matplotlib depends on numpy. Numpy has already a orphan definition. Is there a way to include numpy as dependency inside the matplotlib-definition, so that I did not need to fetch and compile numpy inside of matplotlib? I tried to specify it beforehand but that did not work. Thanks! Bjoern ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] the release_2013.04.01 update crashes our galaxy server
Hi, After a crash recovery and a second trial I must admit that I cannot upgrade our galaxy instance. I need some help... the facts: initial instance: $ hg head changeset: 8796:961ae35ba612 branch: stable tag: tip parent: 8794:1c7174911392 user:Nate Coraor n...@bx.psu.edu date:Fri Feb 08 11:23:04 2013 -0500 summary: Added tag release_2013.02.08 for changeset 1c7174911392 changeset: 8795:9fd7fe0c5712 parent: 8785:25bd9d387135 parent: 8794:1c7174911392 user:Nate Coraor n...@bx.psu.edu date:Fri Feb 08 11:22:54 2013 -0500 summary: merge from stable Then I pull the upgrade using mercurial and update the postgresql database as follows: $ hg pull pulling from https://bitbucket.org/galaxy/galaxy-dist/ searching for changes adding changesets adding manifests adding file changes added 516 changesets with 1568 changes to 710 files (-1 heads) (run 'hg update' to get a working copy) galaxy@GED-Server:~/galaxy-dist$ hg update release_2013.04.01 merging lib/galaxy/web/form_builder.py merging tools/ngs_rna/tophat_wrapper.xml 750 files updated, 2 files merged, 433 files removed, 0 files unresolved galaxy@GED-Server:~/galaxy-dist$ hg branch stable galaxy@GED-Server:~/galaxy-dist$ sh manage_db.sh upgrade 110 - 111... Add support for job destinations to the job table done 111 - 112... Migration script to add the data_manager_history_association table and data_manager_job_association. 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-04-16 00:59:22,412 Created data_manager_history_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-04-16 00:59:22,412 Created data_manager_history_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-04-16 00:59:22,428 Created data_manager_job_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-04-16 00:59:22,428 Created data_manager_job_association table done 112 - 113... Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate. done 113 - 114... Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate. done When I restarts the server, I have $ sh run.sh --reload Initializing data_manager_conf.xml from data_manager_conf.xml.sample Initializing shed_data_manager_conf.xml from shed_data_manager_conf.xml.sample Some eggs are out of date, attempting to fetch... Fetched http://eggs.galaxyproject.org/PasteDeploy/PasteDeploy-1.5.0-py2.7.egg Removed conflicting egg: /home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg Fetched http://eggs.galaxyproject.org/raven/raven-3.1.8-py2.7.egg Fetch successful. I suspect that some new python eggs may cause my troubles... and then, ... some usual bla bla... and finishes by galaxy.webapps.galaxy.buildapp DEBUG 2013-04-16 01:00:30,077 Enabling 'Request ID' middleware Traceback (most recent call last): File ./scripts/paster.py, line 33, in module serve.run() File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 1056, in run invoke(command, command_name, options, args[1:]) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 1062, in invoke exit_code = runner.run(args) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 227, in run result = self.command() File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 650, in command app = loadapp( app_spec, name=app_name, relative_to=base, global_conf=vars) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 350, in loadapp return loadobj(APP, uri, name=name, **kw) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 375, in loadobj return context.create() File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 813, in create return self.object_type.invoke(self) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 332, in invoke filtered = context.next_context.create() File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 813, in create return self.object_type.invoke(self) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 249, in invoke return fix_call(context.object, context.global_conf, **context.local_conf) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 97, in fix_call val = callable(*args, **kw) File /home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py, line 172, in app_factory webapp = wrap_in_static( webapp, global_conf, **kwargs ) File /home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py, line 327, in
Re: [galaxy-dev] the release_2013.04.01 update crashes our galaxy server
This looks like just a configuration file problem. What is the value
of the various static_ options (like static_dir =) in your
universe_wsgi.ini?
On Mon, Apr 15, 2013 at 7:25 PM, Christophe Antoniewski
christophe.antoniew...@snv.jussieu.fr wrote:
webapp = wrap_in_static( webapp, global_conf, **kwargs )
File /home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py,
line 327, in wrap_in_static
urlmap[/static] = Static( conf.get( static_dir ), cache_time )
File
/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/static.py,
line 16, in __init__
StaticURLParser.__init__( self, directory )
File
/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/urlparser.py,
line 430, in __init__
self.directory = self.normpath(directory)
File
/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/urlparser.py,
line 435, in normpath
return os.path.normcase(os.path.abspath(path))
File /usr/lib/python2.7/posixpath.py, line 343, in abspath
if not isabs(path):
File /usr/lib/python2.7/posixpath.py, line 53, in isabs
return s.startswith('/')
AttributeError: 'NoneType' object has no attribute 'startswith'
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tool_dependencies inside tool_dependencies
Hi Björn, On Apr 15, 2013, at 6:31 PM, Björn Grüning wrote: Hi Greg, If numpy is not required for compiling matplotlib components (i.e., matplotlib components just use numpy after installation), then you should be able to make this work using a complex repository dependency for numpy in your tool_dependencies.xml definition for matplotlib. The discussion for doing this is at http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_repository_dependencies:_tool_dependency_definitions_that_contain_repository_dependency_definitions Thanks! But it is required at compile time. Ok, we may need to do a bit of work to support this requirement, but I'm not quite sure. What I've described to you should still be your approach, but we'll need to ensure that the package_numpy_1_7_1 repository is installed before the package_matplotlib_1_2_1 is installed. Guaranteeing this is not currently possible, but this is a feature I am hoping to have available this week. This is a feature that Ira Cooke has needed for his repositories. When the feature is available, it will support an attribute named prior_installation_required in the repository tag, so this tag will look something like: repository toolshed=www name=xxx owner=yyy changeset_revision=zzz prior_installation_required=True / What this will do is skip installation of the repository that contains this dependency until the repository that is associated with the prior_installation_required attribute is installed (unless that repository is not in the current list of repositories being installed). What I think still needs to be worked out is how to ensure that the tool_dependencies.xml definition that installs the matplotlib package will find the previously installed numpy binary during compilation of matplotlib. Currently, the numpy binary will only be available to the installed and compiled matplotlib binary. I'll create a Trello card for this and let you know an estimate of when it will be available. By the way, I noticed that revision 2:c5fbe4aa5a74 of your package_numpy_1_7 repository on the test tool shed includes the following contents. Is this the repository you are working with? Strangely, the repository dependency should be invalid because it should not be possible for a repository to define a dependency upon any revision of itself. You may have uncovered a way to do this using a tool dependency definition with a complex repository dependency. I'll look into this and make sure to provide a fix for the scenario you used. Oh, ok. in revision 3 of both packages you should see, what I was trying. Ok, revision 3 looks good as long as it correctly installs and compiles numpy. Instead of the above approach, your approach here should be to include only the tool_dependencies.cml definition file for installing only numpy version 1.7.1 in a r epository named package_numpy_1_7_1 (use the full version in naming the repository). You should create a separate repository named package_matplotlib_1_2_1 that similarly contains a single tool_dependencies.xml file that (in addition to defining how to install and compile mtplotlib) defines a complex repository dependency on the package_numpy_1_7_1 repository as described in the wiki at the link above. This approach creates 2 separate orphan tool dependencies, the second of which (matplotlib) has a complex repository dependency on the first (numpy). When you install the package_matplotlib_1_2_1 repository and check the box for handling tool dependencies during the installation, it will install the package_numpy_1_7_1 repository and create a pointer to the numpy binary in the env.sh file within the package_matplotlib_1_2_1 repository environment. This enables matplotlib to locate the required version of numpy. I know this is a bit tricky, so please let me know if it still does not make sense. Lets see if I got it right. repository_dependencies.xml will be pared first. The defined repo's and the included and populated system variables will be available in tool_dependencies.xml, which is parsed afterwards. Is that correct? I'm not quite sure I understand your statements above, but I've looked at revision 3 of your package_matplotlib_1_2_1 repository and the tool_dependencies.xml definition looks good (with the exception of the currently unsupported prior_installation_required attribute), so I think you've successfully deciphered my documentation. I'll make sure to keep you informed as I make progress on the missing pieces that will support what you need this week. I will try that. Thanks! Bjoern Thanks very much, Greg Von Kuster On Apr 15, 2013, at 3:29 PM, Björn Grüning wrote: Hi, is there a general rule to handle dependencies inside of tool_dependencies.xml? Lets assume I write a matplotlib orphan tool_dependencies.xml file. matplotlib depends on
