Re: [galaxy-dev] How to get cleanup script to run in cron job ?
Hi, Try this : 55 21 * * * cd /media/Store/galaxy/galaxy-dist/ && python scripts/cleanup_datasets/cleanup_datasets.py config/galaxy.ini -d 30 -1 -i >> /media/Store/galaxy/galaxy-dist/temp 2>&1 I thought you need to be in the galaxy-dist folder when launching the commands. Best, Geert On 11/19/2014 11:14 PM, James Mullan wrote: Hello, I am trying to figure out how to run the cleanup scripts in a cron job. The below works on command line: python scripts/cleanup_datasets/cleanup_datasets.py config/galaxy.ini -d 30 -1 -i pysqlite>=2 egg successfully loaded for sqlite dialect ## # 2014-11-19 22:07:32 - Handling stuff older than 30 days # Displaying info only ( --info_only ) Deleted 0 histories Elapsed time: 0.0684289932251 ## but if I try to do this in a cron job as follows: 55 21 * * * python scripts/cleanup_datasets/cleanup_datasets.py config/galaxy.ini -d 30 -1 -i >> /media/Store/galaxy/galaxy-dist/temp 2>&1 python: can't open file 'scripts/cleanup_datasets/cleanup_datasets.py': [Errno 2] No such file or directory or if I try to do this : 00 22 * * * python /media/Store/galaxy/galaxy-dist/scripts/cleanup_datasets/cleanup_datasets.py config/galaxy.ini -d 30 -1 -i >> /media/Store/galaxy/galaxy-dist/temp 2>&1 Traceback (most recent call last): File "/media/Store/galaxy/galaxy-dist/scripts/cleanup_datasets/cleanup_datasets.py", line 13, in from galaxy import eggs ImportError: No module named galaxy Note that I am directing the output to a file only to try to find out why it's not working. Is this a problem with environment? Any suggestions what to try would be much appreciated. Regards James J P Mullan ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] testtoolshed : python-2.7 installation error
sorry about that confusion :-) Yes, I'm talking about the package_python_2.7 repository. The same error is also present in the test runs of the package_python_3* repo. Best, Geert On 08/07/2014 11:45 AM, Björn Grüning wrote: Hi Geert, are you talking about the package_python_* repository? Am 07.08.2014 um 10:00 schrieb Geert Vandeweyer: Hi, I get an installation error on the python 2.7 package in the test toolshed. I used the 'contact owner' function, but wanted to mention it here too, as there hasn't been reaction so far. Sorry for double posting if so. Oh this is interesting, because we did not get notified. It is from IUC correct? Error: tar (child): 5.2.tar.bz2: Cannot open: No such file or directory A similar error is in the Test run outputs. I believe it is related to the following (unnecessary) line in the tool_dependency.xml: .. This can be. The cwd directory handling changed a lot in recent versions, due to bugs. And for a few repositories we still have the workarounds included. I will upload a new version without this workaround and see if it passes. located just after the download_file action for the 5.2.tar.bz2 file. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] testtoolshed : python-2.7 installation error
Hi, I get an installation error on the python 2.7 package in the test toolshed. I used the 'contact owner' function, but wanted to mention it here too, as there hasn't been reaction so far. Sorry for double posting if so. Error: tar (child): 5.2.tar.bz2: Cannot open: No such file or directory A similar error is in the Test run outputs. I believe it is related to the following (unnecessary) line in the tool_dependency.xml: .. located just after the download_file action for the 5.2.tar.bz2 file. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Serious start-up delay in next-stable
I had a similar issue a while ago. Greg mentioned the following setting in universe: # Enable use of an in-memory registry with bi-directional relationships # between repositories (i.e., in addition to lists of dependencies for a # repository, keep an in-memory registry of dependent items for each repository. #manage_dependency_relationships = True Setting this to false made the delay go away. I haven't had issues so far after disabling. See here for details: http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-March/018821.html Best, Geert On 08/05/2014 09:01 PM, Dannon Baker wrote: There's definitely a request that will occasionally block at startup -- I've hit this before too, Martin. Looking into it now. On Tue, Aug 5, 2014 at 2:59 PM, Martin Čech <mailto:mar...@bx.psu.edu>> wrote: I just freshly cloned galaxy-central, switched to next-stable and did initial startup - no problems detected and Galaxy booted in under 2m. On Tue, Aug 5, 2014 at 2:54 PM, Langhorst, Brad mailto:langho...@neb.com>> wrote: I’ve noticed that startup is very slow. It’s about 15 minutes for me to stop and restart my 4 daemons. It was much faster with galaxy on local storage. brad -- Bradley W. Langhorst, Ph.D. Applications and Product Development Scientist On Aug 5, 2014, at 2:41 PM, Björn Grüning mailto:bjoern.gruen...@gmail.com>> wrote: Hi all, does anyone encountered a long start-up delay with the recent next-stable and starting a instance from scratch? For me its hanging at: tool_shed.galaxy_install.migrate.check DEBUG 2014-08-05 18:39:45,207 psycopg2 egg successfully loaded for postgres dialect for 3-5 minutes. Cheers, Bjoern ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] cufflinks suite 2.2.1
Hi all, I've updated the tool wrappers for the cufflinks tools to options supported by cufflinks 2.2.1. I've also added the cuffquant and cuffnorm tools, together with the cxb datatype. As discussed with Jeremy, these updates were welcome :-) Everything is available on the test toolshed, tested and installed on at least one instance (http://biomina.be/app/galaxy), although mixing main and test toolshed installations doesn't work well. The tools are present twice. Therefore, feel free to integrate the updated wrappers & tools into the main cufflinks package (by devteam) after further testing. Best, geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] creating datatype repository : no module named binary
Hi all, I'm trying to create a new binary datatype in galaxy, but I'm running into troubles. The current installer is available at https://testtoolshed.g2.bx.psu.edu/view/geert-vandeweyer/cuffquant_datatype When I install the repository to a local server, I get the errors below (no module binary). Since I'm new to defining datatypes, any insights on what I'm doing wrong is welcome :-) Best, Geert tool_shed.util.repository_dependency_util DEBUG 2014-08-04 14:39:07,313 Building repository dependency relationships... 127.0.0.1 - - [04/Aug/2014:14:39:03 +0200] "POST /admin_toolshed/prepare_for_install HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install?tool_shed_url=https://testtoolshed.g2.bx.psu.edu/&repository_ids=80b80f5c54443265&changeset_revisions=e12a09256097"; "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/34.0.1847.116 Chrome/34.0.1847.116 Safari/537.36" 127.0.0.1 - - [04/Aug/2014:14:39:10 +0200] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install"; "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/34.0.1847.116 Chrome/34.0.1847.116 Safari/537.36" galaxy.datatypes.registry DEBUG 2014-08-04 14:39:11,362 Loading datatypes from /home/gvandeweyer/galaxy-dist/galaxy-dist/database/tmp/tmp-toolshed-acalpdGx1tEP galaxy.datatypes.registry DEBUG 2014-08-04 14:39:11,363 Exception importing proprietary code file /home/gvandeweyer/galaxy-dist/shed_tools/testtoolshed.g2.bx.psu.edu/repos/geert-vandeweyer/cuffquant_datatype/f7801f8191e0/cuffquant_datatype/binary: No module named binary galaxy.datatypes.registry ERROR 2014-08-04 14:39:11,363 Error importing datatype module galaxy.datatypes.binary: 'module' object has no attribute 'Cuffquant' Traceback (most recent call last): File "/home/gvandeweyer/galaxy-dist/galaxy-dist/lib/galaxy/datatypes/registry.py", line 207, in load_datatypes datatype_class = getattr( module, datatype_class_name ) AttributeError: 'module' object has no attribute 'Cuffquant' galaxy.datatypes.registry ERROR 2014-08-04 14:39:11,364 Error calling method Cuffquant from class '/home/gvandeweyer/galaxy-dist/galaxy-dist/lib/galaxy/datatypes/binary.pyc'>: 'module' object has no attribute 'Cuffquant' Traceback (most recent call last): File "/home/gvandeweyer/galaxy-dist/galaxy-dist/lib/galaxy/datatypes/registry.py", line 349, in load_datatype_sniffers aclass = getattr( module, datatype_class_name )() AttributeError: 'module' object has no attribute 'Cuffquant' -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] new_file_path questions
1265, in _rollback_impl\nself._handle_dbapi_exception(e, None,
None, None, None)\n File
"/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
line 1262, in _rollback_impl\n
self.engine.dialect.do_rollback(self.connection)\n File
"/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/mysql/base.py",
line 1910, in do_rollback\n connection.rollback()\n\n(OperationalError)
(2006, \'MySQL server has gone away\') None None',
u'tool_dependency_id': 326L}]
CGI Variables
-
CONTENT_LENGTH: '-1'
CONTENT_TYPE: 'application/x-www-form-urlencoded; charset=UTF-8'
HTTP_ACCEPT: 'text/html, */*; q=0.01'
HTTP_ACCEPT_ENCODING: 'gzip,deflate,sdch'
HTTP_ACCEPT_LANGUAGE: 'nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4'
HTTP_CACHE_CONTROL: 'max-age=0'
HTTP_CONNECTION: 'Keep-Alive'
HTTP_COOKIE:
'__utma=94419599.1127508766.1403299856.1403299856.1405949987.2;
__utmz=94419599.1403299856.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);
galaxysession=6730c1c7f605dc205ea1a59142b8e9026339899a15206ea6665be6a34bb4fa5a635956369e96da7d'
HTTP_HOST: 'biominavm-galaxy.biomina.be'
HTTP_ORIGIN: 'http://biominavm-galaxy.biomina.be'
HTTP_REFERER:
'http://biominavm-galaxy.biomina.be/galaxy/admin_toolshed/prepare_for_install'
HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36
(KHTML, like Gecko) Ubuntu Chromium/34.0.1847.116 Chrome/34.0.1847.116
Safari/537.36'
HTTP_VIA: '1.1 biominavm-galaxy.biomina.be'
HTTP_X_REQUESTED_WITH: 'XMLHttpRequest'
PATH_INFO: '/admin_toolshed/manage_repositories'
REMOTE_ADDR: '143.169.236.220'
REQUEST_METHOD: 'POST'
SCRIPT_NAME: '/galaxy'
SERVER_NAME: 'biominavm-galaxy.biomina.be'
SERVER_PORT: '8080'
SERVER_PROTOCOL: 'HTTP/1.1'
WSGI Variables
--
application:
is_api_request: False
paste.cookies: (__utma='94419599.1127508766.1403299856.1403299856.1405949987.2'
__utmz='94419599.1403299856.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)'
galaxysession='6730c1c7f605dc205ea1a59142b8e9026339899a15206ea6665be6a34bb4fa5a635956369e96da7d'>,
'__utma=94419599.1127508766.1403299856.1403299856.1405949987.2;
__utmz=94419599.1403299856.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);
galaxysession=6730c1c7f605dc205ea1a59142b8e9026339899a15206ea6665be6a34bb4fa5a635956369e96da7d')
paste.expected_exceptions: []
paste.httpexceptions: object at 0xa8df9d0>
paste.httpserver.thread_pool: 0x2ca6450>
paste.recursive.forward:
paste.recursive.include:
paste.recursive.include_app_iter: from /galaxy>
paste.recursive.script_name: '/galaxy'
paste.throw_errors: True
request_id: 'dc296d0a18b111e484a100163602f2ed'
webob._parsed_post_vars: (MultiDict([('operation', 'install'),
('tool_shed_repository_ids', "['dc4272dc4bc65f47']"), ('encoded_kwd',
'8466491acfb960dfb39d0377c52cdd7698cac2ef:7b22737461747573223a2022646f6e65222c20226861735f7265706f7369746f72795f646570656e64656e63696573223a2066616c73652c2022696e636c756465735f746f6f6c735f666f725f646973706c61795f696e5f746f6f6c5f70616e656c223a2066616c73652c2022746f6f6c5f736865645f7265706f7369746f72795f696473223a205b2264633432373264633462633635663437225d2c2022736865645f746f6f6c5f636f6e66223a20222e2f736865645f746f6f6c5f636f6e662e786d6c222c2022696e7374616c6c5f7265706f7369746f72795f646570656e64656e63696573223a2066616c73652c2022746f6f6c5f70617468223a20222e2e2f736865645f746f6f6c73222c20227265706f5f696e666f5f6469637473223a205b7b227061636b6167655f7065726c5f355f3138223a205b22436f6e7461696e73206120746f6f6c20646570656e64656e637920646566696e6974696f6e207468617420646f776e6c6f61647320616e6420636f6d70696c65732076657273696f6e20352e31382e78206f662074686520746865207065726c206c61...t([('op...e')])>)
webob._parsed_query_vars: (MultiDict([]), '')
wsgi process: 'Multithreaded'
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/geertvandeweyer
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error while installing DESeq-hts
deseq2-hts.sh), append line 70 (../bin/get_read_counts...) with 2>&1 . This prevents some octave warnings from crashing the job (missing docstring warnings) Test-run in galaxy seems to work. ## set up DEXSeq-hts-1.0 ### go to //toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/dexseq_1.0 run setup_dexseq-hts.sh - set samtools path - set python_path : I use the galaxy python version (/galaxy/galaxy_env/bin/python), see development docs - set pythonpath: Make sure to include both HTSeq AND numpy here ! : export PYTHONPATH=/galaxy/galaxy_tool_binaries/htseq/0.5.4p5/lparsons/htseq_count/3ffe4e2572a7/lib/python/HTSeq-0.5.4p5-py2.6-linux-x86_64.egg:/galaxy/galaxy_tool_binaries/numpy/1.7.1/iuc/package_numpy_1_7/ef12a3a11d5b/lib/python:$PYTHONPATH - set R_path : I use toolshed version package_r_3_0_2 from iuc (with the installed bioclite packages from above) => FAILS : crashes on estimateDispersions / fitDispersionFunction step in run_DEXseq.R -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail:geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] is there an issue with displaying txt files in latest galaxy?
Discard this, it appears to be solved after changes to htaccess files on our server. On 07/28/2014 05:30 PM, Geert Vandeweyer wrote: Hi, Is there a known issue in the latest galaxy-dist related to displaying txt files? If I click the eye icon for a 'txt' dataset, nothing is shown. The preview shows the data correctly, and if I check the path on the server, the data and contents are definitely there. main.log doesn't show any errors. I updated today using hg pull && hg update latest_2014.06.02: changeset 13584 Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] is there an issue with displaying txt files in latest galaxy?
Hi, Is there a known issue in the latest galaxy-dist related to displaying txt files? If I click the eye icon for a 'txt' dataset, nothing is shown. The preview shows the data correctly, and if I check the path on the server, the data and contents are definitely there. main.log doesn't show any errors. I updated today using hg pull && hg update latest_2014.06.02: changeset 13584 Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Settings multiple Genome Indices
Excellent, thanks !
Geert
On 07/28/2014 05:02 PM, Jeremy Goecks wrote:
There's an as-yet undocumented method that you can use to get
arbitrary fields in a loc file:
__get_data_table_entry__(, , ,
)
So for the novoalign entry you could do:
--novoidx ${__get_data_table_entry__(‘novoalign’, ‘dbkey’,
$input.dbkey, ‘path’)}
Finally, I like your indexing syntax, and it would be great to use
something like that in place of the explicit method call above.
Best,
J.
--
Jeremy Goecks
Assistant Professor of Computational Biology
George Washington University
On Jul 27, 2014, at 10:44 AM, Geert Vandeweyer
<mailto:geert.vandewey...@uantwerpen.be>> wrote:
Hi,
I'm working on tool wrapper that needs three reference genome
locations (2bit/lastz, all_fasta and novoalign). Is there a way to
fetch the three paths from the loc files using a single select box?
I've looked around in a couple of tool wrappers on the toolshed, but
could find any leads. Something like the following would be ok:
Select options from all_fasta.loc
-> upon change, set hidden variables to novoalign.loc +
lastz_seqs.loc using a filter tag or something against the selected
options from all_fasta.
Even more simple : access in the command tag like:
--novoidx novoalign-loc.options[dbkey=$input.dbkey,path] --fasta
all_fasta-loc[dbkey=$input.dbkey,path] etc , using the dbkey from the
input file (which is a bam file in this case)
In this case, error handling should come from the tool, if one of the
indices is missing.
I'm currently using three selection boxes, but that looks a but
redundant to the user, as they all have to select the same thing.
Best,
Geert
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be <mailto:geert.vandewe...@ua.ac.be>
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/geertvandeweyer
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/geertvandeweyer
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Settings multiple Genome Indices
If I understand you correctly, would this mean generating a custom .loc file for this tool containing the concatenated paths? Or can I concatenate paths from different .loc files somehow, which would allow me to use the default loc files? Geert On 07/28/2014 09:33 AM, Hans-Rudolf Hotz wrote: Hi Geert A simple solution might be: you hand over a string containing all three paths to your wrapper, and then the wrapper splits the string into the three individual paths. Hans-Rudolf On 07/27/2014 04:44 PM, Geert Vandeweyer wrote: Hi, I'm working on tool wrapper that needs three reference genome locations (2bit/lastz, all_fasta and novoalign). Is there a way to fetch the three paths from the loc files using a single select box? I've looked around in a couple of tool wrappers on the toolshed, but could find any leads. Something like the following would be ok: Select options from all_fasta.loc -> upon change, set hidden variables to novoalign.loc + lastz_seqs.loc using a filter tag or something against the selected options from all_fasta. Even more simple : access in the command tag like: --novoidx novoalign-loc.options[dbkey=$input.dbkey,path] --fasta all_fasta-loc[dbkey=$input.dbkey,path] etc , using the dbkey from the input file (which is a bam file in this case) In this case, error handling should come from the tool, if one of the indices is missing. I'm currently using three selection boxes, but that looks a but redundant to the user, as they all have to select the same thing. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Settings multiple Genome Indices
Hi, I'm working on tool wrapper that needs three reference genome locations (2bit/lastz, all_fasta and novoalign). Is there a way to fetch the three paths from the loc files using a single select box? I've looked around in a couple of tool wrappers on the toolshed, but could find any leads. Something like the following would be ok: Select options from all_fasta.loc -> upon change, set hidden variables to novoalign.loc + lastz_seqs.loc using a filter tag or something against the selected options from all_fasta. Even more simple : access in the command tag like: --novoidx novoalign-loc.options[dbkey=$input.dbkey,path] --fasta all_fasta-loc[dbkey=$input.dbkey,path] etc , using the dbkey from the input file (which is a bam file in this case) In this case, error handling should come from the tool, if one of the indices is missing. I'm currently using three selection boxes, but that looks a but redundant to the user, as they all have to select the same thing. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] fix for GATK2 wrapper from IUC
Hi, There is a missing parameter in the unified genotyer config from the iuc gatk2 wrapper: The following line should be added around line 204 (just before ): Without this line, activating the advanced analysis options causes the job preparation to fail because there is a reference to it in the section gatk2 wrapper used: version : 8bcc13094767 hg tip: changeset: 3:2553f84b8174 tag: tip user:iuc date:Wed Feb 19 04:39:38 2014 -0500 summary: Uploaded Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Missing navigation pane
Hey Stephen, Not a true solution, but I solved a similar issue (missing history pane) by clearing out the browser cache. Geert On 06/10/2014 04:39 PM, Dannon Baker wrote: Hey Stephen, Can you look in the server logs to see if there are any errors being reported? Or if there are any javascript errors in the browser window? You may also want to try clearing your browser cache. -Dannon On Tue, Jun 10, 2014 at 1:54 AM, Stephen E <mailto:sedwards...@hotmail.com>> wrote: I installed a local instance of galaxy approximately a month ago (build 2014.04.14). I was having an issue with the Data Manager to fetch reference genomes and after checking to see if there was a newer version of galaxy, I decided to update and see if that fixed my problem. I ran "hg pull" and then "hg update latest_2014.06.02". I tried to rerun galaxy but was instructed to run manage_db.sh so I did (sh manage_db.sh upgrade). When I ran galaxy after all this, it started but when opened in a web browser, the navagation pane at the top is missing (i.e. Analyze Data, Workflows, User, etc). There is a blue bar but nothing is on it. How do I get the missing navigation pane back? I can't do a lot of things without it (i.e. check help or change user settings). I can still get to the administrator section but only by appending "/admin" to the url. I need to know how to fix the new version or how to succesfully revert to a previous version without anything else breaking. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] gatk2 wrapper issue
Hi, I installed the GATK2 wrapper from iuc on the lastest galaxy version. It fails to create the GATK2_PATH and GATK2_SITE_OPTIONS structures under the /tool_dependency_dir/environment_settings/ location, without a clear error (says 'never installed'). If I create the files manually, I can see them through : Admin - Manage installed toolshed repos - gatk2 - missing tool dependencies - GATK2_PATH If I click the hyperlink for GATK2_PATH here (this is the install missing dependency page), I can expand and show the GATK2_PATH / env.sh file Still, running GATK2 tools (eg UG, gives me in the logs: galaxy.jobs.handler INFO 2014-06-04 12:18:07,333 (78998) Job dispatched galaxy.tools.deps WARNING 2014-06-04 12:18:07,928 Failed to resolve dependency on 'gatk2', ignoring galaxy.tools.deps WARNING 2014-06-04 12:18:07,971 Failed to resolve dependency on 'GATK2_PATH', ignoring galaxy.tools.deps WARNING 2014-06-04 12:18:07,974 Failed to resolve dependency on 'GATK2_SITE_OPTIONS', ignoring and tool error: Unable to access jarfile /GenomeAnalysisTK.jar For my mutect_wrapper in test.toolshed, which I initially copied from the gatk2 wrapper, the exact same setup seems to work. Any help would be appreciated. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] devteam GATK v1.4 toolshed wrapper depends on java 7
Hi, Could you add java 1.7 to the dependency list of the gatk 1.4 suite in the main toolshed. Moving from built-in to toolshed version (including the downloaded jars) after the last upgrade, GATK 1.4 tools crashed on : Exception in thread "main" java.lang.UnsupportedClassVersionError: org/broadinstitute/sting/gatk/CommandLineGATK : Unsupported major.minor version 51.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(Unknown Source) at java.lang.ClassLoader.defineClass(Unknown Source) at java.security.SecureClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.access$000(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) Could not find the main class: org.broadinstitute.sting.gatk.CommandLineGATK. Program will exit. adding sun-jre-7 to the PATH as first java option solved this. I thought that only gatk > 2.6 depended on java-7, so I'm not sure why this is happening. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] identify tool based on qstat name.
Hi, I would like to know what package/default tools is creating the jobs on our galaxy server like: 76905_converter_gff_to_fli_0_xx...@email.com I see these jobs in the pbs queue, and they use a lot of memory, up to 18Gb of Ram. Since I can't find the job in the tool configuration, I can't set the memory requirements in the job rules :-) Thanks, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Bam upload fails: samtools not found
Peter, After restart of galaxy, the problems was resolved. I've had a similar issue before, where environment settings were not detected after a prolonged uptime of Galaxy. That issue was also resolved after restart. Anyway, problem is fixed for now. thanks for the answers. Geert On 05/05/2014 07:28 PM, Peter Cock wrote: Sorry - that has gone off list, can we continue this on the mailing list please? On Mon, May 5, 2014 at 6:27 PM, Peter Cock wrote: On Mon, May 5, 2014 at 6:22 PM, Geert Vandeweyer wrote: yes, it's on the galaxy users path. I log in using BASH as the shell. path is loaded using .profile in case that matters. geert Does the upload job run on the main Galaxy server, or your cluster? If on the cluster, you will need to check the path settings for jobs - which may not include those .profile settings. Peter -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Bam upload fails: samtools not found
Hi, I'm having issues uploading bam files to our galaxy server. It fails for both FTP and browser upload. Upload.py is executed on the local job runner. The error is: Traceback (most recent call last): File "/galaxy/galaxy-dist/tools/data_source/upload.py", line 390, in __main__() File "/galaxy/galaxy-dist/tools/data_source/upload.py", line 379, in __main__ add_file( dataset, registry, json_file, output_path ) File "/galaxy/galaxy-dist/tools/data_source/upload.py", line 318, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File "/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 144, in dataset_content_needs_grooming version = self._get_samtools_version() File "/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 128, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File "/usr/lib/python2.6/subprocess.py", line 633, in __init__ errread, errwrite) File "/usr/lib/python2.6/subprocess.py", line 1139, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory The logs show 'galaxy.tools.deps WARNING 2014-05-05 15:39:20,676 Failed to resolve dependency on 'samtools', ignoring' I do have samtools in my path (~/bin/samtools). Also, the file is actually uploaded fine (can be downloaded again correctly), but cannot be used in galaxy due to its 'error' state. Any advice on how to solve this? Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Migrate user data to a new server
I successfully moved galaxy to a new server about a year ago (so some things might have changed...). Things to do/keep in mind were: - Try to keep paths identical on the new server. Some paths are stored in the database. I changed the galaxy user-data location, so I had to manually update using mysql UPDATE commands. - users/passwords are migrated just fine if you dump the database to a file and load that in the new server - I simply copied over the entire installation tree, including reference data, user data, binaries and so on. Double check the universie_wsgi for all paths and locations Geert On 04/15/2014 06:00 AM, Simon Gladman wrote: Hi all, I've tried looking in the mailing list for this to no avail. (Maybe my search terms were no good..) I administer a server with about 15 users. The server is failing and so I have set up a new server (with an updated OS, Galaxy version etc (galaxy versions: old - nov 2013, new - Feb 2014). Is there a method for migrating my users and their data across to the new server, without getting them to tell me their passwords Cheers, Simon. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy startup takes very long. Normal?
I'm using galxy-dist. That's why I don't have the setting. Thanks for the information, I'll wait for the next stable release and update the settings then. Geert On 03/26/2014 01:59 PM, Greg Von Kuster wrote: The changeset that made this a configurable setting was committed to the stable branch ind270d3c, which was committed on 2014-02-19 https://bitbucket.org/galaxy/galaxy-central/commits/d270d3cf7627a42b6fac2aa69701deadb89c8ffc If you are tracking the stable branch in the galaxy central repo (which it looks like you are doing), then you should be able to pull it from there. Greg Von Kuster On Mar 26, 2014, at 8:04 AM, Geert Vandeweyer <mailto:geert.vandewey...@uantwerpen.be>> wrote: Hi Greg, The setting was not in my universe_sample file. I added it, set log_info to DEBUG and restarted galaxy. The startup time remained the same, and there were many entries in the log file indicating that the lag is indeed the creation of the dependency system: tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:01,176 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:02,645 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:03,874 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:05,383 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:06,533 Adding an entry for version 2.11.0 of package R to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:08,054 Adding an entry for version 2.11.0 of package R to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:09,126 Adding an entry for version 2.11.0 of package R to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:10,596 Adding an entry for version 2.11.0 of package R to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:11,803 Adding an entry for version 1.02.00 of package lastz to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:13,332 Adding an entry for version 1.02.00 of package lastz to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:14,552 Adding an entry for version 3.0.1 of package R_3_0_1 to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:15,943 Adding an entry for version 3.0.1 of package R_3_0_1 to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:17,162 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:18,594 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. etc... I also note many double entries in the log files. Do I need a specific changeset for the parameter setting to take effect? Universe_wsgi.ini setting: # Enable use of an in-memory registry with bi-directional relationships between repositories manage_dependency_relationships = False Best, Geert On 03/26/2014 11:53 AM, Greg Von Kuster wrote: Hi Geert, The setting should be in your universe_wsgi.ini.sample, and you woud have to manually edit your universe_wsgi.ini to add it. I would say that 125 installed packages is probably what is causing the slow starts. This feature is not currently useful, so it can be set to not function with no problems. In the future I'll introduce additional benefits for this feature, and I'll look at ways to improve the startup speed. Greg Von Kuster On Mar 26, 2014, at 3:43 AM, Geert Vandeweyer <mailto:geert.vandewey...@uantwerpen.be>> wrote: hi Greg, I don't have that setting in my universe. Should I just add it? The output of hg summar
Re: [galaxy-dev] Galaxy startup takes very long. Normal?
Hi Greg, The setting was not in my universe_sample file. I added it, set log_info to DEBUG and restarted galaxy. The startup time remained the same, and there were many entries in the log file indicating that the lag is indeed the creation of the dependency system: tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:01,176 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:02,645 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:03,874 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:05,383 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:06,533 Adding an entry for version 2.11.0 of package R to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:08,054 Adding an entry for version 2.11.0 of package R to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:09,126 Adding an entry for version 2.11.0 of package R to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:10,596 Adding an entry for version 2.11.0 of package R to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:11,803 Adding an entry for version 1.02.00 of package lastz to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:13,332 Adding an entry for version 1.02.00 of package lastz to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:14,552 Adding an entry for version 3.0.1 of package R_3_0_1 to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:15,943 Adding an entry for version 3.0.1 of package R_3_0_1 to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:17,162 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:18,594 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. etc... I also note many double entries in the log files. Do I need a specific changeset for the parameter setting to take effect? Universe_wsgi.ini setting: # Enable use of an in-memory registry with bi-directional relationships between repositories manage_dependency_relationships = False Best, Geert On 03/26/2014 11:53 AM, Greg Von Kuster wrote: Hi Geert, The setting should be in your universe_wsgi.ini.sample, and you woud have to manually edit your universe_wsgi.ini to add it. I would say that 125 installed packages is probably what is causing the slow starts. This feature is not currently useful, so it can be set to not function with no problems. In the future I'll introduce additional benefits for this feature, and I'll look at ways to improve the startup speed. Greg Von Kuster On Mar 26, 2014, at 3:43 AM, Geert Vandeweyer <mailto:geert.vandewey...@uantwerpen.be>> wrote: hi Greg, I don't have that setting in my universe. Should I just add it? The output of hg summary is (hg incoming doesn't show any available updates): parent: 12276:dc067a95261d tip Added tag release_2014.02.10 for changeset 5e605ed6069f branch: stable commit: 22 modified, 1 deleted, 127 unknown update: (current) I currently have 125 packages from toolsheds. Is that considered to be "many"? Most of them are the migrated tools and the devteam mappers/gatk/tophat stuff. Geert On 03/25/2014 05:16 PM, Greg Von Kuster wrote: Hello Geert, Do you have a lot of repositories installed from the Tool Shed into you Galaxy instance? if so, the time you're experience may be due to loading the in-memory installed repository registry. Using this registry is optional, but the default configuration setting is to
Re: [galaxy-dev] Galaxy startup takes very long. Normal?
hi Greg, I don't have that setting in my universe. Should I just add it? The output of hg summary is (hg incoming doesn't show any available updates): parent: 12276:dc067a95261d tip Added tag release_2014.02.10 for changeset 5e605ed6069f branch: stable commit: 22 modified, 1 deleted, 127 unknown update: (current) I currently have 125 packages from toolsheds. Is that considered to be "many"? Most of them are the migrated tools and the devteam mappers/gatk/tophat stuff. Geert On 03/25/2014 05:16 PM, Greg Von Kuster wrote: Hello Geert, Do you have a lot of repositories installed from the Tool Shed into you Galaxy instance? if so, the time you're experience may be due to loading the in-memory installed repository registry. Using this registry is optional, but the default configuration setting is to use it. You can set the following to entry to False in your universe_wsgi.ini file and restart your Galaxy server to see if that is the problem. This registry is not currently used by anythin except to display the set of dependent repositories that will be affected if a repository is uninstalled. If this is ot what is causing the slow statup, then I'm not sure where else to look. # Enable use of an in-memory registry with bi-directional relationships # between repositories (i.e., in addition to lists of dependencies for a # repository, keep an in-memory registry of dependent items for each repository. #manage_dependency_relationships = True Greg Von Kuster On Mar 25, 2014, at 11:08 AM, Geert Vandeweyer <mailto:geert.vandewey...@uantwerpen.be>> wrote: Dear all, I'm wondering if the following behaviour is normal. Since I reinstalled the latest galaxy distribution, every restart hangs/loads for up to 10 minutes at the following lines: galaxy.model.migrate.check INFO 2014-03-25 15:56:57,965 At database version 118 tool_shed.galaxy_install.migrate.check INFO 2014-03-25 15:56:58,477 At migrate_tools version 9 galaxy.config INFO 2014-03-25 15:56:58,694 Install database targetting Galaxy's database configuration. After several minutes of no heavy processing (cpu/database is almost idle), the startup continues. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be <mailto:geert.vandewe...@ua.ac.be> http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Galaxy startup takes very long. Normal?
Dear all, I'm wondering if the following behaviour is normal. Since I reinstalled the latest galaxy distribution, every restart hangs/loads for up to 10 minutes at the following lines: galaxy.model.migrate.check INFO 2014-03-25 15:56:57,965 At database version 118 tool_shed.galaxy_install.migrate.check INFO 2014-03-25 15:56:58,477 At migrate_tools version 9 galaxy.config INFO 2014-03-25 15:56:58,694 Install database targetting Galaxy's database configuration. After several minutes of no heavy processing (cpu/database is almost idle), the startup continues. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How To properly use GALAXY_SLOTS
This is strange.
After restarting galaxy to include the debug tool, all galaxy_slots
values are correctly assigned (tested for BWA and GATK as rerun of
original posted jobs)
If you still need the output, let me know.
Best,
Geert
On 03/14/2014 03:25 PM, John Chilton wrote:
I am not sure I understand what you mean, but if you are setting the
job to run with a ppn of greater than one but GALAXY_SLOTS is being
evaluated as 1 then you have likely uncovered a bug - maybe in the
runner, your job configuration, or the GALAXY_SLOTS logic for
pbs/torque.
I have created a Galaxy tool for debugging the problem -
https://gist.github.com/jmchilton/9548516. If you could run in your
environment and assign it multiple cores it should spit out some
interesting information relevant to debugging the problem. It will
create Galaxy datasets corresponding to the runtime-computed Galaxy
slots, the contents of PBS_NODEFILE which Galaxy attempts to use to
compute Galaxy slots, and the full contents of your worker node
environment. This last one may contain sensitive information - but if
you could post the first two or send them to me directly it would
hopefully help debug the problem.
-John
On Wed, Mar 12, 2014 at 12:37 PM, Geert Vandeweyer
wrote:
How would those statements translate to pbs/torque (pbs_python)
I request resources using the nodes=1:ppn:4 syntax. The -pe argument is not
available it seems...
Best,
Geert
On 03/12/2014 04:43 PM, Björn Grüning wrote:
Hi Geert,
you can give every tool a different amount of SLOTS via the job_conf.xml
file.
https://wiki.galaxyproject.org/Admin/Config/Jobs
For example:
where 12cores_24G is defined with a parallel SGE environment.
Cheers,
Bjoern
Am 12.03.2014 16:32, schrieb Geert Vandeweyer:
Hi,
Is there documentation for the proper setup and utilisation of the
\${GALAXY_SLOTS:-4} style options?
I noticed that tools with this setting, including BWA and GATK run
single threaded, with the following settings respectively:
BWA \${GALAXY_SLOTS:-4}"
===
ps output: python
/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/b4427dbb6ced/bwa_wrappers/bwa_wrapper.py
--threads=1 --fileSource=indexed
--ref=/galaxy/galaxy_references/hg19/bwa-0.5.9/base/hg19.fasta
--do_not_build_index
--input1=/galaxy/galaxy-dist/database/files/093/dataset_93021.dat
--input2=/galaxy/galaxy-dist/database/files/093/dataset_93023.dat
--output=/galaxy/galaxy-dist/database/files/000/112/dataset_112301.dat
--genAlignType=paired --params=pre_set --suppressHeader=false
xml config: --threads="\${GALAXY_SLOTS:-4}"
GATK Base Recalibrator \${GALAXY_SLOTS:-8}
==
python
/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/gatk2/8bcc13094767/gatk2/gatk2_wrapper.py
--max_jvm_heap 6g --stdout
/galaxy/galaxy-dist/database/files/000/112/dataset_112292.dat -d -I
/galaxy/galaxy-dist/database/files/000/112/dataset_112289.dat bam
gatk_input -d
/galaxy/galaxy-dist/database/files/_metadata_files/012/metadata_12686.dat
bam_index
gatk_input -p java -jar
"$GATK2_PATH/GenomeAnalysisTK.jar"-T "BaseRecalibrator"
$GATK2_SITE_OPTIONS --num_cpu_threads_per_data_thread
${GALAXY_SLOTS:-8} --no_standard_covs -R
"/galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta" --out
"/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat" -cov
"ContextCovariate" -cov "CycleCovariate" -d
--knownSites:dbsnp,%(file_type)s
/galaxy/galaxy-dist/database/files/000/101/dataset_101086.dat vcf
input_dbsnp_0 -p --pedigreeValidationType "STRICT" -d --intervals
/galaxy/galaxy-dist/database/files/000/100/dataset_100129.dat bed
input_intervals_0 -p --interval_set_rule "UNION" -p --downsampling_type
"NONE" -p --baq "OFF" --baqGapOpenPenalty
"40.0" --defaultBaseQualities "-1"
--validation_strictness "STRICT" --interval_merging "ALL"
java -Xmx6g -jar
/galaxy/galaxy-dist/tool-data/shared/jars/gatk2//GenomeAnalysisTK.jar -T
BaseRecalibrator --num_cpu_threads_per_data_thread 1 --no_standard_covs
-R /galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta --out
/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat -cov
ContextCovariate -cov CycleCovariate --pedigreeValidationType STRICT
--interval_set_rule UNION --downsampling_type NONE --baq OFF
--baqGapOpenPenalty 40.0 --defaultBaseQualities -1
--validation_strictness STRICT --interval_merging ALL -I
/tmp/tmp-gatk-fIuncH/gatk_input.bam --knownSites:dbsnp,vcf
/tmp/tmp-gatk-fIuncH/input_dbsnp_0.vcf --intervals
/tmp/tmp-gatk-fIuncH/input_intervals_0.bed
Best,
Geert
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/gee
Re: [galaxy-dev] How To properly use GALAXY_SLOTS
How would those statements translate to pbs/torque (pbs_python)
I request resources using the nodes=1:ppn:4 syntax. The -pe argument is
not available it seems...
Best,
Geert
On 03/12/2014 04:43 PM, Björn Grüning wrote:
Hi Geert,
you can give every tool a different amount of SLOTS via the
job_conf.xml file.
https://wiki.galaxyproject.org/Admin/Config/Jobs
For example:
id="toolshed.g2.bx.psu.edu/repos/iuc/gatk2/gatk2_variant_recalibrator"
destination="12cores_24G" />
where 12cores_24G is defined with a parallel SGE environment.
Cheers,
Bjoern
Am 12.03.2014 16:32, schrieb Geert Vandeweyer:
Hi,
Is there documentation for the proper setup and utilisation of the
\${GALAXY_SLOTS:-4} style options?
I noticed that tools with this setting, including BWA and GATK run
single threaded, with the following settings respectively:
BWA \${GALAXY_SLOTS:-4}"
===
ps output: python
/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/b4427dbb6ced/bwa_wrappers/bwa_wrapper.py
--threads=1 --fileSource=indexed
--ref=/galaxy/galaxy_references/hg19/bwa-0.5.9/base/hg19.fasta
--do_not_build_index
--input1=/galaxy/galaxy-dist/database/files/093/dataset_93021.dat
--input2=/galaxy/galaxy-dist/database/files/093/dataset_93023.dat
--output=/galaxy/galaxy-dist/database/files/000/112/dataset_112301.dat
--genAlignType=paired --params=pre_set --suppressHeader=false
xml config: --threads="\${GALAXY_SLOTS:-4}"
GATK Base Recalibrator \${GALAXY_SLOTS:-8}
==
python
/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/gatk2/8bcc13094767/gatk2/gatk2_wrapper.py
--max_jvm_heap 6g --stdout
/galaxy/galaxy-dist/database/files/000/112/dataset_112292.dat -d -I
/galaxy/galaxy-dist/database/files/000/112/dataset_112289.dat bam
gatk_input -d
/galaxy/galaxy-dist/database/files/_metadata_files/012/metadata_12686.dat
bam_index
gatk_input -p java -jar
"$GATK2_PATH/GenomeAnalysisTK.jar"-T "BaseRecalibrator"
$GATK2_SITE_OPTIONS --num_cpu_threads_per_data_thread
${GALAXY_SLOTS:-8} --no_standard_covs -R
"/galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta" --out
"/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat" -cov
"ContextCovariate" -cov "CycleCovariate" -d
--knownSites:dbsnp,%(file_type)s
/galaxy/galaxy-dist/database/files/000/101/dataset_101086.dat vcf
input_dbsnp_0 -p --pedigreeValidationType "STRICT" -d --intervals
/galaxy/galaxy-dist/database/files/000/100/dataset_100129.dat bed
input_intervals_0 -p --interval_set_rule "UNION" -p --downsampling_type
"NONE" -p --baq "OFF" --baqGapOpenPenalty
"40.0" --defaultBaseQualities "-1"
--validation_strictness "STRICT" --interval_merging "ALL"
java -Xmx6g -jar
/galaxy/galaxy-dist/tool-data/shared/jars/gatk2//GenomeAnalysisTK.jar -T
BaseRecalibrator --num_cpu_threads_per_data_thread 1 --no_standard_covs
-R /galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta --out
/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat -cov
ContextCovariate -cov CycleCovariate --pedigreeValidationType STRICT
--interval_set_rule UNION --downsampling_type NONE --baq OFF
--baqGapOpenPenalty 40.0 --defaultBaseQualities -1
--validation_strictness STRICT --interval_merging ALL -I
/tmp/tmp-gatk-fIuncH/gatk_input.bam --knownSites:dbsnp,vcf
/tmp/tmp-gatk-fIuncH/input_dbsnp_0.vcf --intervals
/tmp/tmp-gatk-fIuncH/input_intervals_0.bed
Best,
Geert
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/geertvandeweyer
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] How To properly use GALAXY_SLOTS
Hi,
Is there documentation for the proper setup and utilisation of the
\${GALAXY_SLOTS:-4} style options?
I noticed that tools with this setting, including BWA and GATK run
single threaded, with the following settings respectively:
BWA \${GALAXY_SLOTS:-4}"
===
ps output: python
/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/b4427dbb6ced/bwa_wrappers/bwa_wrapper.py
--threads=1 --fileSource=indexed
--ref=/galaxy/galaxy_references/hg19/bwa-0.5.9/base/hg19.fasta
--do_not_build_index
--input1=/galaxy/galaxy-dist/database/files/093/dataset_93021.dat
--input2=/galaxy/galaxy-dist/database/files/093/dataset_93023.dat
--output=/galaxy/galaxy-dist/database/files/000/112/dataset_112301.dat
--genAlignType=paired --params=pre_set --suppressHeader=false
xml config: --threads="\${GALAXY_SLOTS:-4}"
GATK Base Recalibrator \${GALAXY_SLOTS:-8}
==
python
/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/gatk2/8bcc13094767/gatk2/gatk2_wrapper.py
--max_jvm_heap 6g --stdout
/galaxy/galaxy-dist/database/files/000/112/dataset_112292.dat -d -I
/galaxy/galaxy-dist/database/files/000/112/dataset_112289.dat bam
gatk_input -d
/galaxy/galaxy-dist/database/files/_metadata_files/012/metadata_12686.dat bam_index
gatk_input -p java -jar
"$GATK2_PATH/GenomeAnalysisTK.jar"-T "BaseRecalibrator"
$GATK2_SITE_OPTIONS --num_cpu_threads_per_data_thread
${GALAXY_SLOTS:-8} --no_standard_covs -R
"/galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta" --out
"/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat" -cov
"ContextCovariate" -cov "CycleCovariate" -d
--knownSites:dbsnp,%(file_type)s
/galaxy/galaxy-dist/database/files/000/101/dataset_101086.dat vcf
input_dbsnp_0 -p --pedigreeValidationType "STRICT" -d --intervals
/galaxy/galaxy-dist/database/files/000/100/dataset_100129.dat bed
input_intervals_0 -p --interval_set_rule "UNION" -p --downsampling_type
"NONE" -p --baq "OFF" --baqGapOpenPenalty
"40.0" --defaultBaseQualities "-1"
--validation_strictness "STRICT" --interval_merging "ALL"
java -Xmx6g -jar
/galaxy/galaxy-dist/tool-data/shared/jars/gatk2//GenomeAnalysisTK.jar -T
BaseRecalibrator --num_cpu_threads_per_data_thread 1 --no_standard_covs
-R /galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta --out
/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat -cov
ContextCovariate -cov CycleCovariate --pedigreeValidationType STRICT
--interval_set_rule UNION --downsampling_type NONE --baq OFF
--baqGapOpenPenalty 40.0 --defaultBaseQualities -1
--validation_strictness STRICT --interval_merging ALL -I
/tmp/tmp-gatk-fIuncH/gatk_input.bam --knownSites:dbsnp,vcf
/tmp/tmp-gatk-fIuncH/input_dbsnp_0.vcf --intervals
/tmp/tmp-gatk-fIuncH/input_intervals_0.bed
Best,
Geert
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/geertvandeweyer
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] libpng (devteam) install fails
Hi Bjoern & devteam members, I tried to install R_3_0_2 iuc version from the main toolshed, but got an error from the package_libpng from the devteam. I checked between the working tool_dependency from the testtoolshed and the failing one from the main. working: type="download_by_url">http://downloads.sourceforge.net/project/libpng/libpng16/older-releases/1.6.7/libpng-1.6.7.tar.gz failing : type="download_by_url">http://downloads.sourceforge.net/project/libpng/libpng16/1.6.7/libpng-1.6.7.tar.gz The package_libpng from the devteam is downloading a different version on test and main. Best, Geert On 02/20/2014 12:56 PM, Björn Grüning wrote: Hello, Hi, Compilation/installing works now. Using this R from the commandline no longer complains about missing X. With regards to cairo, I don't think I explicitly need it, but as I understood it, it was used in case there was no X11 server on the system? Not sure, I will add it if anyone complains :) would you add it to the main toolshed? Then I can put is as a dependency on my tools that create plots from R (coverage_report), which has now the system version hardcoded. Done! Thanks for testing! Bjoern Best, Geert On 02/20/2014 01:45 AM, Björn Grüning wrote: Hi Geert, can you retest please, I had a typo. Unfortunately, I can't test it by myself since my internet connection is to bad here in Slovenia to download that R-tarball. Sorry, Bjoern hi, The dependencies installed fine, but R itself gave this error: configure: error: unrecognized option: `-I/galaxy/galaxy_tool_binaries/libpng/1.6.7/devteam/package_libpng_1_6_7/a0b0e0281cc4/include' Try `./configure --help' for more information Any idea what might be wrong? Best, Geert On 02/19/2014 12:55 PM, Björn Grüning wrote: Hi Geert, I tried to update the R package to include the libpng dependency, if you like to test it ... http://testtoolshed.g2.bx.psu.edu/view/iuc/package_r_3_0_2 Do you know if cairo is really needed? If I can I would omit to have a dependency on cairo :) Cheers, Bjoern Hi all, I installed to package_r_2.11 and r_3x from the devteam to use as tool dependency. All installation goes fine, but when I run my tool I get the following error: Error in png(file = "../Plots/outname.png", bg = "white", width = 480, : X11 is not available The tool uses the installed R : /galaxy/galaxy_tool_binaries/R/2.11.0/devteam/package_r_2_11_0/8d0a55bf7aaf/lib/R/bin/R If I force the tool to use the system installed version of R, it works fine. Are there packages available in the toolsheds that allow the 'png' function to work? I already tried the R-packages from boris, but they gave compilation errors... I also tried to install the R-png package from cran as a galaxy dependency, but that didn't help either. Thanks for the help, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail:geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail:geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] package_r_xx problem: X11 is not available
Hi, Compilation/installing works now. Using this R from the commandline no longer complains about missing X. With regards to cairo, I don't think I explicitly need it, but as I understood it, it was used in case there was no X11 server on the system? would you add it to the main toolshed? Then I can put is as a dependency on my tools that create plots from R (coverage_report), which has now the system version hardcoded. Best, Geert On 02/20/2014 01:45 AM, Björn Grüning wrote: Hi Geert, can you retest please, I had a typo. Unfortunately, I can't test it by myself since my internet connection is to bad here in Slovenia to download that R-tarball. Sorry, Bjoern hi, The dependencies installed fine, but R itself gave this error: configure: error: unrecognized option: `-I/galaxy/galaxy_tool_binaries/libpng/1.6.7/devteam/package_libpng_1_6_7/a0b0e0281cc4/include' Try `./configure --help' for more information Any idea what might be wrong? Best, Geert On 02/19/2014 12:55 PM, Björn Grüning wrote: Hi Geert, I tried to update the R package to include the libpng dependency, if you like to test it ... http://testtoolshed.g2.bx.psu.edu/view/iuc/package_r_3_0_2 Do you know if cairo is really needed? If I can I would omit to have a dependency on cairo :) Cheers, Bjoern Hi all, I installed to package_r_2.11 and r_3x from the devteam to use as tool dependency. All installation goes fine, but when I run my tool I get the following error: Error in png(file = "../Plots/outname.png", bg = "white", width = 480, : X11 is not available The tool uses the installed R : /galaxy/galaxy_tool_binaries/R/2.11.0/devteam/package_r_2_11_0/8d0a55bf7aaf/lib/R/bin/R If I force the tool to use the system installed version of R, it works fine. Are there packages available in the toolsheds that allow the 'png' function to work? I already tried the R-packages from boris, but they gave compilation errors... I also tried to install the R-png package from cran as a galaxy dependency, but that didn't help either. Thanks for the help, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail:geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] package_r_xx problem: X11 is not available
hi, The dependencies installed fine, but R itself gave this error: configure: error: unrecognized option: `-I/galaxy/galaxy_tool_binaries/libpng/1.6.7/devteam/package_libpng_1_6_7/a0b0e0281cc4/include' Try `./configure --help' for more information Any idea what might be wrong? Best, Geert On 02/19/2014 12:55 PM, Björn Grüning wrote: Hi Geert, I tried to update the R package to include the libpng dependency, if you like to test it ... http://testtoolshed.g2.bx.psu.edu/view/iuc/package_r_3_0_2 Do you know if cairo is really needed? If I can I would omit to have a dependency on cairo :) Cheers, Bjoern Hi all, I installed to package_r_2.11 and r_3x from the devteam to use as tool dependency. All installation goes fine, but when I run my tool I get the following error: Error in png(file = "../Plots/outname.png", bg = "white", width = 480, : X11 is not available The tool uses the installed R : /galaxy/galaxy_tool_binaries/R/2.11.0/devteam/package_r_2_11_0/8d0a55bf7aaf/lib/R/bin/R If I force the tool to use the system installed version of R, it works fine. Are there packages available in the toolsheds that allow the 'png' function to work? I already tried the R-packages from boris, but they gave compilation errors... I also tried to install the R-png package from cran as a galaxy dependency, but that didn't help either. Thanks for the help, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] package_r_xx problem: X11 is not available
Hi all, I installed to package_r_2.11 and r_3x from the devteam to use as tool dependency. All installation goes fine, but when I run my tool I get the following error: Error in png(file = "../Plots/outname.png", bg = "white", width = 480, : X11 is not available The tool uses the installed R : /galaxy/galaxy_tool_binaries/R/2.11.0/devteam/package_r_2_11_0/8d0a55bf7aaf/lib/R/bin/R If I force the tool to use the system installed version of R, it works fine. Are there packages available in the toolsheds that allow the 'png' function to work? I already tried the R-packages from boris, but they gave compilation errors... I also tried to install the R-png package from cran as a galaxy dependency, but that didn't help either. Thanks for the help, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] migrate_tools error with latest release (hg update release_2014.02.10 )
Hi Dave,
I was able to get galaxy back up by doing a fresh install, moving the
old database folder in the new install, reconfiguring universe and
job_conf, and leaving tool_conf and all shed related items to default. I
also truncated the tables 'tool_dependency', tool_shed_repository and
tool_version associates. Finally I removed in tool_version all entries
that had a tool_shed_repository_id !== NULL.
Now I'm installing all my toolsheds again, and it's still working it seems.
If you want, I can try to make my database dump (post-error, but before
truncating) available to you.
Best,
Geert
On 02/11/2014 10:56 PM, Dave Bouvier wrote:
Geert,
I have not yet been able to reproduce that error in my local Galaxy
installation running on the release_2014.02.10 tag, but I am
continuing to investigate. Are you able to determine which repository
last succeeded in the migration script output?
--Dave B.
On 02/11/2014 10:29 AM, Geert Vandeweyer wrote:
This message is also returned when I try to start the server without the
migrate tools script (but after the initial error).
Can I safely revert to a previous rel
Best,
Geert
On 02/11/2014 03:18 PM, Geert Vandeweyer wrote:
Hi,
I'm having problems upgrading Galaxy to the latest release. The
migrate tools script gives the following error.
No handlers could be found for logger "galaxy.tools"
Traceback (most recent call last):
File "./scripts/migrate_tools/migrate_tools.py", line 21, in
app = MigrateToolsApplication( sys.argv[ 1 ] )
File
"/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/migrate/common.py",
line 45, in __init__
self.installed_repository_manager =
installed_repository_manager.InstalledRepositoryManager( self )
File
"/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/installed_repository_manager.py",
line 66, in __init__
self.load_dependency_relationships()
File
"/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/installed_repository_manager.py",
line 269, in load_dependency_relationships
self.add_entry_to_runtime_tool_dependencies_of_installed_tool_dependencies(
tool_dependency )
File
"/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/installed_repository_manager.py",
line 147, in
add_entry_to_runtime_tool_dependencies_of_installed_tool_dependencies
tool_dependency_util.get_runtime_dependent_tool_dependency_tuples(
self.app, tool_dependency, status=None )
File
"/galaxy/galaxy-dist/lib/tool_shed/util/tool_dependency_util.py", line
323, in get_runtime_dependent_tool_dependency_tuples
env_shell_file_path = td.get_env_shell_file_path( app )
File
"/galaxy/galaxy-dist/lib/galaxy/model/tool_shed_install/__init__.py",
line 533, in get_env_shell_file_path
installation_directory = self.installation_directory( app )
File
"/galaxy/galaxy-dist/lib/galaxy/model/tool_shed_install/__init__.py",
line 548, in installation_directory
self.tool_shed_repository.owner,
AttributeError: 'NoneType' object has no attribute 'owner'
Any help on how to resolve this?
The 0009_tools.xml content is :
id="compute_p-values_correlation_coefficients_feature_occurrences_between_two_datasets_using_discrete_wavelet_transfom"
version="1.0.0" file="execute_dwt_cor_aVa_perClass.xml" />
id="compute_p-values_correlation_coefficients_featureA_featureB_occurrences_between_two_datasets_using_discrete_wavelet_transfom"
version="1.0.0" file="execute_dwt_cor_aVb_all.xml" />
id="compute_p-values_second_moments_feature_occurrences_between_two_datasets_using_discrete_wavelet_transfom"
version="1.0.0" file="execute_dwt_IvC_all.xml" />
id="compute_p-values_max_variances_feature_occurrences_in_one_dataset_using_discrete_wavelet_transfom"
version="1.0.0" file="execute_dwt_var_perClass.xml" />
file="express_wrapper.xml" />
file="fastq_stats.xml" />
Best,
Geert
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
Univer
Re: [galaxy-dev] migrate_tools error with latest release (hg update release_2014.02.10 )
This message is also returned when I try to start the server without the migrate tools script (but after the initial error). Can I safely revert to a previous release somehow? Best, Geert On 02/11/2014 03:18 PM, Geert Vandeweyer wrote: Hi, I'm having problems upgrading Galaxy to the latest release. The migrate tools script gives the following error. No handlers could be found for logger "galaxy.tools" Traceback (most recent call last): File "./scripts/migrate_tools/migrate_tools.py", line 21, in app = MigrateToolsApplication( sys.argv[ 1 ] ) File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/migrate/common.py", line 45, in __init__ self.installed_repository_manager = installed_repository_manager.InstalledRepositoryManager( self ) File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/installed_repository_manager.py", line 66, in __init__ self.load_dependency_relationships() File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/installed_repository_manager.py", line 269, in load_dependency_relationships self.add_entry_to_runtime_tool_dependencies_of_installed_tool_dependencies( tool_dependency ) File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/installed_repository_manager.py", line 147, in add_entry_to_runtime_tool_dependencies_of_installed_tool_dependencies tool_dependency_util.get_runtime_dependent_tool_dependency_tuples( self.app, tool_dependency, status=None ) File "/galaxy/galaxy-dist/lib/tool_shed/util/tool_dependency_util.py", line 323, in get_runtime_dependent_tool_dependency_tuples env_shell_file_path = td.get_env_shell_file_path( app ) File "/galaxy/galaxy-dist/lib/galaxy/model/tool_shed_install/__init__.py", line 533, in get_env_shell_file_path installation_directory = self.installation_directory( app ) File "/galaxy/galaxy-dist/lib/galaxy/model/tool_shed_install/__init__.py", line 548, in installation_directory self.tool_shed_repository.owner, AttributeError: 'NoneType' object has no attribute 'owner' Any help on how to resolve this? The 0009_tools.xml content is : name="bowtie2" description="Bowtie2"> name="ccat" description="Control-based ChIP-seq Analysis Tool"> file="ccat_wrapper.xml" /> name="clustalw" description="ClustalW multiple sequence alignment program for DNA or proteins"> name="dwt_cor_ava_perclass" description="Compute P-values and Correlation Coefficients for Feature Occurrences"> id="compute_p-values_correlation_coefficients_feature_occurrences_between_two_datasets_using_discrete_wavelet_transfom" version="1.0.0" file="execute_dwt_cor_aVa_perClass.xml" /> name="dwt_cor_avb_all" description="Compute P-values and Correlation Coefficients for Occurrences of Two Set of Features"> id="compute_p-values_correlation_coefficients_featureA_featureB_occurrences_between_two_datasets_using_discrete_wavelet_transfom" version="1.0.0" file="execute_dwt_cor_aVb_all.xml" /> name="dwt_ivc_all" description="Compute P-values and Second Moments for Feature Occurrences"> id="compute_p-values_second_moments_feature_occurrences_between_two_datasets_using_discrete_wavelet_transfom" version="1.0.0" file="execute_dwt_IvC_all.xml" /> name="dwt_var_perclass" description="Compute P-values and Max Variances for Feature Occurrences"> id="compute_p-values_max_variances_feature_occurrences_in_one_dataset_using_discrete_wavelet_transfom" version="1.0.0" file="execute_dwt_var_perClass.xml" /> name="dwt_var_perfeature" description="Wavelet variance using Discrete Wavelet Transfoms"> file="execute_dwt_var_perFeature.xml" /> name="express" description="Quantify the abundances of a set of target sequences from sampled subsequences"> name="fastqc" description="Read QC reports using FastQC"> name="fastq_combiner" description="Combine FASTA and QUAL into FASTQ."> file="fastq_combiner.xml" /> name="fastq_filter" description="Filter FASTQ reads by quality score and length"> file="fastq_filter.xml" /> name="fastq_groomer" description="Convert between various FASTQ quality formats."> file="fastq_groomer.xml" /> name="fastq_manipulation" description="Man
[galaxy-dev] migrate_tools error with latest release (hg update release_2014.02.10 )
fastq_paired_end_deinterlacer" description="FASTQ de-interlacer on paired end reads."> file="fastq_paired_end_deinterlacer.xml" /> name="fastq_paired_end_interlacer" description="FASTQ interlacer on paired end reads"> file="fastq_paired_end_interlacer.xml" /> name="fastq_paired_end_joiner" description="FASTQ joiner on paired end reads"> file="fastq_paired_end_joiner.xml" /> name="fastq_paired_end_splitter" description="FASTQ splitter on joined paired end reads"> file="fastq_paired_end_splitter.xml" /> name="fastq_stats" description="FASTQ Summary Statistics by column"> name="fastqtofasta" description="FASTQ to FASTA converter"> file="fastq_to_fasta.xml" /> name="fastq_to_tabular" description="FASTQ to Tabular converter"> file="fastq_to_tabular.xml" /> name="fastq_trimmer" description="FASTQ Trimmer by quality"> file="fastq_trimmer.xml" /> name="fastq_trimmer_by_quality" description="FASTQ Quality Trimmer by sliding window"> file="fastq_trimmer_by_quality.xml" /> name="filter_transcripts_via_tracking" description="Filter Combined Transcripts"> file="filter_transcripts_via_tracking.xml" /> name="find_diag_hits" description="Identify sequence reads corresponding to a particular taxonomic group"> file="find_diag_hits.xml" /> name="freebayes_wrapper" description="Call SNPS with Freebayes"> file="freebayes.xml" /> name="gi2taxonomy" description="Fetch taxonomic representation"> file="gi2taxonomy.xml" /> name="gmaj" description="GMAJ Multiple Alignment Viewer"> name="lca_wrapper" description="Find lowest diagnostic rank"> name="macs" description="Model-based Analysis of ChIP-Seq"> file="macs_wrapper.xml" /> name="poisson2test" description="Poisson two-sample test"> name="sicer" description="Statistical approach for the Identification of ChIP-Enriched Regions"> file="sicer_wrapper.xml" /> name="t2ps" description="Draw phylogeny"> file="t2ps_wrapper.xml" /> name="t2t_report" description="Summarize taxonomy"> name="tabular_to_fastq" description="Tabular to FASTQ converter"> file="tabular_to_fastq.xml" /> name="tophat" description="Find splice junctions using RNA-seq data"> name="tophat2" description="Gapped-read mapper for RNA-seq data"> name="vcf_annotate" description="Annotate a VCF file (dbSNP, hapmap)"> name="vcf_extract" description="Extract reads from a specified region"> name="vcf_filter" description="Filter a VCF file"> name="vcf_intersect" description="Generate the intersection of two VCF files"> file="vcf_tools/intersect.xml" /> name="weblogo3" description="Sequence Logo generator for fasta (eg Clustal alignments)"> Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] filter tag in output, multiple criteria
Hi, Thanks for the quick suggestions. The typo didn't exists in the original code, so that didn't solve it unfortunately. I fixed it for now by using the negative filter (!=), suggested by Peter. Best, Geert On 01/31/2014 12:09 PM, Nicola Soranzo wrote: Il 2014-01-31 11:52 Geert Vandeweyer ha scritto: Hi, I'm looking for the correct syntax to achieve the following: outtype == "0" || outttype == "2" outtype == "1" || outttype == "2" I tried the above, with and without parentheses, but it creates the second dataset also when outtype == 0. Is this type of dual filtering possible? I don't know if this is the reason, but you seem to have a typo: outttype == "2" should be outtype == "2" Best, Nicola ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] filter tag in output, multiple criteria
Hi, I'm looking for the correct syntax to achieve the following: outtype == "0" || outttype == "2" outtype == "1" || outttype == "2" I tried the above, with and without parentheses, but it creates the second dataset also when outtype == 0. Is this type of dual filtering possible? Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Status of GATK in Galaxy
Hi Bjoern,
I'm using the full GATK version 2.4-9-g532efad in galaxy.
Best,
Geert
On 11/19/2013 08:28 PM, Björn Grüning wrote:
Hi Geert,
thanks for the information. Are you running GATK-2.4 or GATK-Lite-2.4.
As far I know the lite version is deprecated.
JJ are your wrappers in bitbucket/github? If you agree I would try to
migrate to a recent GATK version and implement GALAXY_SLOTS. Not sure
about the installation recipe ...
Ciao,
Bjoern
Hi,
I have GATK 2.4 running in our local galaxy, next to the default galaxy
1.x version. Both are in the tool menu, but only users that have
provided a licence or proof that they are from an academic site have
access to the 2.4 version. This is based on a usergroup check in a
dynamic job runner configuration.
I used & modified the GATK 2 tool configuration from the test toolshed
(by jjohnson). Modifications to the xmls included:
--max_jvm_heap "xG" instead of --max_jvm_heap_fraction "1" : we
use shared nodes, so it's not allowed to use all ram.
--num_threads : some hard coded thread settings were changed
made sure the '--et 'NO_ET' is commented out, as we don't have a key.
relevant dynamic_runner part:
from galaxy.jobs.mapper import JobMappingException
if 'gatk2' in tool_id >= 0:
user_group_assocs = user.groups or []
user_has_license = 'gatk2_license' in
[user_group_assoc.group.name for user_group_assoc in user_group_assocs]
if not user_has_license:
raise JobMappingException("GATKv2 is
only available to non-profit, academic users. Please send a mail to
geert.vandewey...@uantwerpen.be to confirm this.")
Best,
Geert
On 11/19/2013 12:26 PM, Bjoern Gruening wrote:
Hi,
I found that the GATK tools do not have the same requirement tag. For
example indel-realigner has version 1.4, realigner_target_creator has
version 1.3. Is that correct? What is the recent version that is
required for GATK?
Also realigner_target_creator uses the "gatk_picard_indexes" location
file, but under tool-data is only a gatk_sorted_picard_indexes.loc
file.
Are there any concrete plans for the GATK wrappers. Especially to update
to a newer version. Or is that not feasible because of this complicated
license policy and the phone-home option ...
I found the following trello card, but its a little bit outdated:
https://trello.com/c/IPkT2spd
Is anyone running a recent GATK version in Galaxy?
Thanks!
Bjoern
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/geertvandeweyer
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Status of GATK in Galaxy
Hi,
I have GATK 2.4 running in our local galaxy, next to the default galaxy
1.x version. Both are in the tool menu, but only users that have
provided a licence or proof that they are from an academic site have
access to the 2.4 version. This is based on a usergroup check in a
dynamic job runner configuration.
I used & modified the GATK 2 tool configuration from the test toolshed
(by jjohnson). Modifications to the xmls included:
--max_jvm_heap "xG" instead of --max_jvm_heap_fraction "1" : we
use shared nodes, so it's not allowed to use all ram.
--num_threads : some hard coded thread settings were changed
made sure the '--et 'NO_ET' is commented out, as we don't have a key.
relevant dynamic_runner part:
from galaxy.jobs.mapper import JobMappingException
if 'gatk2' in tool_id >= 0:
user_group_assocs = user.groups or []
user_has_license = 'gatk2_license' in
[user_group_assoc.group.name for user_group_assoc in user_group_assocs]
if not user_has_license:
raise JobMappingException("GATKv2 is
only available to non-profit, academic users. Please send a mail to
geert.vandewey...@uantwerpen.be to confirm this.")
Best,
Geert
On 11/19/2013 12:26 PM, Bjoern Gruening wrote:
Hi,
I found that the GATK tools do not have the same requirement tag. For
example indel-realigner has version 1.4, realigner_target_creator has
version 1.3. Is that correct? What is the recent version that is
required for GATK?
Also realigner_target_creator uses the "gatk_picard_indexes" location
file, but under tool-data is only a gatk_sorted_picard_indexes.loc
file.
Are there any concrete plans for the GATK wrappers. Especially to update
to a newer version. Or is that not feasible because of this complicated
license policy and the phone-home option ...
I found the following trello card, but its a little bit outdated:
https://trello.com/c/IPkT2spd
Is anyone running a recent GATK version in Galaxy?
Thanks!
Bjoern
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/geertvandeweyer
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] toolshed contact not working (varscan)
Hi, I tried to contact the owner of the varscan repository through the main toolshed ( http://toolshed.g2.bx.psu.edu/) "repository actions, contact owner function. Upon submission I got the following error : An error occurred sending your message by email: 'Configuration' object has no attribute 'smtp_ssl' Is this a known issue? The option is not available if I browse the repository from my local galaxy admin pages. In case fcaramia reads this, please contact me, I have some updates to your varscan somatics script, as the default scripts failed to run for me. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] error checking active jobs : KeyError: 'destination'
Dear all,
This error persists in the August release.
To reproduce:
- start workflows (and/or jobs)
- while jobs are running and/or waiting to run : restart galaxy
handler will now give these errors. Deleting the running/waiting jobs
and restarting galaxy resolves this.
Best,
Geert
On 04/03/2013 11:38 AM, Geert Vandeweyer wrote:
I was able to solve this by setting job state in the database from
queued and running to "failed" for two jobs that had a wrong state
after galaxy-restart.
best,
geert
On 04/02/2013 05:27 PM, Geert Vandeweyer wrote:
Hi,
I'm getting errors in the handler logs after upgrading galaxy stable
to today's release.
I specified the "dynamic" job runner as default, which returns a job
for pbs.
These errors appear as soon as the server is up and running. What
could be the issue here?
galaxy.jobs.runners ERROR 2013-04-02 17:24:23,411 Unhandled exception
checking active jobs
Traceback (most recent call last):
File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py",
line 336, in monitor
self.check_watched_items()
File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line
382, in check_watched_items
( failures, statuses ) = self.check_all_jobs()
File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line
463, in check_all_jobs
pbs_server_name =
self.__get_pbs_server(pbs_job_state.job_destination.params)
File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line
221, in __get_pbs_server
return job_destination_params['destination'].split('@')[-1]
KeyError: 'destination'
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Map with BWA results in correpted SAM file
to add on this : we have similar issues (sam-to-bam conversion fails with similar errors). it seems to be related to the BWA output getting messed up, with (part of) columns missing or duplicated on some lines. I have not found a systematic pattern in the errors, they seem to happen rather random. On 08/08/2013 05:06 PM, Moritz Juchler wrote: Dear Galaxy Community, I have a local instance and installed 0.5.9-r16 BWA and the toolshed wrapper. The mapping is successful. I then use the Filter Sam Tool on the sam file from the alignment, but it spits out this error: *Dataset 26: Filter SAM on data 24* Tool execution generated the following error message: Traceback (most recent call last): File "/home/trr/galaxy-dist/tools/samtools/sam_bitwise_flag_filter.py", line 148, in if __name__ == "__main__": main() File "/home/trr/galaxy-dist/tools/samtools/sam_bitwise_flag_filter.py", line 137, in main flags = int( fields[flag_col] ) ValueError: invalid literal for int() with base 10: 'RG:Z:lane712s006433' I have the same workflow online and did the exact same steps on the same fastq files. Is there anything I am missing? Is there any information I can provide to answer this question? Best Moritz ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] shed tool dependencies
Hi, Is there a default notation to specify perl modules (threading modules) in a tool configuration or in a submission to the toolshed? I've got some tools to share that make extensive use of these modules, and they are not default in a perl distribution. Hence, I'd like to inform the users that they need to install these modules (or that they get installed by the the toolshed using cpan ?) Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] workflow startup issues
Aside: If I add steps to the beginning of the workfow, the step that
runs too early, typically occurs earlier in the flow. Can this indicate
a race condition where not all dependencies are available (stored?) yet
in the database at the time of querying for job eligible for queueing?
Best,
Geert
On 05/21/2013 11:06 PM, Geert Vandeweyer wrote:
Hi Dannon,
I pulled the new updates and restarted galaxy. However, the issue is
not resolved. Is there other information you might need?
hg summary:
parent: 9332:0f51ae4f1c84 tip
Unicode fixes for workflow names in Tool Menu.
branch: stable
Best,
Geert
On 05/21/2013 06:08 PM, Dannon Baker wrote:
Ok, so you're running the old tip of -dist. There are a few new
updates you can retrieve using `hg pull -u -b stable
http://bitbucket.org/galaxy/galaxy-central`
<http://bitbucket.org/galaxy/galaxy-central%60>.
-Dannon
On Tue, May 21, 2013 at 12:01 PM, Geert Vandeweyer
mailto:geert.vandewey...@ua.ac.be>> wrote:
hg summary:
parent: 9320:47ddf167c9f1 tip
Use Galaxy's ErrorMiddleware since Paste's doesn't return
start_response. Fixes downloading tarballs from the Tool Shed
when use_debug = false.
branch: stable
commit: 18 modified, 178 unknown
update: (current)
Geert
On 05/21/2013 04:59 PM, Dannon Baker wrote:
Geert,
This should be fixed in the up-to-date stable branch as far as I
know. Can you tell me what 'hg summary' states?
-Dannon
On Mon, May 20, 2013 at 4:43 PM, Geert Vandeweyer
mailto:geert.vandewey...@ua.ac.be>>
wrote:
Hi,
After the latest upgrade (april), I'm having issues when
starting (batch) workfows. Sometimes, it seems that step
dependencies are not stored correctly, leading to some steps
being started too soon, without prior datasets being
completed. Specifically, for some workflows, some of the
later steps start immediately. The issue seems to be random:
in batch workflow submission, some run flawlessly, others
show the issue, at different steps. I use one handler in the
job_conf to exclude race conditions and shared variable clashes.
I suspect there is an issue with fetching or storing the
steps to/from the database (a remote mysql database in my
case). On the mysql-side there are no errors in the log
files, and the galaxy-logs don't show clear issues either.
Does the job new job handler check for failed database
connections? If not, could somebody point me to the code
that performs these steps, I'd like to add a check & retry
routine to see if this improves the situation.
best,
Geert
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
<mailto:geert.vandewe...@ua.ac.be>
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail:geert.vandewe...@ua.ac.be <mailto:geert.vandewe...@ua.ac.be>
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail:geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
__
Re: [galaxy-dev] workflow startup issues
Hi Dannon,
I pulled the new updates and restarted galaxy. However, the issue is not
resolved. Is there other information you might need?
hg summary:
parent: 9332:0f51ae4f1c84 tip
Unicode fixes for workflow names in Tool Menu.
branch: stable
Best,
Geert
On 05/21/2013 06:08 PM, Dannon Baker wrote:
Ok, so you're running the old tip of -dist. There are a few new
updates you can retrieve using `hg pull -u -b stable
http://bitbucket.org/galaxy/galaxy-central`
<http://bitbucket.org/galaxy/galaxy-central%60>.
-Dannon
On Tue, May 21, 2013 at 12:01 PM, Geert Vandeweyer
mailto:geert.vandewey...@ua.ac.be>> wrote:
hg summary:
parent: 9320:47ddf167c9f1 tip
Use Galaxy's ErrorMiddleware since Paste's doesn't return
start_response. Fixes downloading tarballs from the Tool Shed
when use_debug = false.
branch: stable
commit: 18 modified, 178 unknown
update: (current)
Geert
On 05/21/2013 04:59 PM, Dannon Baker wrote:
Geert,
This should be fixed in the up-to-date stable branch as far as I
know. Can you tell me what 'hg summary' states?
-Dannon
On Mon, May 20, 2013 at 4:43 PM, Geert Vandeweyer
mailto:geert.vandewey...@ua.ac.be>>
wrote:
Hi,
After the latest upgrade (april), I'm having issues when
starting (batch) workfows. Sometimes, it seems that step
dependencies are not stored correctly, leading to some steps
being started too soon, without prior datasets being
completed. Specifically, for some workflows, some of the
later steps start immediately. The issue seems to be random:
in batch workflow submission, some run flawlessly, others
show the issue, at different steps. I use one handler in the
job_conf to exclude race conditions and shared variable clashes.
I suspect there is an issue with fetching or storing the
steps to/from the database (a remote mysql database in my
case). On the mysql-side there are no errors in the log
files, and the galaxy-logs don't show clear issues either.
Does the job new job handler check for failed database
connections? If not, could somebody point me to the code that
performs these steps, I'd like to add a check & retry routine
to see if this improves the situation.
best,
Geert
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
<mailto:geert.vandewe...@ua.ac.be>
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail:geert.vandewe...@ua.ac.be <mailto:geert.vandewe...@ua.ac.be>
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] workflow startup issues
hg summary:
parent: 9320:47ddf167c9f1 tip
Use Galaxy's ErrorMiddleware since Paste's doesn't return
start_response. Fixes downloading tarballs from the Tool Shed when
use_debug = false.
branch: stable
commit: 18 modified, 178 unknown
update: (current)
Geert
On 05/21/2013 04:59 PM, Dannon Baker wrote:
Geert,
This should be fixed in the up-to-date stable branch as far as I know.
Can you tell me what 'hg summary' states?
-Dannon
On Mon, May 20, 2013 at 4:43 PM, Geert Vandeweyer
mailto:geert.vandewey...@ua.ac.be>> wrote:
Hi,
After the latest upgrade (april), I'm having issues when starting
(batch) workfows. Sometimes, it seems that step dependencies are
not stored correctly, leading to some steps being started too
soon, without prior datasets being completed. Specifically, for
some workflows, some of the later steps start immediately. The
issue seems to be random: in batch workflow submission, some run
flawlessly, others show the issue, at different steps. I use one
handler in the job_conf to exclude race conditions and shared
variable clashes.
I suspect there is an issue with fetching or storing the steps
to/from the database (a remote mysql database in my case). On the
mysql-side there are no errors in the log files, and the
galaxy-logs don't show clear issues either.
Does the job new job handler check for failed database
connections? If not, could somebody point me to the code that
performs these steps, I'd like to add a check & retry routine to
see if this improves the situation.
best,
Geert
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be <mailto:geert.vandewe...@ua.ac.be>
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] workflow startup issues
Hi, After the latest upgrade (april), I'm having issues when starting (batch) workfows. Sometimes, it seems that step dependencies are not stored correctly, leading to some steps being started too soon, without prior datasets being completed. Specifically, for some workflows, some of the later steps start immediately. The issue seems to be random: in batch workflow submission, some run flawlessly, others show the issue, at different steps. I use one handler in the job_conf to exclude race conditions and shared variable clashes. I suspect there is an issue with fetching or storing the steps to/from the database (a remote mysql database in my case). On the mysql-side there are no errors in the log files, and the galaxy-logs don't show clear issues either. Does the job new job handler check for failed database connections? If not, could somebody point me to the code that performs these steps, I'd like to add a check & retry routine to see if this improves the situation. best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Track Job Runtime
On 05/08/2013 05:38 PM, Geert Vandeweyer wrote:
self.sa_session.execute('UPDATE job SET runtime = :runtime WHERE id =
:id',{'runtime':runtime,'id':galaxy_job_id})
does anybody have a solution to convert this statement to proper
sqlalchemy syntax, for use in the check_watched_items function in pbs.py ?
Regarding Taylor's suggestion: A separate table is also an option, but
would take more queries & joins to estimate walltime at startup (join
table with job table for job type (on job-id), request two rows per
"finished" jobid, substract end-start timestamp, average. An extra
column in the job table only needs one query on one table (select
runtime from jobs where type = 'x' and state = 'ok').
Best,
Geert
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Track Job Runtime
Hi all,
I'm fiddling with this, and I have a proof of principle working for PBS
jobs using very ugly sql-alchemy hack.
Idea is:
- if job goes from queued to running : store seconds since epoch in
'runtime'
- if job goes from running to finished : compare time, store difference
as runtime.
I've created an extra field 'runtime' for holding this info, using
seconds since epoch.
When querying afterwards, one should filter for 'OK' jobs, and discard
jobs that are still running.
right now, i have these statements to add timestamps to the database
(somewhere in the check_watched_items function in pbs.py) :
self.sa_session.execute('UPDATE job SET runtime = :runtime WHERE id =
:id',{'runtime':runtime,'id':galaxy_job_id})
Does anybody know how to translate this to a proper sqlalchemy statement
such as (which does not work):
self.sa_session.query(self.model.Job).filter_by(id=galaxy_job_id).update({"runtime":runtime},synchronize_session=False)
or
sa_session.execute(self.sa_session.Table('job').update().values(runtime=runtime).where(id=galaxy_job_id))
If I can figure this out, I'll try to polish it and create a pull request.
Best,
Geert
On 05/08/2013 03:58 PM, Bossers, Alex wrote:
+1 for me!
Alex
Van: galaxy-dev-boun...@lists.bx.psu.edu [galaxy-dev-boun...@lists.bx.psu.edu]
namens Peter Cock [p.j.a.c...@googlemail.com]
Verzonden: woensdag 8 mei 2013 12:06
To: Geert Vandeweyer
Cc: galaxy-dev@lists.bx.psu.edu
Onderwerp: Re: [galaxy-dev] Track Job Runtime
On Wed, May 8, 2013 at 10:08 AM, Geert Vandeweyer
wrote:
Hi,
Are there options available to track the actual runtime of jobs on a cluster
and store them in the database?
Not yet, but I'd really like to have that information too.
Peter
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Track Job Runtime
Hi, Are there options available to track the actual runtime of jobs on a cluster and store them in the database? Or are there fields in the database that approximate the job execution duration? This might be useful for fine-grained wall-time estimation in a crowded cluster environment. What I'd like to do is fetch an average runtime / mb of input data for a specific tool from the database, and than use this for wall-time estimation of new jobs in a dynamic job runner script. Has this been done before? Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] toolshed down?
hi,
I'm getting the following error when trying to reach the main/test
toolsheds. Are they down?
This webpage is not available
Chromium's connection attempt to toolshed.g2.bx.psu.edu was rejected.
The website may be down, or your network may not be properly configured.
also from within galaxy, I can't reach it, nor can the galaxy-process
itself:
galaxy.model.migrate.check INFO 2013-05-02 12:12:45,219 At database
version 114
tool_shed.galaxy_install.migrate.check DEBUG 2013-05-02 12:12:45,390
pysqlite>=2 egg successfully loaded for sqlite dialect
The URL
http://toolshed.g2.bx.psu.edu/repository/get_tool_dependencies?name=bowtie_wrappers&owner=devteam&changeset_revision=0c7e4eadfb3c&from_install_manager=True
raised the exception:
The URL
http://toolshed.g2.bx.psu.edu/repository/get_tool_dependencies?name=bowtie_color_wrappers&owner=devteam&changeset_revision=fd0914e451c5&from_install_manager=True
raised the exception:
The URL
http://toolshed.g2.bx.psu.edu/repository/get_tool_dependencies?name=lastz&owner=devteam&changeset_revision=0801f8207d30&from_install_manager=True
raised the exception:
The URL
http://toolshed.g2.bx.psu.edu/repository/get_tool_dependencies?name=lastz_paired_reads&owner=devteam&changeset_revision=96825cee5c25&from_install_manager=True
raised the exception:
Traceback (most recent call last):
File
"/galaxy/galaxy-beta/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py",
line 37, in app_factory
app = UniverseApplication( global_conf = global_conf, **kwargs )
File "/galaxy/galaxy-beta/galaxy-dist/lib/galaxy/app.py", line 54, in
__init__
verify_tools( self, db_url, kwargs.get( 'global_conf', {} ).get(
'__file__', None ), self.config.database_engine_options )
File
"/galaxy/galaxy-beta/galaxy-dist/lib/tool_shed/galaxy_install/migrate/check.py",
line 56, in verify_tools
tool_shed_accessible, missing_tool_configs_dict =
common_util.check_for_missing_tools( app, tool_panel_configs,
latest_tool_migration_script_number )
File
"/galaxy/galaxy-beta/galaxy-dist/lib/tool_shed/util/common_util.py",
line 66, in check_for_missing_tools
return tool_shed_accessible, missing_tool_configs_dict
UnboundLocalError: local variable 'missing_tool_configs_dict' referenced
before assignment
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] negative user data usage
Hi, In case anybody is wondering, the "set_user_disk_usage.py" script corrects this. Best, Geert On 03/28/2013 11:04 AM, Geert Vandeweyer wrote: Hi, Today, I found out that one user in our local galaxy installation (the administrator user) has a negative disk usage. - Reports shows : -72780720701 bytes - Galaxy history shows: -1% Does anybody have suggestions on what might be causing this and how to solve it? There is about 660 Gb of data in the histories of that user, but it was more before. I believe it happened after some histories were deleted and there was a message of one of them being shared. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] error checking active jobs : KeyError: 'destination'
I was able to solve this by setting job state in the database from
queued and running to "failed" for two jobs that had a wrong state after
galaxy-restart.
best,
geert
On 04/02/2013 05:27 PM, Geert Vandeweyer wrote:
Hi,
I'm getting errors in the handler logs after upgrading galaxy stable
to today's release.
I specified the "dynamic" job runner as default, which returns a job
for pbs.
These errors appear as soon as the server is up and running. What
could be the issue here?
galaxy.jobs.runners ERROR 2013-04-02 17:24:23,411 Unhandled exception
checking active jobs
Traceback (most recent call last):
File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line
336, in monitor
self.check_watched_items()
File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 382,
in check_watched_items
( failures, statuses ) = self.check_all_jobs()
File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 463,
in check_all_jobs
pbs_server_name =
self.__get_pbs_server(pbs_job_state.job_destination.params)
File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 221,
in __get_pbs_server
return job_destination_params['destination'].split('@')[-1]
KeyError: 'destination'
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] error checking active jobs : KeyError: 'destination'
Hi,
I'm getting errors in the handler logs after upgrading galaxy stable to
today's release.
I specified the "dynamic" job runner as default, which returns a job for
pbs.
These errors appear as soon as the server is up and running. What could
be the issue here?
galaxy.jobs.runners ERROR 2013-04-02 17:24:23,411 Unhandled exception
checking active jobs
Traceback (most recent call last):
File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line
336, in monitor
self.check_watched_items()
File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 382,
in check_watched_items
( failures, statuses ) = self.check_all_jobs()
File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 463,
in check_all_jobs
pbs_server_name =
self.__get_pbs_server(pbs_job_state.job_destination.params)
File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 221,
in __get_pbs_server
return job_destination_params['destination'].split('@')[-1]
KeyError: 'destination'
--
Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/pub/geert-vandeweyer/26/457/726
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] negative user data usage
Hi, Today, I found out that one user in our local galaxy installation (the administrator user) has a negative disk usage. - Reports shows : -72780720701 bytes - Galaxy history shows: -1% Does anybody have suggestions on what might be causing this and how to solve it? There is about 660 Gb of data in the histories of that user, but it was more before. I believe it happened after some histories were deleted and there was a message of one of them being shared. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Workflows
Hi, I've put together some exempels for activating and using the api. You can find them here: http://geertvandeweyer.zymichost.com/index.php?page=read&id=29 Best regards, Geert vandeweyer Op 16-jan.-2013 om 02:52 heeft het volgende geschreven: > Hi >Sorry if these questions are obvious but I just don’t know how to find the > answers. > > I’m trying to get one of the API examples to work in > http://wiki.galaxyproject.org/Learn/API/Examples . > > I’ve got my API key but how do I get/find the workflow id (f2db41e1fa331b3e > in the examples). I’ve created workflow but don’t know how to access this key. > > Also in the first example it states /api/workflows However I > don’t have the “api/workflows” directory structure in my galaxy-dist, do I > need to create this? > > Thanks > Neil > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Syntax error (!) in latest pull?!
We recently ran into that issue as well. It seems that the used finally: syntax is valid from python 2.5, and we were using python 2.4. Loading a more recent version of python helped. Best, Geert Vandeweyer On 11/07/2012 04:54 AM, Scott McManus wrote: Well, lib/galaxy/datatypes/registry.py hasn't changed in over two months. Out of curiousity, I tried downloading a fresh copy of galaxy-central (without any initial configuration or database, either) and didn't encounter the same problem. I even tried messing with that line of code and didn't have any luck reproducing the same traceback. What version of python are you using? -Scott Got this in the latest pull of galaxy central... Ideas? Traceback (most recent call last): File "./scripts/set_metadata.py", line 24, in ? import galaxy.model.mapping #need to load this before we unpickle, in order to setup properties assigned by the mappers File "/mnt/ngs/analysis/svcgalaxy/galaxy-central/lib/galaxy/model/__init__.py", line 13, in ? import galaxy.datatypes.registry File "/mnt/ngs/analysis/svcgalaxy/galaxy-central/lib/galaxy/datatypes/registry.py", line 145 finally: ^ SyntaxError: invalid syntax ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] dynamically send jobs to second cluster on high load
Hi, The admin pages state that it is possible to specify multiple clusters in the universe file. Currently, we are investigating if we can couple the university HPC platform to galaxy, to handle usage peaks. It would be ideal if the job manager would check the load of the dedicated cluster (eg queue length) and send jobs to the second cluster when load is above a threshold. Does such an approach exists already, or will it become available in the near future? As far as I understand, it is now only possible to specify which jobs run on which cluster, without dynamic switching? Best regards, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] upload files by cluster with pbs-python returns empty files
Hi Nate, The datafile is really empty on the disk (size 0, checked at the "full path" etnry from the dataset info). Best regards, Geert Vandeweyer On 08/29/2012 06:05 PM, Nate Coraor wrote: On Jul 16, 2012, at 6:14 AM, Geert Vandeweyer wrote: Hi, We have set up a local instance of galaxy-dist, using pbs-python to communicate with our HPC. Everything is working great, except for the upload functionality. When I assign the upload1 to local:/// , uploading of small files through the web-site works, and upload of large files by ftp works as well. When I let upload1 to be handled by the default_cluster_job_runner ( = pbs:///), I get empty data sets. Galaxy does not report any errors however and the data state is ok. Has anybody seen this issue and solved it? Using the local:/// job handler causes a massive performance hit on the galaxy process. Best regards, Geert Vandeweyer Hi Geert, Sorry for the delayed response. If you're still having this issue, if you view the empty dataset, can you see the data it's supposed to contain? i.e. does the underlying output file for the upload tool actually contain the data, or is it really empty on disk? --nate ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Looking for recommendations: How to run galaxy workflows
Hi, I've also thought about that approach, but the problem here is that I don't find a way to request the users api key from a tool-xml. So you would have to hard-code an api key in the tool configuration. Or is there a way to request a user's api key from a galaxy-tool. That would also allow the design of tools that would clean up (delete) datasets after workflow execution. Geert On 07/19/2012 09:04 AM, Sascha Kastens wrote: Hi Geert, thanks for your reply. Unfortunately it is not feasible for us to use a script outside Galaxy. As long as no official feature for Galaxy is released I will work on a script which will execute workflows via the API and add this script as a tool into Galaxy. Cheers, Sascha Original message: Hi, In case you'd be interested: we use a script that creates interlaced data of paired end data. We run this outside galaxy on groups of samples ordered in directories at once. We then import the interlaced data into galaxy, enabling batch workflow. The first step of the workflow is a deinterlacing of the datafiles. The script is available here: http://geertvandeweyer.zymichost.com/index.php?page=read&id=27 Best Regards, Geert On 07/18/2012 02:38 PM, Sascha Kastens wrote: > > Hi Dev-Team, > > are you planning or maybe working on an update which enables the > > possiblilty to run workflows in batch mode with paired end data? > > Cheers, > > Sascha > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be <mailto:geert.vandewe...@ua.ac.be> http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Looking for recommendations: How to run galaxy workflows in batch
Hi, In case you'd be interested: we use a script that creates interlaced data of paired end data. We run this outside galaxy on groups of samples ordered in directories at once. We then import the interlaced data into galaxy, enabling batch workflow. The first step of the workflow is a deinterlacing of the datafiles. The script is available here: http://geertvandeweyer.zymichost.com/index.php?page=read&id=27 Best Regards, Geert On 07/18/2012 02:38 PM, Sascha Kastens wrote: Hi Dev-Team, are you planning or maybe working on an update which enables the possiblilty to run workflows in batch mode with paired end data? Cheers, Sascha ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] upload files by cluster with pbs-python returns empty files
Hi, We have set up a local instance of galaxy-dist, using pbs-python to communicate with our HPC. Everything is working great, except for the upload functionality. When I assign the upload1 to local:/// , uploading of small files through the web-site works, and upload of large files by ftp works as well. When I let upload1 to be handled by the default_cluster_job_runner ( = pbs:///), I get empty data sets. Galaxy does not report any errors however and the data state is ok. Has anybody seen this issue and solved it? Using the local:/// job handler causes a massive performance hit on the galaxy process. Best regards, Geert Vandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Unable to queue job, resubmitting might help
Hi Nate, I kept keeping the errors untill I restarted galaxy completely. I could still submit jobs to the HPC queue from other programs. I'm not very familiar with python, but if you have pointers on where to start to solve this, I might be able to contribute. If it would be possible to restart for example just the handlers, this might be enough. best regards, Geert On 07/12/2012 07:05 PM, Nate Coraor wrote: On Jul 12, 2012, at 3:03 AM, Geert Vandeweyer wrote: Hi, Today I ran into a cluster error on our local instance using latest galaxy-dist and torque/pbs with the python-pbs binding. Under heavy load of the galaxy process, it appears that the handler processes failed to contact the pbs-server, although the pbs_server was still up and running. after that, a lot of the following statements kept appearing in the handler.log file: galaxy.jobs.runners.pbs DEBUG 2012-07-11 17:39:06,649 (11647/12788.pbs_master_address) Skipping state check because PBS server connection failed After restarting the galaxy process (run.sh), everything worked again, with no changes to the pbs_server. Would it be possible to setup some checks for this failure? Like: - contact system admin - restart galaxy - auto retry job submission after a while as to not crash workflows. Hi Geert, It'd be useful to retry submission rather than fail. I doubt we'll get to it soon, but would welcome any submissions that did this. Is restarting Galaxy absolutely necessary, or will job submission begin to succeed again after load goes down? --nate best regards, Geert Vandeweyer -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Unable to queue job, resubmitting might help
Hi, Today I ran into a cluster error on our local instance using latest galaxy-dist and torque/pbs with the python-pbs binding. Under heavy load of the galaxy process, it appears that the handler processes failed to contact the pbs-server, although the pbs_server was still up and running. after that, a lot of the following statements kept appearing in the handler.log file: galaxy.jobs.runners.pbs DEBUG 2012-07-11 17:39:06,649 (11647/12788.pbs_master_address) Skipping state check because PBS server connection failed After restarting the galaxy process (run.sh), everything worked again, with no changes to the pbs_server. Would it be possible to setup some checks for this failure? Like: - contact system admin - restart galaxy - auto retry job submission after a while as to not crash workflows. best regards, Geert Vandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Unable to queue job, resubmitting might help
Hi, Today I ran into a cluster error on our local instance using latest galaxy-dist and torque/pbs with the python-pbs binding. Under heavy load of the galaxy process, it appears that the handler processes failed to contact the pbs-server, although the pbs_server was still up and running. after that, a lot of the following statements kept appearing in the handler.log file: galaxy.jobs.runners.pbs DEBUG 2012-07-11 17:39:06,649 (11647/12788.pbs_master_address) Skipping state check because PBS server connection failed After restarting the galaxy process (run.sh), everything worked again, with no changes to the pbs_server. Would it be possible to setup some checks for this failure? Like: - contact system admin - restart galaxy - auto retry job submission after a while as to not crash workflows. best regards, Geert Vandeweyer -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Get bai file for tool-input bam file
Hi, I'm working on a tool that will export vcf files together with bam/bai files to a long term storage location on our local installation. How can i get the path to the automatically created bai file for a provided bam file? Can it be derived from the file bam file path? Best regards, Geert Vandeweyer -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Configuring Proftpd for galaxy.
You should also set "/home/nate/galaxy_dist/database/ftp/%U" to the path were ftp-uploads will be stored. This is specified in universe_wsgi.ini as "ftp_upload_dir = /path/to/galaxy-ftp" Best regards, Geert On 06/18/2012 09:45 AM, Misharl mon wrote: Hi, Thank you very much for your reply. I have tried it and i got 504 and 504. So i must replace in this line: LookupGalaxyUser SELECT"email,password,'504','504','/home/nate/galaxy_dist/database/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U'" Best regards Mish Date: Fri, 15 Jun 2012 15:55:02 +0200 From: geert.vandewey...@ua.ac.be To: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Configuring Proftpd for galaxy. 512 is the unix UID (userid) and GID (groupid) of the user running the galaxy daemon. This should be adapted to meet your situation. in our case it is 1009 and 100 for example. Look it up using : $> id **the username running galaxy** best regards Geert Vandeweyer On 06/15/2012 03:35 PM, Misharl mon wrote: Hi everybody, I'm configuring Proftpd so that it can work with Galaxy installed in a cluster. I don't understand why for the UID and GID, it has been chosen the number "512"? # Define a custom query for lookup that returns a passwd-like entry. UID and GID should match your Galaxy user. SQLUserInfo custom:/LookupGalaxyUser138.102.49.9 SQLNamedQueryLookupGalaxyUser SELECT"email,password,'512','512','/home/nate/galaxy_dist/database/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U'" Because, when a galaxy user tries to connect via filezilla to upload files, the Proftpd server 's logs shows:chdir (/work/...): Permission denied . I think i have a problem with the UID and GID which doesn't match somehow? Thanks in advance to all. Mish --> ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Configuring Proftpd for galaxy.
512 is the unix UID (userid) and GID (groupid) of the user running the galaxy daemon. This should be adapted to meet your situation. in our case it is 1009 and 100 for example. Look it up using : $> id **the username running galaxy** best regards Geert Vandeweyer On 06/15/2012 03:35 PM, Misharl mon wrote: Hi everybody, I'm configuring Proftpd so that it can work with Galaxy installed in a cluster. I don't understand why for the UID and GID, it has been chosen the number "512"? # Define a custom query for lookup that returns a passwd-like entry. UID and GID should match your Galaxy user. SQLUserInfo custom:/LookupGalaxyUser SQLNamedQueryLookupGalaxyUser SELECT"email,password,'512','512','/home/nate/galaxy_dist/database/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U'" Because, when a galaxy user tries to connect via filezilla to upload files, the Proftpd server 's logs shows:chdir (/work/...): Permission denied . I think i have a problem with the UID and GID which doesn't match somehow? Thanks in advance to all. Mish --> ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Problem -> submit job with PBS Torque
Hi, I'm using pbs/torque with galaxy, without any issues. However, I'm not using the drmaa interface, but the pbs-egg. Have you tried that one? See: http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster#pbs_python_egg after scrambling, I just set the following in universe: start_job_runners = pbs default_cluster_job_runner = pbs:/// and optionally per tool configurations: bowtie_wrapper = pbs:///batch/-l nodes=1:ppn=8,mem=2700mb/ bowtie_color_wrapper = pbs:///batch/-l nodes=1:ppn=8,mem=2700mb/ bowtie2 = pbs:///batch/-l nodes=1:ppn=8,mem=2700mb/ tophat = pbs:///batch/-l nodes=1:ppn=8,mem=2700mb/ bwa_wrapper = pbs:///batch/-l nodes=1:ppn=8,mem=3500mb/ I did not set any drmaa-settings ! I also made sure that the galaxy-user was able to submit jobs from the commandline using qsub. geert On 06/01/2012 09:52 AM, Marc Bras wrote: Hi !! I've got a problem with PBS when I want to submit a job to our cluster with Galaxy. Every time, PBS says me /*"pbs_submit failed, PBS error 15023: Bad user - no password entry".*/ Indeed, Galaxy uses my email address (used in Galaxy) as user by default ! But, before, when I used SGE on my previous lab, Galaxy used the galaxy owner user and not my email... Is it possible to set the owner user as previous Galaxy version ? (I found the */"drmaa_external_*"/* session in universe but I don't really understand how to configure this part...). Thanks a lot, Best Regards BRAS Marc ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] BWA Problem
Hi, Have you created the index files using the exact same version of bwa? I've had segfaults when using references indexed on a different computer running a different version. Geert On 05/28/2012 08:54 PM, CHEBBI Mohamed Amine wrote: Hi ! I have a problem when i execute BWA from my local galaxy instance. I have this error message : An error occurred running this job:/BWA Version: 0.6.1-r104 Error indexing reference sequence. [bwa_index] Pack FASTA... 2.35 sec [bwa_index] Construct BWT for the packed sequence... Segmentation fault/ / / Could anyone help me to fix it thanks ! Regards Amine ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Batch limit on Wokflows
I'm looking for a workflow of : Forward reads.fastq => > QC => map => etc Reverse reads.fastq => and would like to specify pairs of files (from MiSeq) in the batch workflow interface. I'm currently looking into interlacing the files outside of galaxy before uploading. Then I can de-interlace as the first step, but this is redundant, as we get seperate files to start. Best regards, Geert On 05/25/2012 02:12 PM, Dannon Baker wrote: Batch mode is currently only allowed for a single input dataset step, so the others are intentionally disabled upon toggling the first one on. We do, however, plan to eventually expand this to allow for pairwise/etc input sets. Out of curiosity, which behavior is it that you're looking for? -Dannon On May 25, 2012, at 8:03 AM, Geert Vandeweyer wrote: Is it possible to use multiple 'input dataset' steps in a single workflow in batch? If I have two, I can only select batch input on one of them. If one is open, the other toggle button does not respond. Thanks, Geert On 02/14/2012 09:55 PM, Petit III, Robert A. wrote: Thats it! Consider this resolved, and total operator error. I tried using the 'Input Dataset' step, and am no longer having issues. Thanks for the help Dannon. From: Petit III, Robert A. Sent: Tuesday, February 14, 2012 3:49 PM To: galaxy-dev@lists.bx.psu.edu Subject: RE: [galaxy-dev] Batch limit on Wokflows Workflow is beginning directly with a tool. An example test workflow I've been trying is, I have 20 fasta files, and the workflow has only the FASTA-to-Tabular tool. Thanks, Robert From: Dannon Baker [dannonba...@me.com] Sent: Tuesday, February 14, 2012 3:33 PM To: Petit III, Robert A. Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Batch limit on Wokflows I hadn't had a chance to follow up again yet, but in your workflow, are you using an Input Dataset step, or do you have the workflow beginning directly with a tool? -Dannon On Feb 14, 2012, at 3:26 PM, Petit III, Robert A. wrote: Here's the latest. I have tested this on Chrome, Firefox, and IE. I've tried the latest revisions of galaxy-central (71031bf3105c) and galaxy-dist (26920e20157f) straight out of the box (hg clone, sh run.sh, go to localhost:8080). I am still running into same issue. All is well with 19 or less datasets in the history, but as soon as I add a 20th I can no longer run a workflow on multiple datasets. Are there any known issues running galaxy on Ubuntu 10.04? This e-mail message (including any attachments) is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message (including any attachments) is strictly prohibited. If you have received this message in error, please contact the sender by reply e-mail message and destroy all copies of the original message (including attachments). ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Batch limit on Wokflows
Is it possible to use multiple 'input dataset' steps in a single workflow in batch? If I have two, I can only select batch input on one of them. If one is open, the other toggle button does not respond. Thanks, Geert On 02/14/2012 09:55 PM, Petit III, Robert A. wrote: Thats it! Consider this resolved, and total operator error. I tried using the 'Input Dataset' step, and am no longer having issues. Thanks for the help Dannon. From: Petit III, Robert A. Sent: Tuesday, February 14, 2012 3:49 PM To: galaxy-dev@lists.bx.psu.edu Subject: RE: [galaxy-dev] Batch limit on Wokflows Workflow is beginning directly with a tool. An example test workflow I've been trying is, I have 20 fasta files, and the workflow has only the FASTA-to-Tabular tool. Thanks, Robert From: Dannon Baker [dannonba...@me.com] Sent: Tuesday, February 14, 2012 3:33 PM To: Petit III, Robert A. Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Batch limit on Wokflows I hadn't had a chance to follow up again yet, but in your workflow, are you using an Input Dataset step, or do you have the workflow beginning directly with a tool? -Dannon On Feb 14, 2012, at 3:26 PM, Petit III, Robert A. wrote: Here's the latest. I have tested this on Chrome, Firefox, and IE. I've tried the latest revisions of galaxy-central (71031bf3105c) and galaxy-dist (26920e20157f) straight out of the box (hg clone, sh run.sh, go to localhost:8080). I am still running into same issue. All is well with 19 or less datasets in the history, but as soon as I add a 20th I can no longer run a workflow on multiple datasets. Are there any known issues running galaxy on Ubuntu 10.04? This e-mail message (including any attachments) is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message (including any attachments) is strictly prohibited. If you have received this message in error, please contact the sender by reply e-mail message and destroy all copies of the original message (including attachments). ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] drmaa://native/ : native options are ignored
Hi, I'm working on an local installation of galaxy using torque with drmaa (the pbs-torque scramble failed). The torque-drmaa works fine so far, except for one issue. I'd like to specify some tool-dependent requirements from the tool_runners section in universe.wsgi.ini. For now I've been testing it with the setting below to have global native arguments : default_cluster_job_runner = drmaa://-l mem=4gb:nodes=1:ppn=6/ This should request 4gb of memory on a single node with 6 threads, but these requests are ignored. They are not listed on 'qstat -R' and more simultaneous jobs than possible are started if the requirements were taken into account. What am I missing here? Best regards, Geert Vandeweyer ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
