Re: [galaxy-dev] Cluster jobs running as real user
Hi Shane, We had enabled this feature on our cluster working with PBS but since we switched to LSF, we never managed to have it working again. Same error as yours if I correctly remember. After hours spent looking into LSF, I think we understood that LSF somehow catches the user change and rejects the job. We finally gave up. I am not sure this helps ... bw Charles On 5 Mar 2014, at 02:55, Shane Sturrock wrote: Has anyone had any success in getting cluster jobs running as the real user? I've got our server happily working with LSF via drmaa but all jobs run as the galaxy user and I would like to be able to separate out the jobs on a per user basis. I've followed the instructions at https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_Jobs_as_the_Real_User but I'm having no luck with it so was wondering if someone would know how I might debug it, like where could I find the script it is trying to execute? All I'm getting is code 18: invalid LSF job id although it does seem to be creating files in my databases/tmp directory owned by the user. Shane -- Dr Shane Sturrock shane.sturr...@biomatters.com Senior Scientist Tel: +64 (0) 9 379 5064 76 Anzac Ave Auckland New Zealand ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ = Charles Girardot Head of Genome Biology Computational Support (GBCS) European Molecular Biology Laboratory Tel: +49 6221 387 -8585 Fax: +49-(0)6221-387-8166 Email: charles.girar...@embl.de Room V205 Meyerhofstraße 1, 69117 Heidelberg, Germany = ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] System developer position available at the University of Oslo, Norway (Galaxy/HyperBrowser work)
Dear Galaxy developers, The University of Oslo is in the process of hiring a system developer that will partly work on Galaxy-based infrastructure in the context of the ELIXIR infrastructure project, and partly as a developer of the Genomic HyperBrowser and related Galaxy-based analysis software. Please see attached information. The application deadline is very soon: Sunday the 9th of March. Please apply or contact us if you are interested! Regards, Sveinung Gundersen, PhD --- Head Engineer: System developer - ELIXIR infrastructure project/ Bioinformatics core Facility --- Position as Head Engineer, SKO (1087) available for a systems developer as part of the national Elixir project, at the Oslo node, having office at Department of Informatics, University of Oslo. It is a full time position for one year, with possibilities for prolongation. Starting date as soon as possible. The position will be located at the University of Oslo, with possible visits at local Oslo user sites as per arrangement. Languages: Norwegian or English. The ELIXIR project is a part of the recently established European bioinformatics Research infrastructure project (http://www.bioinfo.no/elixir, http://bit.ly/nfr_elixir). The position will also be an integral element of The joint Helse Sør-Øst/UiO bioinformatics core facility, consisting of a group of service personnel with a broad combined bioinformatics competence. We provide services to users in the Oslo region, with an emphasis on the biomedical community. For further description of the range of services provided, please consult the Core Facility web page: http://core.rr-research.no/index.php?section=3. Job Description: The successful candidate will be expected to be part of the service team, as well as to undertake development tasks within the Elixir.no/Bioinformatics core facility infrastructure. An important such task will be to participate in further developmental team effort of the Genomic HyperBrowser (http://hyperbrowser.uio.no), a software system catering to statistical analyses of massive genomic datasets. The aim is to find statistically and biologically significant relations between experimental genomic datasets, typically stemming from high-throughput genome sequencing. This system has been developed in Python ( 90.000 code lines), using NumPy for efficient vector calculations and Rpy2 for connecting to the R statistical platform. The HyperBrowser system has been used in a range of bioinformatics and biomedical research projects, both nationally and internationally, and is also the basis for a growing number of master projects in the Department of Informatics. The initial ELIXIR.NO infrastructure, as well as the HyperBrowser system, is based upon the Galaxy framework (http://usegalaxy.org), which is a leading framework for accessible, reproducible and transparent biomedical research. Setup, tool development and some maintenance of Galaxy installations will preferably also be a part of the position. Qualifications: We are seeking a candidate with a Baceholor level degree (or higher) within computer science and with an interest to work in bioinformatics. Competence within biology or bioinformatics is a bonus, but not a requirement for the position. Prior learning relevant for the particular tasks can compensate education requirement. We are a looking for a candidate with solid experience in Python or other high-level programming languages, furthermore competence/experience in the following areas is desirable: • Algorithms • System architecture/design patterns • Agile/extreme programming • Statistics • Linux Salary: Pay grade 52 – 60 (NOK 435 100 – 499 200 per year) (depending on competence) Application should contain: • Application letter • CV (complete overview of education, work experience and academic work • Copies of educational certificates, transcript of records and letters of recommendation • Names and contact details of 2-3 references (name, relation to candidate and telephone number) Foreign applicants are advised to attach an explanation of their University's grading system. Please remember that all documents should be in English or a Scandinavian language. In accordance with the University of Oslo's equal opportunities policy, we invite applications from all interested individuals regardless of gender or ethnicity. UiO has an agreement for all employees, aiming to secure rights to research results a.o Region: Oslo Job type: Contract Working hours: Full-time Working days: Day Application deadline: March 9, 2014 Location: Blindern Reference number: 2014/1036 Home page: http://www.ifi.uio.no Contacts: • Head Engineer Sveinung Gundersen, E-mail: sveinung.gundersen (at) medisin.uio.no, Telephone: +47 22840862 • Professor II Eivind Hovig, E-mail: ehovig (at) ifi.uio.no --- More information about the
[galaxy-dev] Galaxy hack-a-thon at Michigan State HPCC
Hi Galaxy devs, We're deploying Galaxy on the Michigan State HPCC and doing it as a hack-a-thon today. Our authentication is Shibboleth and our scheduler is Torque/Moab, and we're setting it up with 'run as user' with a shared file space /mnt/research/galaxy To set up we've installed base images, set up firewall rules and set up the authentication for Galaxy We're documenting and communicating with a HipChat channel www.hipchat.com/gOY4xLgfb that anyone can join. and some live Tweets #msugalaxy If anyone wants to hop in the room during the day, we'd be happy for comments, feedback or just general camaraderie. We're already really grateful for the Galaxy docs and dev community, and we probably will have some questions that we'll post to the list during the day as well. Hopefully by the end we'll have some useful perspective for getting Galaxy up on an HPCC system like ours that we can share with others too. We're excited to get Galaxy up and running on our cluster! Best, -Tracy Tracy K. Teal tkt...@msu.edu Research Specialist Institute for Cyber Enabled Research Michigan State University ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] FTP password and web interface password
Hello, does anybody have any idea of what I can do to fix the problem? Maybe an update is required? I currently use the changeset: 11219:5c789ab4144a thanks Yec'han Dr. Yec'han LAIZET Ingenieur Bioinformatique Tel: +33 (0)5 57 12 27 75 _ INRA-UMR BIOGECO 1202 Equipe Genetique 69 route d'Arcachon 33612 CESTAS Le 18/02/2014 08:39, Yec'han Laizet a écrit : Hi Bjoern, I indeed followed the wiki tutorial to set up my FTP service some time ago. It seems, as you suggest, that newly created users cannot connect by SFTP. I followed the fix by putting the use_pbkdf2 = False line just below the [app:main] and restarted the galaxy server. I have reseted a newly created user password but it still does not work. Yec'han Dr. Yec'han LAIZET Ingenieur Bioinformatique Tel: +33 (0)5 57 12 27 75 _ INRA-UMR BIOGECO 1202 Equipe Genetique 69 route d'Arcachon 33612 CESTAS Le 17/02/2014 18:12, Björn Grüning a écrit : Hi Yec'han, please have a look at https://wiki.galaxyproject.org/Admin/Config/Upload%20via%20FTP If you are running postgres and you only newly created users can't access the server its probably due to encryption changes. Set use_pbkdf2 = False and reset all passwort for new users. Cheers, Bjoern Am 17.02.2014 17:27, schrieb Yec'han Laizet: Hello, I set up a FTP server with SFTP support on my galaxy instance. I have a strange behavior when trying to connect by SFTP. Some users cannot authentify (access denied) whereas other can. As all users can login to the web interface with their credentials, I wanted to check if the length of the password could be the problem with SFTP. To do so, I went to the admin interface to reset the password of a user who could connect by SFTP. Then, this user can connect to the galaxy web interface with the new password but not by SFTP ; if we use the old password, it still works for SFTP authenfication as if both passwords are independent. Could you help me to solve the problem? Yec'han Dr. Yec'han LAIZET Ingenieur Bioinformatique Tel: +33 (0)5 57 12 27 75 _ INRA-UMR BIOGECO 1202 Equipe Genetique 69 route d'Arcachon 33612 CESTAS ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] read_bits64 error when loading large genomes/data into trackster
Would it be possible that you have an old copy of the bx-python egg? You should have bx-python 0.7.1 If you check your eggs directory and you see version 0.7.1, then there may be something wrong with bx-python. In this case, please upload your build and bam to our public server and try again; if it fails on our public server, share the datasets with me and I'll take a look. Thanks, J. -- Jeremy Goecks Assistant Professor, Computational Biology Institute George Washington University On Mar 4, 2014, at 5:14 PM, Raj Ayyampalayam ra...@uga.edu wrote: Hello, I am trying to visualize a large genome (Large number of scaffolds) and a large (bam file) in trackster on our local galaxy instance (running release_2014.02.10). When ever I try to do the above I see the following error in the logs: File bbi_file.pyx, line 215, in bx.bbi.bbi_file.BBIFile.query (lib/bx/bbi/bbi_file.c:5596) File bbi_file.pyx, line 222, in bx.bbi.bbi_file.BBIFile.query (lib/bx/bbi/bbi_file.c:5210) File bbi_file.pyx, line 183, in bx.bbi.bbi_file.BBIFile.summarize (lib/bx/bbi/bbi_file.c:4475) File bbi_file.pyx, line 248, in bx.bbi.bbi_file.BBIFile._get_chrom_id_and_size (lib/bx/bbi/bbi_file.c:5656) File bpt_file.pyx, line 76, in bx.bbi.bpt_file.BPTFile.find (lib/bx/bbi/bpt_file.c:1388) File bpt_file.pyx, line 55, in bx.bbi.bpt_file.BPTFile.r_find (lib/bx/bbi/bpt_file.c:1154) AttributeError: 'BinaryFileReader' object has no attribute 'read_bits64' Trackster works OK when I load smaller data sets. It seems that there was a fix for this in bx-python code, as per mail from Jeremy (http://dev.list.galaxyproject.org/trackster-error-for-viewing-rat-data-rn5-tp4662664p4662672.html). How do I get the fixed code into my galaxy instance? Thanks, -Raj ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How to install bowtie2 tool in galaxy
Hi Ravi, I checked and the bowtie2 tool repository had a misconfigured data table name in its tool_data_table_conf.xml.sample file that was erroneously creating a tophat2_indexes data table, whereas the tool itself uses bowtie2_indexes. Can you try to uninstall and reinstall bowtie2 from the toolshed and see if that will fix your problem? More information on ways to debug this are below (wrote email before checking the repository): You’ll need to check that the various location files are being loaded into the proper data tables, see: https://wiki.galaxyproject.org/Admin/Tools/Data%20Tables. Galaxy has 2 files that specify the link between a location file and a data table: tool_data_table_conf.xml and shed_tool_data_table_conf.xml, for distribution/locally defined data tables and shed installed data tables, respectively — these are merged together during run time for each named data table from the various specified location files. Can you check these files to confirm that you have a data table named ‘bowtie2_indexes’ defined and that one of the files referenced from there has your additions? Thanks for using Galaxy, Dan On Mar 4, 2014, at 3:21 PM, Ravi Alla ravi.a...@berkeley.edu wrote: Bjoern, This is getting frustrating. There are three places bowtie2_indices.loc file is expected. I really don't know which one is the one I should modify. galaxy-dist/tool-data/bowtie2_indices.loc galaxy-dist/tool-data/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/96d2e31a3938/bowtie2_indices.loc ../shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/96d2e31a3938/bowtie2/tool-data/bowtie2_indices.loc What is the difference between these 3 files? I changed the 2nd file to include the path to the indices and this shows up as a valid preloaded index for tophat2 but does not for bowtie2. I also changed the normal bowtie_indices.loc file at the second location and this works and I can see the bowtie1 indices for that bowtie tool. Currently only the bowtie2 indices are acting up. I wish this was easier. Thanks Ravi. On Mar 4, 2014, at 11:28 AM, Björn Grüning bjoern.gruen...@gmail.com wrote: Hi, can you check if the file tool_data_table_conf.xml contains an entry about bowtie2_indexes ... or the table shed_data_table_conf.xml ... in one of them should be one entry with bowtie2_indexes. That warning should not be there I think: galaxy.tools.parameters.dynamic_options WARNING 2014-03-04 10:14:33,335 Data table named 'bowtie2_indexes' is required by tool but not configured Hope you can figure it out, Bjoern Am 04.03.2014 19:19, schrieb Ravi Alla: Hi Bjoern, Please find the bowtie2 and bowtie .loc files. In the paster.log file I see these entries for bowtie galaxy.tools.data DEBUG 2014-03-04 10:14:11,048 Loaded tool data table 'bowtie_indexes' galaxy.tools.data DEBUG 2014-03-04 10:14:11,051 Loading another instance of data table 'bowtie_indexes', attempting to merge content. galaxy.tools.parameters.dynamic_options WARNING 2014-03-04 10:14:33,335 Data table named 'bowtie2_indexes' is required by tool but not configured And ls -l /global/referenceData/databases/bowtie2/hg19 is drwxr-xr-x 2 ralla cgrl 2048 Sep 10 11:14 . drwxr-xr-x 3 ralla cgrl 2048 Sep 10 11:06 .. -rw-rw-r-- 1 ralla cgrl 960018873 May 2 2012 hg19.1.bt2 -rw-rw-r-- 1 ralla cgrl 716863572 May 2 2012 hg19.2.bt2 -rw-rw-r-- 1 ralla cgrl 3833 May 2 2012 hg19.3.bt2 -rw-rw-r-- 1 ralla cgrl 716863565 May 2 2012 hg19.4.bt2 -rw-rw-r-- 1 ralla cgrl 960018873 May 2 2012 hg19.rev.1.bt2 -rw-rw-r-- 1 ralla cgrl 716863572 May 2 2012 hg19.rev.2.bt2 and ls -l /global/referenceData/databases/bowtie/hg19 is drwxr-xr-x 2 ralla cgrl 2048 Mar 4 10:07 . drwxr-xr-x 3 ralla cgrl 2048 Nov 30 2012 .. -rw-rw-r-- 1 ralla cgrl 821725563 Nov 13 2009 hg19.1.ebwt -rw-rw-r-- 1 ralla cgrl 357667968 Nov 13 2009 hg19.2.ebwt -rw-rw-r-- 1 ralla cgrl 3284 Nov 13 2009 hg19.3.ebwt -rw-rw-r-- 1 ralla cgrl 715335926 Nov 13 2009 hg19.4.ebwt lrwxrwxrwx 1 ralla cgrl45 Aug 21 2013 hg19.fa - /global/referenceData/genomes/hs/hg19/hg19.fa -rw-rw-r-- 1 ralla cgrl 821725563 Nov 13 2009 hg19.rev.1.ebwt -rw-rw-r-- 1 ralla cgrl 357667968 Nov 13 2009 hg19.rev.2.ebwt Thanks for looking into this. Ravi. On Mar 4, 2014, at 9:13 AM, Björn Grüning bjoern.gruen...@gmail.com wrote: Hi Ravi, can you attach the loc file, do you see anything in the Galaxy log files about bowtie2, try grepping for bowtie2. Cheers, Bjoern Am 04.03.2014 18:07, schrieb Ravi Alla: Hi Bjoern, Thank you for your clarifications. I have tried installing bowtie2 using both variations, once with both wrapper and dependencies and once with only wrapper (since I have dependencies installed on the system). In either case even after editing the .loc file I cannot see the indices in galaxy. I tried the same edits to .loc file with bwa and the indices show
[galaxy-dev] Uploaded files not previewing at random
Greetings,
Since yesterday, I have been experiencing an unusual problem with Uploading
files.
Seemingly at random, an uploaded will finish successfully (history item in
green) but no preview is available. Under the title of the item, the keyword
empty is found and preview box says no peek. Such an item cannot be
previewed or downloaded; attempts to do so result in this error displayed in
the browser:
OK
The requested URL /datasets/{API ID}/display/ was not found on this server.
Apache/2.2.25 (Unix) mod_ssl/2.2.25 OpenSSL/1.0.0-fips DAV/2 Server at our
galaxy URL Port 80
However, the Full Path of the item is still valid and, when viewed in command
line, shows all the data. Furthermore, the item can still be used in tools and
so forth.
I and other users have tried multiple attempts at uploading the exact, same
file (a small FASTA) and the results are the same: sometimes this issue is
present, other times it is not. There seems to be no pattern nor are there any
errors in the paster.log.
--
Ravi Sanka
ICS – Sr. Bioinformatics Engineer
J. Craig Venter Institute
301-795-7743
--
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Unable to share data
Greetings, I and another individual are attempting to test Galaxy's ability to share data. We have attempted to share histories and workflows, selecting only each other's user email address for people to allow sharing with. All cases have failed, we go to the Shared Data section in both accounts, but nothing appears. Is there some kind of parameter in a config file that must be set to enable sharing? -- Ravi Sanka ICS – Sr. Bioinformatics Engineer J. Craig Venter Institute 301-795-7743 -- ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Unable to share data
Hi Ravi, To view histories shared with you, click on the history options (gear icon in the right) and click “View Histories Shared with Me”. To view shared workflows, go into the workflow interface and scroll down to the “Workflows shared with you by others”. Please let us know if that doesn’t help. We should probably add an option under Shared Data “Items shared with me” or similar though. Thanks for using Galaxy, Dan On Mar 5, 2014, at 11:49 AM, Sanka, Ravi rsa...@jcvi.org wrote: Greetings, I and another individual are attempting to test Galaxy's ability to share data. We have attempted to share histories and workflows, selecting only each other's user email address for people to allow sharing with. All cases have failed, we go to the Shared Data section in both accounts, but nothing appears. Is there some kind of parameter in a config file that must be set to enable sharing? -- Ravi Sanka ICS – Sr. Bioinformatics Engineer J. Craig Venter Institute 301-795-7743 -- ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Unable to share data
Hi Dan, Yes, we see them now. Thank you. -- Ravi Sanka ICS – Sr. Bioinformatics Engineer J. Craig Venter Institute 301-795-7743 -- From: Daniel Blankenberg d...@bx.psu.edumailto:d...@bx.psu.edu Date: Wednesday, March 5, 2014 11:56 AM To: Ravi Sanka rsa...@jcvi.orgmailto:rsa...@jcvi.org Cc: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Unable to share data Hi Ravi, To view histories shared with you, click on the history options (gear icon in the right) and click “View Histories Shared with Me”. To view shared workflows, go into the workflow interface and scroll down to the “Workflows shared with you by others”. Please let us know if that doesn’t help. We should probably add an option under Shared Data “Items shared with me” or similar though. Thanks for using Galaxy, Dan On Mar 5, 2014, at 11:49 AM, Sanka, Ravi rsa...@jcvi.orgmailto:rsa...@jcvi.org wrote: Greetings, I and another individual are attempting to test Galaxy's ability to share data. We have attempted to share histories and workflows, selecting only each other's user email address for people to allow sharing with. All cases have failed, we go to the Shared Data section in both accounts, but nothing appears. Is there some kind of parameter in a config file that must be set to enable sharing? -- Ravi Sanka ICS – Sr. Bioinformatics Engineer J. Craig Venter Institute 301-795-7743 -- ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How to install bowtie2 tool in galaxy
Hi Ravi, The directory structure for the installation of ToolShed tools changed, which is why you have three directories. You perhaps had bowtie2 installed once before, then reinstalled (without completely removing the older version and associated data)? Or updated without resetting the metadata? In either case, the ../shed_tools directory is the new one. Having this as the path in your .xml configuration files (as Bjoern suggested earlier) and moving all contents data to be under the same location will be the simplest global solution ongoing. Links near the end of my reply can help explain how-to. For the specific reason why bowtie2 indices are not working: I noticed that the reference genome .fa file is not linked from the directory containing the indexes. This is required. Adding it in, the same way that you did for the bowtie2 indexes, into this dir: /global/referenceData/databases/bowtie2/hg19 will probably solve that part of the problem. I didn't see this posted - but I apologize if I am duplicating advice already given. I also tend to advise keeping all data under the same master data directory - all indexes and sequence data - as symbolic links to additional file system paths that are unknown to the 'galaxy user' cause a different set of problems. However, that said, this doesn't seem to be an issue in your specific case: if the bowtie2 indexes are functioning correctly - then environment is set up so that the other dir hierarchy where the .fa files are kept must included in the 'galaxy' user's ENV. Symbolic links that go outside of the local dir structure are known to cause problems unless the ENV config is carefully set up, and to my knowledge are best avoided entirely for certain uses such as upload by file path into libraries. For reference: NGS data set-up is described in this wiki - including expected content for each type of index: https://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup For examples, you can rsync a genome or two and examine the contents. Or, our /location dir and have a look at the .loc files. https://wiki.galaxyproject.org/Admin/DataIntegration Tool Shed help (very detailed): https://wiki.galaxyproject.org/Tool%20Shed In particular, if you had previously installed repositories (this is not clear, just suspected from the duplications), updating the Metadata with certain distribution updates can be very important. This has been necessary for the last few releases to update to changes. The News Brief noted this, and included a link to this wiki page. Also see the Related Pages lower down on the wiki. https://wiki.galaxyproject.org/ResettingMetadataForInstalledRepositories This may also be useful: https://wiki.galaxyproject.org/RepairingInstalledRepositories Hopefully you have sorted most of this out by now, or this helps! Jen Galaxy team On 3/4/14 12:21 PM, Ravi Alla wrote: Bjoern, This is getting frustrating. There are three places bowtie2_indices.loc file is expected. I really don't know which one is the one I should modify. galaxy-dist/tool-data/bowtie2_indices.loc galaxy-dist/tool-data/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/96d2e31a3938/bowtie2_indices.loc http://bx.psu.edu/repos/devteam/bowtie2/96d2e31a3938/bowtie2_indices.loc ../shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/96d2e31a3938/bowtie2/tool-data/bowtie2_indices.loc http://bx.psu.edu/repos/devteam/bowtie2/96d2e31a3938/bowtie2/tool-data/bowtie2_indices.loc What is the difference between these 3 files? I changed the 2nd file to include the path to the indices and this shows up as a valid preloaded index for tophat2 but does not for bowtie2. I also changed the normal bowtie_indices.loc file at the second location and this works and I can see the bowtie1 indices for that bowtie tool. Currently only the bowtie2 indices are acting up. I wish this was easier. Thanks Ravi. On Mar 4, 2014, at 11:28 AM, Björn Grüning bjoern.gruen...@gmail.com mailto:bjoern.gruen...@gmail.com wrote: Hi, can you check if the file tool_data_table_conf.xml contains an entry about bowtie2_indexes ... or the table shed_data_table_conf.xml ... in one of them should be one entry with bowtie2_indexes. That warning should not be there I think: galaxy.tools.parameters.dynamic_options WARNING 2014-03-04 10:14:33,335 Data table named 'bowtie2_indexes' is required by tool but not configured Hope you can figure it out, Bjoern Am 04.03.2014 19:19, schrieb Ravi Alla: Hi Bjoern, Please find the bowtie2 and bowtie .loc files. In the paster.log file I see these entries for bowtie galaxy.tools.data DEBUG 2014-03-04 10:14:11,048 Loaded tool data table 'bowtie_indexes' galaxy.tools.data DEBUG 2014-03-04 10:14:11,051 Loading another instance of data table 'bowtie_indexes', attempting to merge content. galaxy.tools.parameters.dynamic_options WARNING 2014-03-04 10:14:33,335 Data table named 'bowtie2_indexes' is required by tool but not configured And ls -l
Re: [galaxy-dev] Cluster jobs running as real user
Hi Charles, That actually helps. I don't have any choice in running LSF but knowing that it didn't work because LSF is rejecting it helps when I talk to my boss about why I couldn't do it. Shane On 5/03/2014, at 10:28 pm, Charles Girardot charles.girar...@embl.de wrote: Hi Shane, We had enabled this feature on our cluster working with PBS but since we switched to LSF, we never managed to have it working again. Same error as yours if I correctly remember. After hours spent looking into LSF, I think we understood that LSF somehow catches the user change and rejects the job. We finally gave up. I am not sure this helps ... bw Charles On 5 Mar 2014, at 02:55, Shane Sturrock wrote: Has anyone had any success in getting cluster jobs running as the real user? I've got our server happily working with LSF via drmaa but all jobs run as the galaxy user and I would like to be able to separate out the jobs on a per user basis. I've followed the instructions at https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_Jobs_as_the_Real_User but I'm having no luck with it so was wondering if someone would know how I might debug it, like where could I find the script it is trying to execute? All I'm getting is code 18: invalid LSF job id although it does seem to be creating files in my databases/tmp directory owned by the user. Shane -- Dr Shane Sturrock shane.sturr...@biomatters.com Senior Scientist Tel: +64 (0) 9 379 5064 76 Anzac Ave Auckland New Zealand ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ = Charles Girardot Head of Genome Biology Computational Support (GBCS) European Molecular Biology Laboratory Tel: +49 6221 387 -8585 Fax: +49-(0)6221-387-8166 Email: charles.girar...@embl.de Room V205 Meyerhofstraße 1, 69117 Heidelberg, Germany = -- Dr Shane Sturrock shane.sturr...@biomatters.com Senior Scientist Tel: +64 (0) 9 379 5064 76 Anzac Ave Auckland New Zealand ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] FTP password and web interface password
Hi Yec'han, Could you check that the 'password' column for the user in question in the 'galaxy_user' table in the database does not begin with $PBKDF2$? If not, do you have any debug logs from the FTP session and server that provide details on the failure? --nate On Wed, Mar 5, 2014 at 10:36 AM, Yec'han Laizet ylai...@pierroton.inra.fr wrote: Hello, does anybody have any idea of what I can do to fix the problem? Maybe an update is required? I currently use the changeset: 11219:5c789ab4144a thanks Yec'han Dr. Yec'han LAIZET Ingenieur Bioinformatique Tel: +33 (0)5 57 12 27 75 _ INRA-UMR BIOGECO 1202 Equipe Genetique 69 route d'Arcachon 33612 CESTAS Le 18/02/2014 08:39, Yec'han Laizet a écrit : Hi Bjoern, I indeed followed the wiki tutorial to set up my FTP service some time ago. It seems, as you suggest, that newly created users cannot connect by SFTP. I followed the fix by putting the use_pbkdf2 = False line just below the [app:main] and restarted the galaxy server. I have reseted a newly created user password but it still does not work. Yec'han Dr. Yec'han LAIZET Ingenieur Bioinformatique Tel: +33 (0)5 57 12 27 75 _ INRA-UMR BIOGECO 1202 Equipe Genetique 69 route d'Arcachon 33612 CESTAS Le 17/02/2014 18:12, Björn Grüning a écrit : Hi Yec'han, please have a look at https://wiki.galaxyproject.org/Admin/Config/Upload%20via%20FTP If you are running postgres and you only newly created users can't access the server its probably due to encryption changes. Set use_pbkdf2 = False and reset all passwort for new users. Cheers, Bjoern Am 17.02.2014 17:27, schrieb Yec'han Laizet: Hello, I set up a FTP server with SFTP support on my galaxy instance. I have a strange behavior when trying to connect by SFTP. Some users cannot authentify (access denied) whereas other can. As all users can login to the web interface with their credentials, I wanted to check if the length of the password could be the problem with SFTP. To do so, I went to the admin interface to reset the password of a user who could connect by SFTP. Then, this user can connect to the galaxy web interface with the new password but not by SFTP ; if we use the old password, it still works for SFTP authenfication as if both passwords are independent. Could you help me to solve the problem? Yec'han Dr. Yec'han LAIZET Ingenieur Bioinformatique Tel: +33 (0)5 57 12 27 75 _ INRA-UMR BIOGECO 1202 Equipe Genetique 69 route d'Arcachon 33612 CESTAS ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Uploaded files not previewing at random
Hi Ravi,
Are your datasets stored on a shared filesystem (e.g. NFS)? If yes,
you could try setting `retry_job_output_collection` to something 0
in universe_wsgi.ini? NFS attribute caching can produce behavior like
this.
--nate
On Wed, Mar 5, 2014 at 11:46 AM, Sanka, Ravi rsa...@jcvi.org wrote:
Greetings,
Since yesterday, I have been experiencing an unusual problem with Uploading
files.
Seemingly at random, an uploaded will finish successfully (history item in
green) but no preview is available. Under the title of the item, the keyword
empty is found and preview box says no peek. Such an item cannot be
previewed or downloaded; attempts to do so result in this error displayed in
the browser:
OK
The requested URL /datasets/{API ID}/display/ was not found on this server.
Apache/2.2.25 (Unix) mod_ssl/2.2.25 OpenSSL/1.0.0-fips DAV/2 Server at our
galaxy URL Port 80
However, the Full Path of the item is still valid and, when viewed in
command line, shows all the data. Furthermore, the item can still be used in
tools and so forth.
I and other users have tried multiple attempts at uploading the exact, same
file (a small FASTA) and the results are the same: sometimes this issue is
present, other times it is not. There seems to be no pattern nor are there
any errors in the paster.log.
--
Ravi Sanka
ICS - Sr. Bioinformatics Engineer
J. Craig Venter Institute
301-795-7743
--
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How to install bowtie2 tool in galaxy
Hi Jennifer, Thank you for this information. I was able to troubleshoot the bowtie2_indices. Like Bjoern said the bowtie2 tool needed to be reinstalled because the tool_table does not load the correct indices. I am still trying to wrap my head around the different toolsheds. The main tool shed resides in the galaxy-dist folder and uses tool_conf.xml to load tools into the side bar and tool_data_table_conf.xml to load indices (and other data) for the default tools that come with galaxy. The shed tools reside in the ../shed_tools/ directory and use shed_tools_conf.xml to load tools into the side bar and the shed_tools_data_table_conf.xml to load indices. The shed_tool xml is setup in the universe.ini file. The .loc files for default main tools reside in galaxy-dist/tool-data and the .loc files for the shed_tools reside in ../shed_tools/.../tool-name/tool_id/tool-name/ And when I uninstall tools and reinstall I would have to update the metadata for that tool. If ../shed_tools dir is not present then shed-tools get installed under galaxy-dist/tool-data/toolshed.g2.bx.psu.edu/ And it is always better to install tools as wrappers + dependencies when possible. Am I on the right track with this tool organization? Thanks Ravi On Mar 5, 2014, at 11:06 AM, Jennifer Jackson j...@bx.psu.edu wrote: Hi Ravi, The directory structure for the installation of ToolShed tools changed, which is why you have three directories. You perhaps had bowtie2 installed once before, then reinstalled (without completely removing the older version and associated data)? Or updated without resetting the metadata? In either case, the ../shed_tools directory is the new one. Having this as the path in your .xml configuration files (as Bjoern suggested earlier) and moving all contents data to be under the same location will be the simplest global solution ongoing. Links near the end of my reply can help explain how-to. For the specific reason why bowtie2 indices are not working: I noticed that the reference genome .fa file is not linked from the directory containing the indexes. This is required. Adding it in, the same way that you did for the bowtie2 indexes, into this dir: /global/referenceData/databases/bowtie2/hg19 will probably solve that part of the problem. I didn't see this posted - but I apologize if I am duplicating advice already given. I also tend to advise keeping all data under the same master data directory - all indexes and sequence data - as symbolic links to additional file system paths that are unknown to the 'galaxy user' cause a different set of problems. However, that said, this doesn't seem to be an issue in your specific case: if the bowtie2 indexes are functioning correctly - then environment is set up so that the other dir hierarchy where the .fa files are kept must included in the 'galaxy' user's ENV. Symbolic links that go outside of the local dir structure are known to cause problems unless the ENV config is carefully set up, and to my knowledge are best avoided entirely for certain uses such as upload by file path into libraries. For reference: NGS data set-up is described in this wiki - including expected content for each type of index: https://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup For examples, you can rsync a genome or two and examine the contents. Or, our /location dir and have a look at the .loc files. https://wiki.galaxyproject.org/Admin/DataIntegration Tool Shed help (very detailed): https://wiki.galaxyproject.org/Tool%20Shed In particular, if you had previously installed repositories (this is not clear, just suspected from the duplications), updating the Metadata with certain distribution updates can be very important. This has been necessary for the last few releases to update to changes. The News Brief noted this, and included a link to this wiki page. Also see the Related Pages lower down on the wiki. https://wiki.galaxyproject.org/ResettingMetadataForInstalledRepositories This may also be useful: https://wiki.galaxyproject.org/RepairingInstalledRepositories Hopefully you have sorted most of this out by now, or this helps! Jen Galaxy team On 3/4/14 12:21 PM, Ravi Alla wrote: Bjoern, This is getting frustrating. There are three places bowtie2_indices.loc file is expected. I really don't know which one is the one I should modify. galaxy-dist/tool-data/bowtie2_indices.loc galaxy-dist/tool-data/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/96d2e31a3938/bowtie2_indices.loc ../shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/96d2e31a3938/bowtie2/tool-data/bowtie2_indices.loc What is the difference between these 3 files? I changed the 2nd file to include the path to the indices and this shows up as a valid preloaded index for tophat2 but does not for bowtie2. I also changed the normal bowtie_indices.loc file at the second location and this works
[galaxy-dev] Galaxy database size and location
Hi fellow galaxy devs, I am trying to understand how to implement the galaxy database and get an idea of how big it could get. Currently we are running galaxy on a webserver, and want to have the postgresql db on locally mounted partition and not on an NFS partition. This limits us to around 100GB of storage for the db. We will create data libraries for users to load their data without copying to galaxy, so input files won't be duplicated. Is there anything we can do about the output files? Do these files need to end up in the database or can we put them on the NFS partition somewhere with the db holding information about their location? I noticed that on a routine small analysis I could easily have 20GB or more of output files and history and all this is in the database. If output files and history files are written to the database, are they cleaned up daily to avoid storage issues? Please advise. Thanks Ravi Alla ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Workflows with Conditionals and Loops
Is there a way to have conditionals and loops for the various steps in a Galaxy workflow? For example, a conditional might be used to branch to various steps based on the previous steps. Also, a step may need to be repeated until it produces a particular output. Here is an actual use case: We are invoking a web service in one of our workflows. The first step returns a unique identifier that you need to use later in order to get the actual results (since the web service operation is asynchronous). Later, using another operation, you can check on the status of the job being performed. Sometimes, this operation/step will inform us that the results are not ready. We would like to be able to loop on that second operation/step until it says the job is done, and then move on to a third operation/step to actually get the results. Any ideas? Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Department Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
